How to automatically backup MySQL database to Dropbox

Ever wanted to backup your MySQL database to Dropbox automatically. Well, there is a nice portable solution with Dropbox Uploader. Dropbox Uploader is a BASH script which can be used to upload, download, delete, list files, move, copy and much more from Dropbox. It’s written in the BASH scripting language and only needs cURL to work. I’ve tested it on my Windows/Cygwin environment at it works like a charm.
(more…)

Read More

Export a subset of a database table to reproduce a query

One frequently needs to export some MySQL tables along with certain queries that will work with that table for testing. Usually one only needs a subset of records from the table which will work with the selected queries, rather then the complete table. This can be required when the table contains thousands or millions of records and we do not want to export the complete data set, as this can be time consuming during import, or maybe we only want to provide the other user with some selected records for security reasons.

Say we have a simple WordPress database with a SELECT statement like the following:

SELECT * FROM wp_posts WHERE post_status = 'draft'

If you want to only export records that will match the corresponding query above, then we can use the ‘mysqldump’ command-line utility with the ‘–where’ option.
(more…)

Read More

Splitting large MySQL dump files

One of the frustrating things with working with MySQL is of importing large sql dump files. Either you get a ‘max execution time exceeded’ error from PHP or a ‘Max_allowed_packet_size’ from MySQL. In a recent task I needed to import a table of around a million records on a remote host, which quickly became an exercise in frustration due to various limitations on the server. SSH was of no help as changing the configuration files was restricted to the root user.

My last resort was to split the huge ‘INSERT’ statements into smaller size files. Manually doing the same is obviously time consuming and error prone; the only other solution is to write a small script to split the insert statements. This was to be a quick hack so the parsing code was to be of minimum complexity. Splitting a sql dump containing extended insert statements is somewhat complex so you need to have the dump file in a simple format – each insert statement should be on its own line as shown below.
(more…)

Read More

Creating SQL schemas with Doctrine DBAL

A tedious task during web development is that of database schema creation. A schema containing a few tables comprising of a small set of rows is quick, while that containing dozens of tables and large numbers of columns is a tedious process. I usually resort to a small php script with some regular expression tossed in to automatically create a schema from a text file definition. But that is a little buggy as I’ve to manually add the indexes and other small things. Now that Doctrine has released a DBAL library, this will provide a nice ability to automatically create sql schemas.
(more…)

Read More

Increase your MySQL productivity with Toad

Recently my favorite MySQL Swiss Army Knife has to be Toad for MySQL. Not only does it have a plethora of tools and interesting features, it is free. A few days back I’d to compare database schemas of different versions of a application to see if some table fields had changed between versions. Comparing a database schema containing above 200 tables can be a time consuming job if you do not have the right tools. Toad makes the work easier with its schema and data compare feature, which lets you easily compare schema and also the data from two different databases. You can even synchronize the two schemas or the data therein so both the databases contain the same schema and data.

Besides these it has a nice query builder, somewhat like the one in Microsoft Access. Toad also can manage your Amazon EC2 instances with its built-in tool, which I’ve yet to try. Other than these it has the other regular features – a database explorer, table builder, database diagram creator etc. Toad is now my favorite tool for managing MySQL data.

Read More

Adminer – a fast MySQL administration tool

adminer1Before you go ahead and think that you don’t need yet another MySQL management tool, I think you should at least take a quick look at this one. Adminer is a single file (yes! everything in a single php file) tool to manage all your MySQL related tasks. All you have to do is upload the adminer.php file to your server and voila! you are ready to go. No messing around with huge software’s to manage your database. Weighing only 170KB, its easy to upload to any server. A demo of the same is included on the Adminer site.
(more…)

Read More

Selecting all except some columns in MySQL

The MySQL SELECT is a ubiquitous statement. You can select rows using the ‘*’ operator or by listing the individual column names. But many times you may require using all the columns from a table except a couple of them. For example you may have a table containing twelve columns from which you require only eleven columns. Most of us will use the ‘*’ operator in the SELECT statement rather than explicitly list all the eleven column names. But what if the column we do not require contains a large amount of BLOB data. It is inefficient to include the BLOB column in the query as it will unnecessarily increase the query load. What we want is to select all the columns except the BLOB column. We can construct the required statement using the ‘SHOW COLUMNS’ statement.
(more…)

Read More