Export a subset of a database table to reproduce a query

One frequently needs to export some MySQL tables along with certain queries that will work with that table for testing. Usually one only needs a subset of records from the table which will work with the selected queries, rather then the complete table. This can be required when the table contains thousands or millions of records and we do not want to export the complete data set, as this can be time consuming during import, or maybe we only want to provide the other user with some selected records for security reasons.

Say we have a simple WordPress database with a SELECT statement like the following:

SELECT * FROM wp_posts WHERE post_status = 'draft'

If you want to only export records that will match the corresponding query above, then we can use the ‘mysqldump’ command-line utility with the ‘–where’ option.
Read More

Cherry picking SQL query fields to increase performance

The first SQL query most people learn is the SELECT statement with a query some thing like the following.

SELECT * FROM users WHERE id > 100

Nothing wrong with the syntax itself; but more often than not, what the user really wants is just a few columns from the table as shown in the PHP code below. Here we are only using three columns from the table. The usual practice of developers is to use the * modifier to get all the columns from the table as it is easier then specifying individual column names. It is quicker to select all columns from a table and worry later which to actually use in the code.
Read More

How to check when a MySQL table was last updated

I recently had to update a MySQL schema and import new data into the table. But before I could do that I needed to check that no one had updated the table during the last 7 days and no new data had been stored. As the table itself did not have any update field of itself the only other option was to look into the MySQL ‘information_schema’ database.

The ‘information_schema’ database contains a ‘tables’ table which contain the update information for each database and its tables. So all you have to do is grab that information.
Read More

Splitting large MySQL dump files

One of the frustrating things with working with MySQL is of importing large sql dump files. Either you get a ‘max execution time exceeded’ error from PHP or a ‘Max_allowed_packet_size’ from MySQL. In a recent task I needed to import a table of around a million records on a remote host, which quickly became an exercise in frustration due to various limitations on the server. SSH was of no help as changing the configuration files was restricted to the root user.

My last resort was to split the huge ‘INSERT’ statements into smaller size files. Manually doing the same is obviously time consuming and error prone; the only other solution is to write a small script to split the insert statements. This was to be a quick hack so the parsing code was to be of minimum complexity. Splitting a sql dump containing extended insert statements is somewhat complex so you need to have the dump file in a simple format – each insert statement should be on its own line as shown below.
Read More

Migrating Access MDB to MySQL

A recent task needed me to import a large amount of data from a Access MDB database to MySQL. My first choice for the job was the mdb-tools set of utilities. mdb-tools provides a set of tools and applications to export MDB data and schema to other databases such as MySQL, Oracle, Sybase, PostgreSQL, and others.

mdb-tools is available for Linux systems, and as I use Ubuntu, installation was a breeze using the package manager. To install is from the shell use the following.
Read More

Automatically create PHP classes from MySQL

Creating a database driven web application involves commonly used paradigms for data modification, which we commonly refer to as CRUD. Frameworks provides nice ORM wrappers to help the programmer. But for small projects frameworks can be an overkill. Still the programmer needs to design the basic database CRUD functionality, which can be quite tedious and repetitive. This is where auto database class generators can be helpful. If you have your database schema ready, you can automatically create the respective class wrappers for the tables.
Read More