mdb-tools is available for Linux systems, and as I use Ubuntu, installation was a breeze using the package manager. To install is from the shell use the following.

Installing mdbtools

sudo apt-get install mdbtools

The following command lines tools are available for migrating a MDB to a different database.

Just to check if the correct tables are in the Access database, we will use the ‘mdb-tables’ command which will list the table names in the database.

mdb-tables database.mdb

Exporting schema

Once we are satisfied that the appropriate tables are there we can generate a schema to be used with MySQL. For this we will use the ‘mdb-schema’ command. mdb-schema produces data definition language output for the given database. This can be further passed to another database engine (MySQL, Oracle, Sybase, PostgreSQL etc) to create a replica of the original access table format.

mdb-schema database.mdb  > schema.sql

The above will generate the schema for all the tables in the ‘database.mdb’ database and save it to the schema.sql file. If you want to output the schema for a particular table rather than all the tables, you will need to use the ‘-T’ flag. The ‘-T’ flag will restrict the schema for a particular table.

mdb-schema -T table_name database.mdb > schema.sql

Rather than saving the schema to a file we can directly import it into MySQL. The following will directly create a MySQL table using the schema from the MDB database by piping the output of mdb-schema to MySQL.

mdb-schema database.mdb | mysql -u username -p database_name

Exporting data

mdb-tools lets you export the data in two formats – CSV or SQL. To export the data in a CSV format use the following command. Note that you need to specify the table name from which the data will be exported.

mdb-export database.mdb table_name > export.csv

You can export the data as SQL ‘INSERT’ statements you need to use the ‘-I’ flag.

mdb-export -I database.mdb table_name > export.sql

But there is one problem here. All the export statements end with a newline and not a semi-colon as MySQL expects, so we need to add one. The following line shows how with the help of the ‘-R’ flag. This flag lets you specify a row delimiter.

mdb-export -I -R ';' database.mdb table_name > export.sql

This will now end all INSERT statements with a semi-colon instead of a newline. This is fine for small data, but for a huge number of INSERT statements this can be a problem as all the statements are now on one single line ( the newlines are replaced by the semi-colon). To solve this problem we will need to use ‘sed’ to insert the semi-color before the ending newline and the closing parenthesis of the INSERT statement. This is shown below.

mdb-export -I database.mdb table_name | sed -e 's/)$/)\;/' > export.sql

Using Toad for MySQL to import CSV

For my data import job I needed to import a select few columns from a particular table to MySQL. mdb-export does not allow me to select columns from a table, so I needed to find another way to import the data to MySQL. As I already had Toad MySQL installed I decided to use it.

First I exported the data from the MDB table to a CSV file and then used the Toad MySQL importer to import particular columns from the CSV file to my new MySQL database. The Toad importer lets you map columns from the CSV file to the MySQL schema as shown below.

Click to enlarge

For large files the Toad importer can take a while to complete. If you have a Toad version prior to 6.0 then upgrade it to the latest version. On my previous version of 5.x the importer used to crash before writing the imported data to the disk; the bug seems to have been resolved in the new version.

Below is a sample markup using the schema.org vocabulary and microdata.

Original.

<img src="dell-30in-lcd.jpg" />
Dell UltraSharp 30" LCD Monitor
 
87 out of 100 based on 24 user ratings
 
$1250 to $1495 from 8 sellers
 
Sellers:
<a href="save-a-lot-monitors.com/dell-30.html">
  Save A Lot Monitors - $1250</a>
<a href="jondoe-gadgets.com/dell-30.html">
  Jon Doe's Gadgets - $1350</a>

With semantic markup added.

<div itemscope itemtype="http://schema.org/Product">
  <img itemprop="image" src="dell-30in-lcd.jpg" />
  <span itemprop="name">Dell UltraSharp 30" LCD Monitor</span>
 
  <div itemprop="aggregateRating" 
    itemscope itemtype="http://schema.org/AggregateRating">
    <span itemprop="ratingValue">87</span>
    out of <span itemprop="bestRating">100</span>
    based on <span itemprop="ratingCount">24</span> user ratings
  </div>
 
  <div itemprop="offers" 
    itemscope itemtype="http://schema.org/AggregateOffer">
    <span itemprop="lowPrice">$1250</span>
    to <span itemprop="highPrice">$1495</span>
    from <span itemprop="offerCount">8</span> sellers
 
  Sellers:
  <div itemprop="offers" itemscope itemtype="http://schema.org/Offer">
    <a itemprop="url" href="save-a-lot-monitors.com/dell-30.html">
     Save A Lot Monitors - $1250</a>
  </div>
  <div itemprop="offers" itemscope itemtype="http://schema.org/Offer">
    <a itemprop="url" href="jondoe-gadgets.com/dell-30.html">
     Jon Doe's Gadgets - $1350</a>
  </div>
  ...
</div>

Although this may seem a lot of extra work and time, the efforts will pay handsomely over a period as more people adopt semantic markup and companies come up with better tools and APIs to work with the new data.

Airport			Code
Aalborg, Denmark 	AAL
Aalesund, Norway	AES
Aasiaat, Greenland 	JEG
Abadan, Iran 		ABD
Abakan, Russia		ABA
.
.

After scouting around for a while I stumbled upon a HTML list at orbitz. To make it usable in PHP I further converted the data to various formats (CSV, SQL, XML) for ease of use. The complete list of files is available below for download. The files contain codes for 3592 international airports sorted by name. This can be useful if your web application needs to convert airport names to their respective IATA codes.

Further Reading:
Airport ABCs: An Explanation of Airport Identifier Codes

A GIF file header

With the requirement that we are not allowed to use any graphics functions, to solve the problem we need to get the relevant data from the GIF file itself. Unlike a HTML or XML or other text format files, a GIF file and most other image formats are stored in a binary format. Most binary files carry a header at the top of the file which provides the meta information regarding the particular file. We can use this information to find out the type of the file and other things, such as height an width in case of a GIF file. A typical raw GIF header is shown below, using a hex editor such as WinHex.

The detailed description of the header is given below.

Offset   Length   Contents
  0      3 bytes  "GIF"
  3      3 bytes  "87a" or "89a"
  6      2 bytes  <Logical Screen Width>
  8      2 bytes  <Logical Screen Height>
 10      1 byte   bit 0:    Global Color Table Flag (GCTF)
                  bit 1..3: Color Resolution
                  bit 4:    Sort Flag to Global Color Table
                  bit 5..7: Size of Global Color Table: 2^(1+n)
 11      1 byte   <Background Color Index>
 12      1 byte   <Pixel Aspect Ratio>
 13      ? bytes  <Global Color Table(0..255 x 3 bytes) if GCTF is one>
         ? bytes  <Blocks>
         1 bytes  <Trailer> (0x3b)

So to check if the image file is a valid GIF, we need to check the starting 3 bytes of the header, which have the ‘GIF’ marker, and the next 3 bytes, which give the version number; either ’87a’ or ’89a’. It is for tasks such as these that the unpack() function is indispensable. Before we look at the solution, we will take a quick look at the unpack() function itself.

Using the unpack() function

unpack() is the complement of pack() – it transforms binary data into an associative array based on the format specified. It is somewhat along the lines of sprintf, transforming string data according to some given format. These two functions allow us to read and write buffers of binary data according to a specified format string. This easily enables a programmer to exchange data with programs written in other languages or other formats. Take a small example.

$data = unpack('C*', 'codediesel');
var_dump($data);

This will print the following, decimal codes for ‘codediesel’ :

array
  1 => int 99
  2 => int 111
  3 => int 100
  4 => int 101
  5 => int 100
  6 => int 105
  7 => int 101
  8 => int 115
  9 => int 101
  10 => int 108

In the above example the first argument is the format string and the second the actual data. The format string specifies how the data argument should be parsed. In this example the first part of the format ‘C’, specifies that we should treat the first character of the data as a unsigned byte. The next part ‘*’, tells the function to apply the previously specified format code to all the remaining characters.

Although this may look confusing, the next section provides a concrete example.

Grabbing the header data

Below is the solution to our GIF problem using the unpack() function. The is_gif() function will return true if the given file is in a GIF format.

function is_gif($image_file)
{
 
    /* Open the image file in binary mode */
    if(!$fp = fopen ($image_file, 'rb')) return 0;
 
    /* Read 20 bytes from the top of the file */
    if(!$data = fread ($fp, 20)) return 0;
 
    /* Create a format specifier */
    $header_format = 'A6version';  # Get the first 6 bytes

    /* Unpack the header data */
    $header = unpack ($header_format, $data);
 
    $ver = $header['version'];
 
    return ($ver == 'GIF87a' || $ver == 'GIF89a')? true : false;
 
}
 
/* Run our example */
echo is_gif("aboutus.gif");

The important line to note is the format specifier. The ‘A6′ characters specifies that the unpack() function is to get the the first 6 bytes of the data and interpret it as a string. The retrieved data is then stored in an associate array with the key named ‘version’.

Another example is given below. This returns some additional header data of the GIF file, including the image width and height.

function get_gif_header($image_file)
{
 
    /* Open the image file in binary mode */
    if(!$fp = fopen ($image_file, 'rb')) return 0;
 
    /* Read 20 bytes from the top of the file */
    if(!$data = fread ($fp, 20)) return 0;
 
    /* Create a format specifier */
    $header_format = 
            'A6Version/' . # Get the first 6 bytes
            'C2Width/' .   # Get the next 2 bytes
            'C2Height/' .  # Get the next 2 bytes
            'C1Flag/' .    # Get the next 1 byte
            '@11/' .       # Jump to the 12th byte
            'C1Aspect';    # Get the next 1 byte

    /* Unpack the header data */
    $header = unpack ($header_format, $data);
 
    $ver = $header['Version'];
 
    if($ver == 'GIF87a' || $ver == 'GIF89a') {
        return $header;
    } else {
        return 0;
    }
}
 
/* Run our example */
print_r(get_gif_header("aboutus.gif"));

The above example will print the following when run.

Array
(
    [Version] => GIF89a
    [Width1] => 97
    [Width2] => 0
    [Height1] => 33
    [Height2] => 0
    [Flag] => 247
    [Aspect] => 0
)

Below we will go into the details of how the format specifier works. I’ll split the format, giving the details for each character.

$header_format = 'A6Version/C2Width/C2Height/C1Flag/@11/C1Aspect';

A - Read a byte and interpret it as a string. 
    Number of bytes to read is given next
6 - Read a total of 6 bytes, starting from position 0
Version - Name of key in the associative array where data 
    retrieved by 'A6' is stored
 
/ - Start a new code format
C - Interpret the next data as an unsigned byte
2 - Read a total of 2 bytes
Width - Key in the associative array
 
/ - Start a new code format
C - Interpret the data as an unsigned byte
2 - Read a total of 2 bytes
Height- Key in the associative array
 
/ - Start a new code format
C - Interpret the data as an unsigned byte
1 - Read a total of 2 bytes
Flag - Key in the associative array
 
/ - Start a new code format
@ - Move to the byte offset specified by the following number.
      Remember that the first position in the binary string is 0. 
11 - Move to position 11
 
/ - Start a new code format
C - Interpret the data as an unsigned byte
1 - Read a total of 1 bytes
Aspect - Key in the associative array

More format options can be found here. Although I’ve only shown a small example, the pack/unpack is capable of much complex work than presented here.

.
.
<select  id="days" multiple="multiple" name="days" >
<option value="Monday">Monday</option>
<option value="Tuesday">Tuesday</option>
<option value="Wednesday">Wednesday</option>
<option value="Thursday">Thursday</option>
<option value="Friday">Friday</option>
<option value="Saturday">Saturday</option>
<option value="Sunday">Sunday</option>
 </select>
.
.

After looking at the php code I found the problem, it was sending the values for the ‘days’ field using the following line.

<input type="hidden" name="days" value="<?= $_POST['days']; ?>" />

Which I naturally changed to the following. Note the array specifier for ‘days’.

foreach($_POST['days'] as $day) 
{                
    echo '<input type="hidden" name="days[]" value="'.$day.'" />';
}

and the html to:

<select  id="days" multiple="multiple" name="days[]" >
.
.

But which surprisingly did not work. Salesforce was still showing only the last value of the multi-select. After spending some confused time over the issue, it struck me that maybe the endpoint at Salesforce was using raw http post data. The post data that we get in the PHP $_POST variable is a ‘cooked’ version of the actual data sent by the browser. I say ‘cooked’ because PHP neatly stores the retrieved data in an associative array, ready for our easy consumption. I wrongly assumed that Salesforce was using the same $_POST variable (or the variants in whatever language they use for the backend) to access post data.

I removed the field array specifier [] as given below, and the code worked.

foreach($_POST['days'] as $day) 
{                
    echo '<input type="hidden" name="days" value="'.$day.'" />';
}

The point here being that we are so accustomed to one way of doing things, that when something different comes along it takes us by surprise.

Raw vs. cooked POST data in PHP

Here is a little more on the php $_POST var. Say you have something like the following HTML:

Example 1.

.
.
<input type="text" name="number" value="1" />
<input type="text" name="number" value="2" />
<input type="text" name="number" value="3" />
 
<input type="text" name="name" value="sam" />
.
.

If you do a var_dump($_POST) on the above data, you will get the following.

array
  'number' => string '3' (length=1)
  'name' => string 'sam' (length=3)

Notice that the ‘number’ field takes the last value of the three. Because as you know, PHP converts the POST variables in an associative array. So the last value overwrites the previous values in the ‘number’ field.

Whereas if you read the POST data using the following:

$postdata = file_get_contents("php://input");
var_dump($postdata);

You will get all the fields, although in an raw encoded form and not in an array. Below is the var_dump of the same.

string 'number=1&number=2&number=3&name=sam' (length=35)

So in conclusion, if you are working with POST data, do not assume that the program on the other end will process the data the same way as you do.

Faker.js is a JavaScript implementation inspired by Benjamin Curtis’s Ruby Gem Faker and Perl’s Data::Faker that lets you generate commonly required data quickly. You can check the demo page to get an idea.

Getting Fake.js

You can download the library form github and include in your pages where you need to generate the data.

..
<script src = "Faker/Faker.js" type = "text/javascript"></script>
..

Generating fake data

The API is divided into various sections as show below:

# Name
    * findName
# Address
    * zipCode
    * city
    * streetName
    * streetAddress
    * secondaryAddress
    * ukCounty
    * ukCountry
# PhoneNumber
    * phoneNumber
# Internet
    * email
    * userName
    * domainName
    * domainWord
...

So if you need to generate a random email address or a zip code you could do it like this:

var randomEmail = Faker.Internet.email();
var randomZip = Faker.Address.zipCode();

Some sample API methods are shown below, but there are more of them which you can find here in the API section.

<script>
var longParagraph = Faker.Lorem.paragraphs();
var singleSentence = Faker.Lorem.sentence();
var companyName= Faker.Company.companyName();
var catchPhrase = Faker.Company.catchPhrase();
</script>

One of the interesting methods is ‘ Faker.Company.catchPhrase()’, which generates catchy technological phrases. A kind of a tag line generator for your new multi-million dollar company . A sample output is shown below.

Stand-alone 5th generation utilisation
Ergonomic explicit focus group
Horizontal human-resource solution
Automated contextually-based knowledge base
Distributed multi-state encoding
Extended zero administration interface
Proactive coherent productivity
Streamlined national approach
Total transitional algorithm
Stand-alone fault-tolerant moderator

The API also includes several helper methods that lets you create bulk fake data using the single API methods. For example you can generate complete user information templates cards by using a single helper method ‘Faker.Helpers.userCard()’. A sample run is shown below:

name: Blair Nikolaus
username: Fernando_Olson
email: Jacklyn_Brekke@aurore.name
address:street: Kozey Meadow
suite: Apt. 664
city: Felipastad
zipcode: 88071
 
phone: 571.540.8605 x136
website: reba.co.uk
company:name: Kuhic and Sons
catchPhrase: Integrated solution-oriented Graphical User Interface
bs: seize scalable web services

The ‘Faker.Helpers.userCard()’ method returns a nested object, so you will need to recursively traverse the object to extract the data. A simple function to do the same is given below:

<script>
// Recursively traverse a nested Javascript object and return it.
function printObj(obj)
{
  var prop, res = '';
 
  for(prop in obj)
  {
    if(typeof obj[prop] == 'object'){
        res += prop + ':' + printObj(obj[prop]) + '<br>';
    }else{
        res += prop + ': ' + obj[prop] + '<br>';
    }
  }
   return res;
}
</script>

So now you can do something like this:

...
var obj = Faker.Helpers.userCard();
alert( printObj(obj) );
...

Another Randomizer library

Club AJAX Mock Data Randomizer library is another library that lets you generate random data quickly. This library lets you generate random dates, colors, boolean values which is lacking in the Faker library. Check a simple demo using this library.

$values = array("66345:CDF", "61179:HGT", "64146:ABA",
                "68768:BNG", "68015:ZCZ", "80231:LPO",
                "64146:QWP", "68736:HHB", "86801:MNV",
                "80178:OIU", "80178:ASE", "88178:BRT",
                "801782OIU", "801378ASE", "881578BRT");

A quick solution was the following. But I’m sure that this is not the fastest, as a matter of fact it is quite slow for larger arrays.

 
<?php
 
$values = array("66345:CDF", "61179:HGT", "64146:ABA",
                "68768:BNG", "68015:ZCZ", "80231:LPO",
                "64146:QWP", "68736:HHB", "86801:MNV",
                "80178:OIU", "80178:ASE", "88178:BRT",
                "801782OIU", "801378ASE", "881578BRT");
 
 
function my_cmp($a, $b)
{
    $pieces_a = explode(":", $a);
    $pieces_b = explode(":", $b);
 
    if(!isset($pieces_a[1]) && isset($pieces_b[1])) {
        return 1;
    }
    elseif(!isset($pieces_b[1]) && isset($pieces_a[1])) {
        return -1;
    }
    elseif(!isset($pieces_a[1]) && !isset($pieces_b[1])) {
        return 0;
    }
 
    return strcasecmp($pieces_a[1], $pieces_b[1]);
}
 
usort($values, "my_cmp");
 
?>

So what will be the fastest algorithm for the same?

Data Generation

Once you connect ADG to your MySQL server and register a database, you are ready to go. Generating the test data is quite simple. First you create a project for your data generation work. Once done, you select the table and fields you want to generate data for, select the type of test data to generate – dates, urls, names etc. Set the total number of records to generate and then run the project. This will create the test data for your database project. The test data can be saved directly to the connected database or you can save it as a separate SQL script, YAML or CSV file. Below is a sample screenshot of a data generation project.

The Advanced Data Generator comes with a very useful repository of real-life like data, in order to generate meaningful test-data for the database. You can also use custom SQL scripts or Macros to generate custom string data.

In closing

Although not free, its a small price to pay for the excellent automation it provides during database creation.

Installation

XML_Beautifier being a Pear package we will use the Pear installer as below. I recommend to always use the Pear installer to download packages rather than dowloading it manually as the Pear installer automatically downloads any dependent packages.

pear install XML_Beautifier

Usage

You can beautify a xml file or a on-the-fly generated xml string. The following will nicely format a untidy xml file. The input file must be a valid xml document or the parser will flag an error.

<?php
 
require_once "XML/Beautifier.php";
 
$fmt = new XML_Beautifier();
$result = $fmt->formatFile('unformated.xml', 'beautified.xml');
 
if (PEAR::isError($result)) {
    echo $result->getMessage();
    exit();
}
 
echo "Done";
 
?>

Formatting a xml string is quite as simple.

<?php
 
require_once "XML/Beautifier.php";
 
/* Unformatted xml string */
$xml = '<rootNode><foo   bar = "pear">hello world!</foo></rootNode>';
$fmt = new XML_Beautifier();
echo $fmt->formatString($xml);
 
?>

Formatted xml string.

<rootNode>
    <foo bar="pear">hello world!</foo>
</rootNode>

If you frequently need to format xml documents than it would be better to have a command line access to the beautifier as below.

<?php
 
/** xmlformat.php
 *  usage: php xmlformat.php untidy.xml beautified.xml
 */
 
if ($argc < 3 ) {
    echo "\nUsage: $argv[0] unformatted.xml formatted.xml\n";
} else {
 
    require_once "XML/Beautifier.php";
 
    echo "Formatting {$argv[1]}...";
 
    $fmt = new XML_Beautifier();
    $result = $fmt->formatFile($argv[1], $argv[2]);
 
    if (PEAR::isError($result)) {
        echo $result->getMessage();
        exit();
    }
    echo "Done";
}
 
?>

Options

The XML_Beautifier class also accepts a array of options which you can pass to the constructor during initialization. The details of the options can be found here.

.
.
$options = array(
                    "caseFolding"       => true,
                    "caseFoldingTo"     => "uppercase",
                    "normalizeComments" => true
                );
 
$fmt = new XML_Beautifier($options);
.
.

One thing to note in closing is that the formatting of big xml documents can take some time depending on your machine configuration, so if you plan on formatting documents on a live server you need to take that into account.

Varieties of barcodes

Barcodes bascially come in two main type: linear (1-dimensional) and Matrix (2-dimensional) with each offering a variety of formats, depending on your application purpose. The barcodes we usually see on books and other standard products are of the linear type. These only encode data in one dimension, from left to right. The following shows a typical UPC barcode.

Matrix or 2-dimensional barcode encode information in both directions (x-y axis). So they are able to store more information in the same space then linear barcodes. There are many type of matrix codes, the one shown below is that of QR-Code or Quick Response code. QR-Codes are mostly used for access with cell phones and have become a de-facto standard for Japanese cell phones. Other type of matrix codes include ShotCode, Semacode. The QR-code below shows the string ‘codediesel.com’ encoded.

In this post we will see how to generate linear barcodes using a pear library.

Installation

We will be using the ‘Image_Barcode’ Pear library to generate linear barcodes. You can download the library manually from here, or use the pear installer.

pear install Image_Barcode-1.1.0

Sample Code

A sample code for generating a CODE128 barcode is shown below.

<?php
 
   require_once 'Barcode.php';
   /* Data that will be encoded in the bar code */
   $bar_code_data = "TRSD5656";
 
   /* The third parameter can accept any from the following,
    * jpg, png and gif.
    */
   Image_Barcode::draw($bar_code_data, 'code128', 'png');
 
 ?>

By default, the image generated by the barcode is directly output to the browser. If you do not want this, you can pass ‘false’ as the fourth parameter and it will return the GD image resource object; allowing you to do further things with it.

Image_Barcode::draw($bar_code_data, 'code128', 'png', false);

Supported Types

The ‘Image_Barcode’ library supports the following barcode types:

• Code 39
• Code 128
• EAN 13
• INT 25
• PostNet
• UPCA

Some sample generated barcodes

Some barcodes generated with the above library are shown below. The last one is the United States Postal Service format.

Google charts and QR-Codes

Before closing this post we will take a look at generating QR-Codes using Google charts. The following line for example generates the QR-Code shown below. The ‘chl’ parameter specifies the data that will be encoded (‘www.codediesel.com’ here).

http://chart.apis.google.com/chart
?cht=qr&chs=150x150&chl=www.codediesel.com

Which you can embed in your html as below:

<img src="http://chart.apis.google.com/chart
?cht=qr&chs=150x150&chl=www.codediesel.com" />

Testing bar codes

Ok, now you have created all the barcodes you want, but the problem is you do not have a barcode reader handy; no sweat, Qualitysoft.de offers a free barcode testing application for checking a plethora of barcode formats.

Additional Resources

1. http://qrcode.kaywa.com/
2. http://www.denso-wave.com/qrcode/index-e.html
3. http://en.wikipedia.org/wiki/Barcode