Archive for the ‘data’ Category

CSV files are ubiquitous in the programming world and most data interchange is done using the CSV format. One problem however is that you cannot quickly check for certain rows depending on a particular field criteria. As an example, say you need to find all the rows from a CSV file that has a particular […]

One of the important tasks in any web application is proper sanitization and standardization of data. Any data stored in a database should be in a standardized format, specially data that comes from a variety of sources. Scrubbers or data cleaners are an important part of the data transformation process. Whenever you are involved in […]

Descriptive statistics can be quite useful for simple analysis of records in a database. For example, to calculate average numbers of sales or products for a particular duration, or the Variance of sales for a month etc. We can easily calculate standard descriptive statistic measures in MySQL such as MEAN, SUM, STANDARD DEVIATION, VARIANCE, MIN […]

With an increasing number of users migrating to WordPress, moving data from an existing CMS to WordPress has become an essential process in the transition. Although migrating small databases from another CMS can be done in a few hours, large databases can take days or even weeks. In one of my recent endeavors of the […]

A recent project of mine entailed replicating a RETS database on a local MySQL database server. The client had a new real-estate mobile search app in development and wanted to have a local copy of the RETS database for search queries instead of a remote RETS server. This approach has several advantages: Response times for […]

In one of my recent project requiring a CSV text file import, the data columns had to be in certain predefined positions for the program to work correctly. The import did not depend on field labels, which is obviously a wrong way to process data. It would not have been much of a concern had […]

A recent task needed me to import a large amount of data from a Access MDB database to MySQL. My first choice for the job was the mdb-tools set of utilities. mdb-tools provides a set of tools and applications to export MDB data and schema to other databases such as MySQL, Oracle, Sybase, PostgreSQL, and […]

Search-engines have reached a impasse with respect to understanding of HTML data. Although most sites, especially databases centric sites use structured data in the backend, once the data is rendered as HTML the data looses all its structure and makes it difficult for search engines to understand them correctly. Until now most search engines have […]

During a recent project I needed a database of IATA airport codes. Airports around the world are universally known by a unique three-letter code: the “International Air Transport Association (IATA) Location Identifier”. It is much easier for pilots, travel agents, frequent flyers, baggage handlers and anyone having anything to do with airlines to say and […]

To set the stage we will start with a programming problem, this will keep the discussion anchored to a relevant context. The problem is this : We want to write a function that takes a image file as an argument and tells us whether the file is a GIF image; irrelevant with whatever the extension […]

Sign up for fresh content in your email