As any one who has programmed knows about configuration files. Configuration files are mostly text files used to configure the parameters and initial settings for computer programs – mostly user applications, operating system settings. Below is a small list of frequently used file formats.
– XML
– Windows INI
– YAML
– JSON
– toml
(more…)
While working on a database design I came across the problem of finding combinations or ‘cartesian product’ between all the rows between one table to another table, but not repeating the same permutations.
Take the ‘Teams’ table below which lists four cricket teams. We want to create a combination list of matches between each teams.
(more…)
Like the QWERTY keyboard design, which was originally designed so as to slow down typing and prevent key jams in the original typewriters; US Area codes also depend on the design of the then rotary telephones. On the rotary telephone dial, the digits are arranged in a circular layout so that a finger wheel may be rotated with one finger from the position of each digit to a fixed stop position.
(more…)
A recent web visualization project forced me to explore some data visualization libraries for JavaScript. Below are a few I narrowed down during the development phase.
Leaflet is the leading open-source JavaScript library for mobile-friendly interactive maps. Weighing around 38 KB of JS, it has all the mapping features most developers will ever need.
Although Leaflet is lightweight, and focuses on a core set of features, an easy way to extend its functionality is to use third-party plugins. There are hundreds of nice leaflet plugins you can choose from to extend the functionality of Leaflet. This is one of my favorite mapping libraries and which I’ll surely cover in another post.
(more…)
This month I’ll complete 1 year riding with Uber. The process of booking a ride and the response time has been amazing. This post however is on a different matter. As a data aficionado I was curious on the various locations I had travelled over the year and the cost each month. Heading over to the Uber API docs was a disappointment as Uber does not provide any api for getting the ride history data. My next plan was to scrape the data from Uber pages using PHP or Python. Just when I was going to start the project to scrape Uber trip data, a little Google search returned a nice bookmarklet by @ummjackson that scrapes the data and exports it to a CSV file.
(more…)
There is nothing painful than having to rebuild a database after you find that the records in the database are corrupted. Data corruption in databases can take many forms – from allowing inaccurate data to be entered to data field truncation due to inadequate field length. This post gives an introduction to the data quality issues you will encounter when designing database applications.
(more…)
During database projects I frequently need to export certain views or intermediate results as CSV files. These are further used for importing into other databases. However, many times it is required to display the CSV files in web pages as HTML tables. As most times the data is generated on the fly it requires some JavaScript to turn the CSV data to HTML. One of my current favorite is the csv-to-html-table library. This allows you to easily integrate CSV files into your web pages.
(more…)
Every database programmer knows about database normalization – the process of organizing the columns and tables a relational database to minimize data redundancy. Semantic Normalization is the process of reshaping free form entered data. I first encountered the concept in Nathan Marzs Big Data: Principles and best practices of scalable realtime data systems
(more…)
Database schema visualization is an essential process during the initial project design phase. Also, it can be useful when you need to reverse engineer a big schema and need to understand the various relationships between different tables. Below are given various database diagramming and visualization tools that will help you make sense of your complex database designs.
(more…)