code-diesel » php

Rejecting unwanted characters from input

sameer — Sat, 07 Jan 2012 09:23:12 +0000

It seems that some common elements of programming stump us from time to time. Take the task of filtering a input search string in PHP to remove unwanted characters. Using a RegEx many developers find it easy to search for a substring, but find it difficult to use the same to reject some particular characters from a string. A simple solution is shown below, which rejects all the characters from the input except alphanumeric and a space.

$search = "the great /%&&world ,fair of 1964";
$cleaned = preg_replace("/[^A-Za-z0-9 ]/", "", $search);

Returns:

the great world fair of 1964

The important part of the regular expression is the caret ^ along with the character class [...]. The normal character class will match elements specified in the class. For example the following will match the alphanumeric characters and a space in the class and replace them with a empty character, effectively removing them, because we have specified a empty string as the second parameter in preg_replace.

$search = "the great /%&&world ,fair of 1964";
$cleaned = preg_replace("/[A-Za-z0-9 ]/", "", $search);

Returns:

/%&&,

However if we use a negated character class, which is a character class starting with a caret ^ sign, we invert the meaning of the class. Now it means match all the characters not in the character class, which basically removes all the unwanted characters.

If you are using ereg_replace then you need to use the following,which does not have any delimiters. Note that this function has been DEPRECATED as of PHP 5.3.0, and will throw a ‘Deprecated’ error.

$cleaned = ereg_replace("[^A-Za-z0-9 ]", "", $search);

The whacky art of Lullian poetry

sameer — Thu, 01 Dec 2011 15:04:55 +0000

The article describes a Lullian poetry generator in PHP, the idea for which was inspired from Clifford Rickover’s wonderful and eclectic, The Loom of God. The idea is attributed to Ramon Lull, a Spanish theologian, who lived around the 13th century. Lull believed that every branch of knowledge was governed by simple basic principles, and that we could explore all knowledge by evaluating all possible combinations of those principles.

The design for the Lullian poetry sprouts from this premise. The Lullian generator works by randomly selecting words and phrases which are then placed in a pre-defined ‘Semantic Schema’. The philosophical idea being that randomly chosen words can be used to stimulate the mind in producing startling fictional plots and verbal imagery that you might not otherwise consider.

Take the following example short poems generated by the program. Although the resulting poems are nothing like the ones you would ordinarily encounter, the real facet of the poems is their whimsical nature of word combinations, sometimes ridiculously so.

A delightful stranger
A delightful stranger misgiving while riding the fantastic flower.
calmly , the stranger withholding .
The flower thrusting while weeping a eager pleasure.

A fine beast
A fine beast shriving while eating the gentle ocean.
always, the beast melting .
The ocean casting while shooting a fantastic weather.

A agreeable cat
A agreeable cat learning while riding the elated pest.
enormously, the cat springing.
The pest outshining while hanging a calm arm.

The ‘Semantic Schema’ used to generate the above poems is shown below.

A (adjective1) (noun1)
A (adjective1) (noun1) (verb) (phrase) the (adjective) (noun2).
(adverb), the (noun1) (verb).
The (noun2) (verb) (phrase) a (adjective) (noun3).

Now all you need to do is to randomly select a word from a word class (verb, adjective, noun etc.), and replace it with its corresponding placeholder in the schema. The PHP code given below does this automatically for you. Of course you can change the schema to get different types of poem structure.


 
/**
  * Function to replace every occurrence of a English
  * word class with a random selected word from an array.
  */
 
function substitute($match)
{
    global $adjective, $noun, $verb, $adverb, $phrase;
    global $stack;
    $match[0] = preg_replace('/[\(\)]/', "", $match[0]);
    $mytype = $match[0];
 
    if(!ctype_alpha($match[0]))
    {
        $base = substr($match[0],0, strlen($match[0])-1);
 
        if(array_key_exists($match[0], $stack)) {
            return $stack[$match[0]];
        } else {
            $randomOffset = rand(0, count(${$base}) - 1);
            $stack[$match[0]] = ${$base}[$randomOffset];
            return $stack[$match[0]];
        }
    }
 
    $randomOffset = rand(0, count(${$mytype}) - 1);
    return ${$mytype}[$randomOffset];
}
 
 
/* Start */
$stack = array();
$data = array("noun", "verb", "adjective", "adverb", "phrase");
 
/* Read all the words from the files into the approriate array */
foreach($data as $type) {
    ${$type} = file("{$type}.txt", FILE_IGNORE_NEW_LINES);
}
 
/* Poem semantic schema */
$template = <<;
 
/* Create the actual random poem */
$poem=preg_replace_callback('/\([a-z 1-9]*\)/', "substitute", $template);
echo $poem ;
 
?>

The code uses five text files for each word class, containing random words and phrases. The quirkiness of the poems depends on these word collection; so if you meticulously select some interesting words for these files, you can create some offbeat poems.

Another thing you can do with the above idea it to use it to generate book or movie titles, by defining an appropriate schema. Some schema templates are shown below, with the examples.

(noun) and (noun)
Example: Pride and Prejudice, Crime and Punishment, War and Peace

The (noun) Who
Example: The Spy Who Loved Me, The Man Who Melted

Of course, to create some nice titles you require some good, selected repository of words. The same method can also be used to generate website domain names. So for example if your business vertical is apparels, you can create a word list of various terms from the apparel industry and let the generator churn out some names.

To play further, you can download the code from below.

Download code files
Downloads : 43 / File size : 4.2 kB

Building a simple Parser and Lexer in PHP

sameer — Thu, 17 Nov 2011 05:32:59 +0000

Compiler design is a complex endeavor, but also one of the most satisfying projects you can undertake. Lately I’ve been interested in compiler and parser design; my interest piqued by Debasish Ghosh’s wonderful book, DSLs in Action. Web development in general provides a far less opportunity to work in the domain of compiler or interpreter design. So the idea of building DSL’s was the perfect excuse for learning parser design.

To design a good DSL it is necessary to have some compiler design knowledge under your belt. The classic Dragon book is usually the first choice of reference to pickup on compiler design, but it is too theory oriented, and what I needed was something to get me started with writing code quickly, rather than mull over automata theory, syntax trees and other compiler theory concepts. My first choice was Ronald Maks Writing Compilers and Interpreters. Although it is an incredible book, it is rather elaborate in its presentation. What I instead wanted was a cookbook style presentation, using which I could quickly design some working code, and only later refer to the above books for more in-depth understanding.

After looking around for a while I settled for Terence Parr’s Language Implementation Patterns. This is exactly what I needed – bit sized patterns on compiler and parser design with working code. The book provides a recipe style approach, gradually moving from simple to complex compiler/parser design issues. As I primarily work with PHP, I thought of porting some code to PHP to see how it works.

A simple Lexer

Below are some of the few recipes from the initial chapter of the book, on designing a LL(1) lexer and parser. The language we want to recognize is an extremely simple one – a list of names like [a,b,c,d,e]. A LL(1) lexer for the same is shown below.

test_lexer.php


 
require_once('ListLexer.php');
require_once('Token.php');
 
$lexer = new ListLexer($argv[1]);
$token = $lexer->nextToken();
 
while($token->type != 1) {
    echo $token . "\n";
    $token = $lexer->nextToken();
}
 
?>

Testing the lexer from the command line is given below with the language string passed as a argument. The lexer just outputs the tokens of the language, throwing an exception if a character outside the language is encountered.

D:\localhost\test\lexer>php test_lexer.php [a,b,c,d]
<'[',LBRACK>
<'a',NAME>
<',',COMMA>
<'b',NAME>
<',',COMMA>
<'c',NAME>
<',',COMMA>
<'d',NAME>
<']',RBRACK>

ListLexer.php


 
require_once('lexer.php');
 
class ListLexer extends Lexer {
    const NAME      = 2;
    const COMMA     = 3;
    const LBRACK    = 4;
    const RBRACK    = 5;
    static $tokenNames = array("n/a", "",
                               "NAME", "COMMA",
                               "LBRACK", "RBRACK" );
 
    public function getTokenName($x) {
        return ListLexer::$tokenNames[$x];
    }
 
    public function ListLexer($input) {
        parent::__construct($input);
    }
 
    public function isLETTER() {
        return $this->c >= 'a' &&
               $this->c <= 'z' ||
               $this->c >= 'A' &&
               $this->c <= 'Z';
    }
 
    public function nextToken() {
        while ( $this->c != self::EOF ) {
            switch ( $this->c ) {
                case ' ' :  case '\t': case '\n': case '\r': $this->WS();
                           continue;
                case ',' : $this->consume();
                           return new Token(self::COMMA, ",");
                case '[' : $this->consume();
                           return new Token(self::LBRACK, "[");
                case ']' : $this->consume();
                           return new Token(self::RBRACK, "]");
                default:
                    if ($this->isLETTER() ) return $this->NAME();
                    throw new Exception("invalid character: " + $this->c);
            }
        }
        return new Token(self::EOF_TYPE,"");
    }
 
    /** NAME : ('a'..'z'|'A'..'Z')+; // NAME is sequence of >=1 letter */
    public function NAME() {
        $buf = '';
        do {
            $buf .= $this->c;
            $this->consume();
        } while ($this->isLETTER());
 
        return new Token(self::NAME, $buf);
    }
 
    /** WS : (' '|'\t'|'\n'|'\r')* ; // ignore any whitespace */
    public function WS() {
        while(ctype_space($this->c)) {
            $this->consume();
        }
    }
}
 
?>

The main lexer file.
Lexer.php


 
abstract class Lexer {
 
    const EOF       = -1; // represent end of file char
    const EOF_TYPE  = 1;  // represent EOF token type
    protected $input;     // input string
    protected $p = 0;     // index into input of current character
    protected $c;         // current character
 
    public function Lexer($input) {
        $this->input = $input;
        // prime lookahead
        $this->c = substr($input, $this->p, 1);
    }
 
    /** Move one character; detect "end of file" */
    public function consume() {
        $this->p++;
        if ($this->p >= strlen($this->input)) {
            $this->c = Lexer::EOF;
        }
        else {
            $this->c = substr($this->input, $this->p, 1);
        }
    }
 
    public abstract function nextToken();
    public abstract function getTokenName($tokenType);
}
 
?>

Token.php


 
class Token {
    public $type;
    public $text;
 
    public function Token($type, $text) {
        $this->type = $type;
        $this->text = $text;
    }
 
    public function __toString() {
        $tname = ListLexer::$tokenNames[$this->type];
        return "<'" . $this->text . "'," . $tname . ">";
    }
}
 
?>

A simple parser

A parser for our simple language is given below along with the test script. This uses the Token and Lexer classes we defined earlier.

test_parser.php


 
require_once('ListLexer.php');
require_once('Token.php');
require_once('ListParser.php');
 
 
$lexer = new ListLexer($argv[1]);
$parser = new ListParser($lexer);
$parser->rlist(); // begin parsing at rule list
 
?>

The parser takes our input language as a argument from the command-line. Passing a correct string to the parser will return nothing, as we have not implemented any language processing code. Passing an incorrect string will throw an exception.

D:\localhost\parser>php test_parser.php [a,,c,d]
PHP Fatal error:  Uncaught exception 'Exception' with message 
'Expecting name or list : Found <',',COMMA>'

ListParser.php


 
require_once('Parser.php');
 
class ListParser extends Parser {
    public function ListParser(Lexer $input) {
        parent::__construct($input);
    }
 
    /** list : '[' elements ']' ; // match bracketed list */
    public function rlist() {
        $this->match(ListLexer::LBRACK);
        $this->elements();
        $this->match(ListLexer::RBRACK);
    }
    /** elements : element (',' element)* ; */
    function elements() {
        $this->element();
        while ($this->lookahead->type == ListLexer::COMMA ) {
            $this->match(ListLexer::COMMA);
            $this->element();
        }
    }
    /** element : name | list ; // element is name or nested list */
    function element() {
        if ($this->lookahead->type == ListLexer::NAME ) {
            $this->match(ListLexer::NAME);
        }
        else if ($this->lookahead->type == ListLexer::LBRACK) {
            $this->rlist();
        }
        else {
            throw new Exception("Expecting name or list : Found "  . 
                                 $this->lookahead);
        }
    }
}
 
?>

Parser.php


 
abstract class Parser {
    public $input;     // from where do we get tokens?
    public $lookahead; // the current lookahead token
 
    public function Parser(ListLexer $input) {
        $this->input = $input;
        $this->consume();
    }
 
    /** If lookahead token type matches x, consume & return else error */
    public function match($x) {
        if ($this->lookahead->type == $x ) {
            $this->consume();
        } else {
            throw new Exception("Expecting token " .
                                $this->input->getTokenName($x) .
                                ":Found " . $this->lookahead);
        }
    }
    public function consume() {
        $this->lookahead = $this->input->nextToken();
    }
}
 
?>

After working through the code, you may realize that PHP is not a very good candidate for developing programs of such kind. Even though you can accomplish the task in hand, developing a much more complex parser design in PHP can be a challenging task.

If you want to play with the code, you can download it from below.

Download code files
Downloads : 256 / File size : 2.8 kB

Source Code Signatures in PHP

sameer — Thu, 10 Nov 2011 05:38:55 +0000

This post is based on Ward Cunningham’s Signature Survey: A Method for Browsing Unfamiliar Code article. Ward suggests that when studying new and unfamiliar source codes, it is vital that a programmer get a intuitive sense of the code structure under consideration. One method he suggests is that of summarizing on a single line select punctuation characters from each source file. He calls this the file’s “signature”. This will help the developer gauge the complexity of the code base.

So for example, if we only consider using braces ‘{}’, and semicolon ‘;’ – one of the common punctuation elements in a PHP program, a programmer can get a feel for the structure of a source file. Usually braces delimit blocks and functions in PHP, and many other languages, so this can give a programmer a sense of the length of code blocks or function sizes.

A sample source file taken from a WordPress system, and its signature using select punctuation characters ( ‘{}’ and ‘;’ ) is given below.

Sample WordPress source file – wp-blog-header.php:


/**
 * Loads the WordPress environment and template.
 *
 * @package WordPress
 */
 
if ( !isset($wp_did_header) ) {
	$wp_did_header = true;
	require_once( dirname(__FILE__) . '/wp-load.php' );
	wp();
	require_once( ABSPATH . WPINC . '/template-loader.php' );
}
?>

Signature of the above file:

wp-blog-header.php : {;;;;}

This shows that the file has one block section (function, class or other), containing four statements. Of course to get a quick intuitive feel for the signature, you will have to work with at least a few diverse code bases. This will prime you to estimate the complexity of the code by just looking at the signature.

If we also include parenthesis ‘()’ in the signature, we get the following. But this looks a little complex and less intuitive than the previous one.

wp-blog-header.php : (()){;(());();();}

Some more signatures of various WordPress root files is given below.

index.php : ;;
wp-atom.php : ;;;
wp-blog-header.php : {;;;;}
wp-comments-post.php : {;;;;};;;;{;;};;{;;}{;;}{;;}{;;}{;};;;;;{;;;;
{{;;}}}{;};{;;};;;;;{;;;;};;;;
wp-commentsrss2.php : ;;;
wp-config-sample.php : {};;;;;;{};;;;;;;;;;;;;
wp-config.php : ;{};;;;;;{}}{{{;{};;{;;{;{};{{{;{;}{;;;;;;
wp-cron.php : ;;;{;};;;;{;{{;{;;};;}}};
wp-feed.php : ;;;
wp-links-opml.php : ;;;;{;;};;;;;;;;;;;;;;;;
xmlrpc.php : ;;{;};;{;;;;};;;;;;{;{;;;;;};};;;;;
wp-load.php : ;;{;}{;}{;;;;;;;;;;;;}
wp-pass.php : ;;;;;
wp-rdf.php : ;;;
wp-register.php : ;;;
wp-rss.php : ;;;
wp-rss2.php : ;;;
wp-login.php : ;{{;;}{;;}}{;;;;;;;;;;;;;;;;{;;;;{;}{;}}{{;}};;{;;;}{;;
;};;{;;}{;;{;{;;}};;}}{;;;{;;}{};;;}{;;{;{;}{;{;;}}};{;}{;}{;;{{;};}{{
;;}{}}}{;;;;;};}{;;;{;;}{;;};;{;;};;;;;;;;{;;;};;;;;;;;;;;;;}{;;;;;;;}
{;;;}{;;;{;}{;;}{;}{;}{;;;}{;};;;;;{;;;};;;};;;;;;;{;;;};;;;;{;;;;;;{;
{;;;}};;;;;;;;;;;;;{;;};{;}{;;;;};;;;;;;;;;;{;;}{;;};;{;;;{;;;}};;;;;;
;;;;;{;{{;;}}}{;;}{;};;;;{{;;{};;;}{;;;};;};;;;;;;;;;;;;;;;;;{}{;}{;}{
{{{;;}{;{;}};;}{}};}{;};;;}
wp-mail.php : ;;;;;;;;;;;;;;{;;;};;{;;;;;;;;;;{;{;}{{;;;;{;;};}{;;;;;}
{;;;}{;;{;}{;};;}{;;;{;;{;}{;;}}{;}}{;;{;};;;;;;;;;;{{;}};;;;;}}}{;;}{
;};{;;{;;};};;{;}{;};;;;;;;;;;;;;;;{;;;}{;}};
wp-settings.php : ;;;;;;;;;;;;;;;;;;;;;;;;;;;;{;;}{;};;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;{;;;};{;};{{;};};;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;{;
;};;;;;{{;;};};
wp-signup.php : ;;;{;;}{;};{;}{;;}{;;};{{;;}{;}{;;;;;}{;;;}{;;}{;;;;}{;
}{;;;;;}};;;{;;;{};;{;;;}{};;{};{};;}{;;;}{;{;};;;{}{;};}{;}{;;{;};;;;;
{;};;{{;;}};;;}{;;;;;{;;};;;;;;}{{}{}{};;;}{;;;;;;;;;;;{}{}{}}{;;{;;}{;
;};;;}{;;}{;;;;;;;;;;}{;;{;;};;{;;};;;;;;}{{}{}{};;;;;};;;;;;;;;;;;{;}{
;;;;}{;{;;;;;;;;;;;;;;{;;;};}};;
wp-trackback.php : {;;}{;;{;;;;;;}{;;;;}};{;;};;;;;;;;{;;;};;;;;{;;}{;;
;;;;;;;;;;;;;;;}
wp-activate.php : ;;;{;;};;{;};{{;}{;;}{;}{;}{;;;;;}};;{;}{;;{{;;;{;;
;}{;;;};}{;;}}{;;;;;;;;;;}};;;

To generate the signatures yourself, you can use the following shell script or a PHP program. The shell script is shown below.

for i in *.php; do
    echo -n "$i: "
    sed 's/[^{};]//g' $i | tr -d '\n'
    echo 
done

And below is a PHP version to do the same.


 
/* Get all PHP files in the current directory */
$dir = './*.php';
 
/* Use the SPL Iterator class */
$files = new GlobIterator($dir);
 
/* Get the signature of each file */
foreach ($files as $file) {
    $data = file_get_contents($file);
    preg_match_all('/[{};]/',$data,$out);
    $signature = '';
 
    foreach($out[0] as $line) {
        $signature .= $line;
    }
 
    echo $file->getFilename() . " : " . $signature . "\n";
}

Furthermore, we can make the output more attractive and user friendly, by outputting it to HTML, with links to each source file.

The forgotten PHP logical operator

sameer — Fri, 04 Nov 2011 13:55:53 +0000

It is very rare that I have seen some code that employs the PHP XOR logical operator. It is not the case that it is not required or programmers are unfamiliar with it; it probably seems that most programmers have become comfortable with using the more common ‘&&’ and ‘||’ operators to construct all logical comparisons; the XOR operator relegated to the subconscious black hole.

Take the following artificial example. We want to check if only one of the directories – either ‘$dir1′ or ‘$dir2′, but not both exists. The code using only the logical ‘&&’ , ‘||’ and ‘!’ operators is shown below.


 
$dir1 = 'download_A';
$dir2 = 'download_B';
 
if((is_dir($dir1) && !is_dir($dir2)) || (!is_dir($dir1) && is_dir($dir2))) {
    echo "Only one Directory exists!";
}
 
?>

The code obviously looks confusing, which we could easily simplify using the ‘XOR’ operator as given below. The code is concise, easier to understand, and the chances of introducing bugs is smaller than the previous version.


 
$dir1 = 'download_A';
$dir2 = 'download_B';
 
if(is_dir($dir1) XOR is_dir($dir2)) {
    echo "Only one Directory exists!";
}
 
?>

Grabbing the referrer search engine keywords for a site

sameer — Mon, 17 Oct 2011 13:20:54 +0000

A couple of weeks back I had to write a solution for a client to track the referrer search engine from where the user came to his sites contact page, without using Google Analytics. If a user was to fill the contact form on the website, the referring search engine name and the keyword for which it was refereed was to be emailed along with the contact information. The following is a solution for the same.

To get the referrer information we need to use the $_SERVER['HTTP_REFERER'] variable and store it in a session so that it is not overwritten as the user browses the site before he submits the contact information. If the users comes to the website from a search engine he could land on any page, so we need to add some code to capture the referrer url on each page of the site. We only need to save the referrer if the referrer is from outside the current domain.

The following is the code that need to be added to each web page on the website. The referrer url will then be stored in a session and carried along as the user browses the site.


/* saveReferer.php */
 
$my_site = "http://www.codediesel.com";
 
/* If the referrer is from outside the domain, store the url in a session */
if(!substr($_SERVER['HTTP_REFERER'], 0, strlen($my_site)) !== $my_site) {
    if(!isset($_SESSION['my_referer'])) {
        $_SESSION['my_referer'] = $_SERVER['HTTP_REFERER'];
    }
} 
?>

You can now include the above file in all your web pages.


 
session_start();
require_once('saveReferer.php');
.
.
?>

To grab the search keyword from the referrer url we can use the get_keyword function given below.


 
function get_keyword($referer)
{
    $search_phrase = '';
    $engines = array('dmoz'     => 'q=',
                     'aol'      => 'q=',
                     'ask'      => 'q=',
                     'google'   => 'q=',
                     'bing'     => 'q=',
                     'hotbot'   => 'q=',
                     'teoma'    => 'q=',
                     'yahoo'    => 'p=',
                     'altavista'=> 'p=',
                     'lycos'    => 'query=',
                     'kanoodle' => 'query='
                     );
 
    foreach($engines as $engine => $query_param) {
        // Check if the referer is a search engine from our list.
        // Also check if the query parameter is valid.
        if (strpos($referer, $engine.".") !==  false && 
            strpos($referer, $query_param) !==  false) {
 
            // Grab the keyword from the referer url
            $referer .= "&";
            $pattern = "/[?&]{$query_param}(.*?)&/si";
            preg_match($pattern, $referer, $matches);
            $search_phrase = urldecode($matches[1]);
            return array($engine, $search_phrase);
        }   
    }
    return;
}
 
?>

The following is a sample contact page which sends the referrer search engine name and the keyword to the website admin.


 
/* contact.php */
session_start();
 
$engine = '';
$search_keyword = '';
 
$data = get_keyword($_SESSION['my_referer']);
 
if($data) {
   $engine = $data[0];
   $search_keyword = $data[1];
}
 
$message  = "SOME MESSAGE HERE\r\n";
$message .= "Engine : $engine \n";
$message .= "Keyword: $search_keyword \n";
 
$headers .= "From: test@test.com ". "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
mail("admin@test.com", "Test Email", $message, $headers);
 
/* Make sure you reset the referrer session variable */
unset($_SESSION['my_referer']);
 
?>

Splitting large MySQL dump files

sameer — Sun, 02 Oct 2011 11:48:09 +0000

One of the frustrating things with working with MySQL is of importing large sql dump files. Either you get a ‘max execution time exceeded’ error from PHP or a ‘Max_allowed_packet_size’ from MySQL. In a recent task I needed to import a table of around a million records on a remote host, which quickly became an exercise in frustration due to various limitations on the server. SSH was of no help as changing the configuration files was restricted to the root user.

My last resort was to split the huge ‘INSERT’ statements into smaller size files. Manually doing the same is obviously time consuming and error prone; the only other solution is to write a small script to split the insert statements. This was to be a quick hack so the parsing code was to be of minimum complexity. Splitting a sql dump containing extended insert statements is somewhat complex so you need to have the dump file in a simple format – each insert statement should be on its own line as shown below.

INSERT INTO `dt_codes` VALUES ...
INSERT INTO `dt_codes` VALUES ...
INSERT INTO `dt_codes` VALUES ...
.
.

You can create the initial dump file in a simple format using the following command or you can do it using phpMyAdmin.

mysqldump -uUSER -pPASS --databases DATABASE_NAME --tables TABLE_NAME \
--extended-insert = FALSE > dump.sql

Once you have the big dump file you can use the following PHP script for splitting the file into smaller chunks. Change the variable ‘$max_lines_per_split’ to whatever value you think will work on your system.


 
set_time_limit(600);
 
/* Number of 'insert' statements per file */
$max_lines_per_split = 50000;
 
$dump_file = "dump.sql";
$split_file = "dump-split-%d.sql";
$dump_directory = "./sql-dump/";
 
$line_count = 0;
$file_count = 1;
$total_lines = 0;
 
$handle = @fopen($dump_file, "r");
$buffer = "";
 
if ($handle) {
    while(($line = fgets($handle)) !== false) {
        /* Only read 'insert' statements */
        if(!preg_match("/insert/i", $line)) continue;
        $buffer .= $line;
        $line_count++;
 
        /* Copy buffer to the split file */
        if($line_count >= $max_lines_per_split) {
            $file_name = $dump_directory . sprintf($split_file, $file_count);
            $out_write = @fopen($file_name, "w+");
            fputs($out_write, $buffer);
            fclose($out_write);
            $buffer = '';
            $line_count = 0;
            $file_count++;
        }
    }
 
    if($buffer) {
        /* Write out the remaining buffer */
        $file_name = $dump_directory . sprintf($split_file, $file_count);
        $out_write = @fopen($file_name, "w+");
        fputs($out_write, $buffer);
        fclose($out_write);
    }
 
    fclose($handle);
    echo "done.";
}

Once the dump file has been split into smaller ones, you can gzip them to reduce the size further.

gzip dump-split-*

Automatically create PHP classes from MySQL

sameer — Thu, 15 Sep 2011 06:37:54 +0000

Creating a database driven web application involves commonly used paradigms for data modification, which we commonly refer to as CRUD. Frameworks provides nice ORM wrappers to help the programmer. But for small projects frameworks can be an overkill. Still the programmer needs to design the basic database CRUD functionality, which can be quite tedious and repetitive. This is where auto database class generators can be helpful. If you have your database schema ready, you can automatically create the respective class wrappers for the tables.

I found two libraries which accomplish the task without much of an overhead.
Structy and table2class.

Take a sample database schema shown below.

CREATE TABLE `user` (
  `username` varchar(100) NOT NULL default '',
  `password` varchar(100) NOT NULL default '',
  KEY `username` (`username`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

The following is a php class automatically generated by ‘table2class’ from the above schema. The generated class file also includes additional support database initialization files. Note that some functions here may need modification to suit your particular application, but still, the major code required to get the application up and running is there. Any automation is better than nothing. And each such help to the programmer means faster project delivery time for the client.


/*************************************************************************
* Class Name:       user
* File Name:        class.user.php
* Generated:        Thursday, Sep 15, 2011 - 6:39:33 UTC
*  - for Table:     user
*   - in Database:  user
**************************************************************************/
 
// Files required by class:
require_once("class.database.php");
 
// Begin Class "user"
class user {
    // Variable declaration
    public $username; // Primary Key
    public $database;
 
    // Class Constructor
    public function __construct() {
        $this->database = new Database();
        $this->database->SetSettings("localhost", "user", "pass", "user");
    }
 
    // Class Destructor
    public function __destruct() {
        unset($this->database);
    }
 
    // GET Functions
    public function getusername() {
        return($this->username);
    }
 
    // SET Functions
    public function setusername($mValue) {
        $this->username = $mValue;
    }
 
    public function select($mID) { // SELECT Function
        // Execute SQL Query to get record.
        $sSQL = "SELECT * FROM user WHERE username = $mID;";
        $oResult = $this->database->query($sSQL);
        $oResult = $this->database->result;
        $oRow = mysql_fetch_object($oResult);
 
        // Assign results to class.
        $this->username = $oRow->username; // Primary Key
    }
 
    public function insert() {
        $this->username = NULL; // Remove primary key value for insert
        $sSQL = "INSERT INTO user () VALUES ();";
        $oResult = $this->database->query($sSQL);
        $this->username = $this->database->lastinsertid;
    }
 
    function update($mID) {
        $sSQL = "UPDATE user SET (username = '$this->username') 
                 WHERE username = $mID;";
        $oResult = $this->database->Query($sSQL);
    }
 
    public function delete($mID) {
        $sSQL = "DELETE FROM user WHERE username = $mID;";
        $oResult = $this->database->Query($sSQL);
    }
 
}
// End Class "user"
?>

‘Structy’ provides a somewhat different flavor as you can see from the code class generated by the library. This is more complete than ‘table2class’ but also a little complex. The ‘RET’ constant you see in the code below is generated by the library and is automatically defined in the other support files.


 
/**********************************************************************
user.class.php
Generated by STRUCTY 2011.09.15 08:33:41.
Copyright 2011 Structy, Frédéric Aebi. All rights reserved.
**********************************************************************/
 
define("USER", "user"); 
 
class user {
 
    private $username;
    private $password;
 
    public function setusername($pArg="0") {$this->username=$pArg;}
    public function setpassword($pArg="0") {$this->password=$pArg;}
 
    public function getusername() {return $this->username;}
    public function getpassword() {return $this->password;}
 
    public function readObject($array = array()) {
        $qry = "SELECT *".RET."FROM ".USER.RET;
        $and = "WHERE".RET;
 
        if($array['username'] != "") {
            $qry .= $and."username = '".$array['username']."'".RET;
            $and = "AND".RET;
        }
 
        if($array['password'] != "") {
            $qry .= $and."password = '".$array['password']."'".RET;
            $and = "AND".RET;
        }
 
        $record = Database::select($qry);
        if(count($record[0]) == 0) {
            return array();
        } else {
            $record = $record[0];
            $this->setusername($record['username']);
            $this->setpassword($record['password']);
            return true;
        }
    }
 
    public static function readArray($array = array()) {
        $qry = "SELECT *".RET."FROM ".USER.RET;
        $and = "WHERE".RET;
 
        if($array['username'] != "") {
            $qry .= $and."username = '".$array['username']."'".RET;
            $and = "AND".RET;
        }
 
        if($array['password'] != "") {
            $qry .= $and."password = '".$array['password']."'".RET;
            $and = "AND".RET;
        }
 
        $recordset = Database::select($qry);
        $class_objects = array();
        if(is_array($recordset) == true) {
            while(list($i, $record) = each($recordset)) {
                $class_object = new user();
                $class_object->setusername($record['username']);
                $class_object->setpassword($record['password']);
                $class_objects[$class_object->getusername()] = $class_object;
            }
        }
        return $class_objects;
    }
 
    public function insert($update = FALSE) {
        if($update == TRUE) {
            $qry  = "UPDATE ".USER.RET."SET".RET.
            "username = '".$this->getusername()."',".RET.
            "password = '".$this->getpassword()."'".RET.
    .RET;
 
            Database::insert($qry);
        } else {
            $qry  = "INSERT INTO ".USER." (".RET.
            "username, password".RET.
            ") VALUES (".RET.
            "'".$this->getusername()."',".RET.
            "'".$this->getpassword()."'".RET.
            ")".RET;
 
            Database::insert($qry);
        }
    }
 
    public static function delete($array = array()) {
        $qry = "DELETE".RET."FROM ".USER.RET;
        $and = "WHERE".RET;
 
        if($array['username'] != "") {
            $qry .= $and."username = '".$array['username']."'".RET;
            $and = "AND".RET;
        }
 
        if($array['password'] != "") {
            $qry .= $and."password = '".$array['password']."'".RET;
            $and = "AND".RET;
        }
 
        Database::delete($qry);
    }
}
 
?>

Functional style programming with Underscore.php

sameer — Tue, 13 Sep 2011 04:11:00 +0000

Underscore.php is a PHP port of the popular Underscore.js library. Underscore.php provides a utility library for PHP that provides a lot of the functional programming support that a programmer would expect in Ruby, but without adding much overhead during execution. The only caveat is that underscore.php requires PHP 5.3 or greater. Although you could accomplish some of the things using PHP’s built in functions, the functional style approach looks intuitive and easy to work with. Note that this not a purely functional programming like Haskell. It would be nice to integrate the library in your CakePHP framework, which will help bring some functional flavor to the framework.

Take a quick example of the ‘pluck’ method.

include_once('underscore.php');
 
$members= array(
  array('name'=>'bill', 'age'=>40, 'gender' => 'm'),
  array('name'=>'john', 'age'=>50, 'gender' => 'm'),
  array('name'=>'sarah','age'=>60, 'gender' => 'f')
);
 
$ret = __::pluck($members, 'name');
print_r($ret);

returns…

Array
(
    [0] => bill
    [1] => john
    [2] => sarah
)

Here is another using the ‘map’ method.

$ret = __::map(array(1, 2, 3), function($num) { return $num * 3; });
print_r($ret);

returns…

Array
(
    [0] => 3
    [1] => 6
    [2] => 9
)

Underscore.php works in both object-oriented and static styles. The following lines give the examples of both.

$members= array(
  array('name'=>'bill', 'age'=>40, 'gender' => 'm'),
  array('name'=>'john', 'age'=>50, 'gender' => 'm'),
  array('name'=>'sarah','age'=>60, 'gender' => 'f')
);
 
/* Static style */
$ret = __::max($members, function($member) { return $member['age']; });
 
/* Object Oriented style */
$ret = __($members)->max(function($member) { return $member['age']; });
 
print_r($ret);

returns…

Array
(
    [name] => sarah
    [age] => 60
    [gender] => f
)

Another example using the ‘template’ method.

$members = array(
  array('name'=>'bill', 'age'=>40, 'gender' => 'm'),
  array('name'=>'john', 'age'=>50, 'gender' => 'm'),
  array('name'=>'sarah','age'=>60, 'gender' => 'f')
);
 
$template = '<% __::each($members, function($member) { 
             %> <%= $member["name"] %>
 <% }); %>';
 
$ret = __::template($template, array('members'=>$members));
 
print_r($ret);

returns…

bill  
john  
sarah

How does it work

The underscore.php files defines a ‘__’ class which encapsulates all the functional methods and uses the functional programming ideas introduced in PHP 5.3.

// Underscore.php
 
class __ {
  // Return an array of values by mapping each item through the iterator
  public function map($collection=null, $iterator=null) {
     ...
  }
  .
  .
  /* other methods */
}

Presently the library supports the following methods.

Collections
each, map, reduce, reduceRight, detect, select, reject, all, any, includ, invoke, pluck, max, min, groupBy, sortBy, sortedIndex, toArray, size

Arrays
first, rest, last, compact, flatten, without, uniq, union, intersection, difference, zip, indexOf, lastIndexOf, range

Functions
memoize, throttle, once, after, wrap, compose

Objects
keys, values, functions, extend, defaults, clon, tap, isEqual, isEmpty, isObject, isArray, isFunction, isString, isNumber, isBoolean, isDate, isNaN, isNull

Utility
identity, times, mixin, uniqueId, template

Chaining
chain, value

Get Facebook user profile using their photo filename

sameer — Mon, 12 Sep 2011 10:27:36 +0000

If you find some image file saved from Facebook, you can quickly get the users profile ID and other details from the image file name. Every image saved from a Facebook profile has a unique filename such as ’86948_8239343856_5678_n.jpg’ (this is a dummy number).

The 2nd number from left in the filename, in this case 8239343856, which is the unique profile ID of the Facebook user.

So now the Facebook profile URL for the user is http://www.facebook.com/profile.php?id=8239343856. The following PHP script will help you get the various details like name, username, gender and locale.


 
/* filename like 86948_8239343856_5678_n.jpg */
$photo_name = "PHOTO_FILENAME";
 
$profileId = explode("_", $photo_name);
 
/* filename format */
switch(count($profileId)) {
    case 6 :    $id = $profileId[2];
                break;
    case 4 :    $id = $profileId[1];
                break;
    case 3 :    $temp = array("n", "N");
                $id = str_replace($temp, "", $profileId[0]);
                break;
    default:    break;
}
 
$profileLink = "http://www.facebook.com/profile.php?id=".$id;
$profileImage = "http://graph.facebook.com/".$id."/picture?type=large";
$graphAPI = file_get_contents("http://graph.facebook.com/".$id);
$graphJSON = json_decode($graphAPI, true);
 
print_r($graphJSON);
 
?>