Search This Blog

Thursday, July 30, 2009

Loading MySQL data via stdin

Here's a handy trick to put in a cron script to automate importing of data into mysql. As long as the tab delimited fields are in the same order as your mysql table columns, you can pipe data directly into it with LOAD DATA LOCAL INFILE '/dev/stdin'. This eliminates the need for an intermediary file - unless you're dealing with millions of rows, then a intermediary file might be neccessary.

/usr/bin/dump_customers | sed 's/abc/xyz/g' | mysql -u impy -pSeCrEt -e"use mydb; DELETE FROM customers; LOAD DATA LOCAL INFILE '/dev/stdin' REPLACE INTO TABLE customers FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n'"

I injected a simple sed search and replace in order to hotfix some data. Also note, that dates need to be in the format of YYYY-MM-DD.

1 comment:

Anonymous said...

thanks!