Archive for the ‘Tip’ Category

Switch to classic gnome menu in Ubuntu 11.10 how to

February 10, 2012 Leave a comment

1) open a terminal and run

sudo apt-get -y install gnome-session-fallback
sudo sed -i 's#\(user-session=\)ubuntu#\1gnome-classic#' /etc/lightdm/lightdm.conf

2) Disable auto login if it’s on
User icon on the top-right corner of the screen -> User Accounts -> Unlock -> Automatic Login Off

3) Reboot, click the setting icon on the login screen to make sure GNOME Classic is selected. This needs to be done only once.


To switch back to Unity,

sudo sed -i 's#\(user-session=\)gnome-classic#\1ubuntu#' /etc/lightdm/lightdm.conf
Categories: Tip, ubuntu

Facetime “The server encountered an error processing registration please try again later” fix on Hackintosh

October 1, 2011 4 comments

I have Lion running on my hackintosh (thanks to and the usb camera that I have (a Logitech QuickCam Orbit AF Webcam) happens to be supported by my build so I fired up Facetime the other night only to find the following error:

The server encountered an error processing registration please try again later

after the login credential is verified. After reading some posts on the forum I found the solution here and it turns out to be quite easy to fix: just download, install the latest Chimera (current version 1.5.4) from here, followed by a reboot and voila, the error no longer gets in the way. Loving my hackintosh.

Categories: lion, mac osx, Tip

Replace if … then … fi with && and || in Bash

October 1, 2011 4 comments

&& and || are the shorthand of if … then … fi in Bash. They
are very useful in writing simple scripts, for example:

1. Run script based on day of week:

[ $(date +%w) -eq 6 ] && echo "do something on Saturdays" || echo "do different things on other days"

The equivalent if block of the above would be:

if [ $(date +%w) -eq 6 ]; then
    echo "do something on Saturdays"
    echo "do different things on other days"

If more commands are needed when if condition matches (or doesn’t match), you can use { } to put the commands together

[ $(date +%w) -eq 6 ] && {echo "do something on Saturdays"; do_some_other_stuff; exit 0;} || echo "do different things on other days"

The shorthand also makes it easier to write stuffs to crontab, for example, we can put the something like this to crontab

1 8 * * * [ $(date +\%d) -eq 1 ] && ~/scripts/run_stuff_on_first_day_of_a_month

The ‘%’ is escaped, you can check out my another post regarding this.

2. Make sure a specific mount exists
Sometimes we need to make sure a mount point exists before any other tasks can be performed, it’s quite simple to do it with the shorthand form of conditional block:

mount|grep -q "^/dev/sdb1 " || mount /mnt/datastore

This will ensure partition /dev/sdb1 will be mounted to /mnt/datastore. A couple of things are worth pointing out:
1) grep option -q will suppress std output from grep, results are stored in $?, from which operator || will determine to run “mount /mnt/datastore” or not.
2) ^/dev/sdb1 instead of /mnt/datastore is used for the regular expression to eliminate false match. It’s unlikely but possible that the following mount point might exists
/dev/somepartition on /mnt/datastore 2 (ext3, local …)
An additional space is inserted to the end of the regexp in case there are partitions such as /dev/sdb1X (x=0-9) mounted.

3. Make sure a specific folder exists, if not, create it before script runs. For example:

[ -d /tmp/mydir/work ] || mkdir -p /tmp/mydir/work
Categories: Bash, Programming, Tip

A couple of plugins that make web developing life easier.

September 18, 2011 Leave a comment

The plugin basically extends the functionality of % in vim, it cycles between opening and closing tags, very handy if you love coding htmls by hand.
Steps to make it work: { windows users pls unzip the files to the vim home directory }
1) download the zip file from the above link
2) unzip -d ~/.vim
3) add the following line to your .vimrc
filetype plugin on
if filetype plugin is set to off [ default setting ]
4) open an existing .html file or create a new one, under normal mode, pressing % will cycle between html tags, v% will select text between opening and closing tags (with the opening tag name selected)

This is yet another handy and powerful vim plugin. As its name suggests it does one thing and it does really well – beatifies javascript files. Steps to make it work:
1) Download it and save it to ~/.vim/plugin
2) Open an existing .js file or create a new one, under normal mode, press \ff and the code is formatted

There’s one thing that this plug in is not able to do – format the selected js code – in other words you can’t beautify the embedded js code in html or php files.

[ Edit 9/18/2011: Actually I just found another better solution to format javascript code – what I like the best is that this indenter formats all javascript code either in stand-alone file or embedded code. Just put the file html.vim from to ~/.vim/indent and use gg=G to format the whole file or = to format selected code. ]

Categories: html, Tip, vim

Something to watch out while using ‘%’ in crontab

August 12, 2011 1 comment

% is a special character in crontab and it has to be escaped with another ‘%’ a ‘\’. I just found out through a fail-to-run crontab task:

Non-working version:

1 5 * * *       /bin/bash /path/to/ `date +%Y%m%d -d '5 days ago'`

When a crontab fails it will drop an email to user’s mailbox. So here are the relavant lines I found in the mailbox (via command mail):

/bin/sh: -c: line 0: unexpected EOF while looking for matching ``'
/bin/sh: -c: line 1: syntax error: unexpected end of file

With those error messages I googled a bit and changed the crontab to the following
working version:
1 5 * * * /bin/bash /path/to/ `date +%%Y%%m%%d -d ‘5 days ago’`

1 5 * * *       /bin/bash /path/to/ `date +\%Y\%m\%d -d '5 days ago'`
Categories: linux, Tip

Attach targeted window using tmux

August 5, 2011 Leave a comment

In tmux sometimes we might need to attach a session with the specific window selected so that we can start working without additional switching window steps. I just found one of the ways to accomplish this is:

tmux select-window -t sess:win && tmux attach -t sess

Credit goes to [ lines 78 & 79 ]

Categories: Tip, tmux

“Bulk” update in mysql with the use of temporary table

June 21, 2011 8 comments

You need to insert or update a large number of rows (at least hundreds of thousands) from an external data source (either from other database or file) into an existing mysql table but you are not allowed to truncate the table first because data source contains less number of columns than the existing table. The external source has a column that will be checked against the table’s primary key to determine if a record is new or not. For new records, using bulk insertion is pretty easy. Updating existing rows is quite a different story as there’s not yet an official way to do “bulk” updating. Prepare/Execute method might help a bit but still the performance is not satisfactory.

The solution:
Through a project I completed recently, I’ve learned a new way to solve the above problem, and the basic idea is to use bulk insertion to accomplish the goal of bulk updates, with the help of mysql temporary table. Let’s assume the table we are going to insert/update has the following structure:

tbl_A (
     userid char(20) not null,
     email varchar(250) not null,
     url varchar(250) ,
     flag_a tiny int,
     primary key(userid)

The basic steps:
1) Before entering the loop, create a temporary table tbl_A_tmp

create temporary table if not exists tbl_A_tmp (
   userid char(20) not null,
     email varchar(250) not null,
     url varchar(250) ,
     flag_a tiny int

Please note this table doesn’t have to have exactly the same data structure as tbl_A, as long as it holds the columns that will be used in step 5 and it’s also a good idea to remove any indexes to speed up insertion (to the temporary table).

2) Use delete or truncate statement to delete any existing rows in tbl_A_tmp

3) Inside the loop, save new records to an array list_new and existing records to an array list_existing, when num_of_elements(list_new) >=BATCHCOUNT, do batch insertion to tbl_A directly, when num_of_elements(list_existing) >=BATCHCOUNT, do batch insertion to tbl_A_tmp, clear list_new or list_existing after each batch insertion

4) When looping is done, Check remaining number of elements in both list_new and list_existing, perform batch insertions for any remaining records to the corresponding tables

4) Add unique or primary key to tbl_A_tmp, without this the next step will be very slow

5) Perform cross-table update like the following

    update tbl_A dest, tbl_A_tmp src set,
    where dest_A.userid=src.userid

The reason why temporary table is used is because of
a) speed, a temporary table is usually created in the memory and will be converted to use disk storage if the temporary table becomes too large (hence data loss is prevented)
b) convenience, when the db link is disconnected,temporary tables are dropped automatically

How much performance gain do I get by using this method? 45 minutes (bulk insertion for new records + row-by-row updates) vs. less than 2 minutes (bulk insertion for new records + temporary table method for existing records).

Categories: mysql, Programming, Tip

Comparing two mysql tables containing huge number of records each

June 15, 2011 Leave a comment

Usually I use the following method to compare two mysql tables with identical structure (or one contains the subset of columns of the other):

select * from db_A.tbl_data
where (col1, col2, col3, ...) not in
(select col1, col2, col3, ... from db_B.tbl_data);

select * from db_B.tbl_data
where (col1, col2, col3, ...) not in
(select col1, col2, col3, ... from db_A.tbl_data)

[ change * to count(*) if only the number of differences is needed ]

This method works pretty well in terms of performance when
1) number of columns to be compared is not too big, usually around 10
2) there are indexes defined in each table, this is very important when number of rows to be compared in each table is huge, I’lve done comparisons between 2M rows agains 2M rows, it can take minutes to finish, even with indexes.

But I ran into performance problem the other day when I tried to compare two identical-structured tables (comparing all 15 columns) with almost 3M rows each, and each table has only 1 index defined. Queries using the first method ran very slowly and I quickly used up patience ( I waited for more than half an hour and even the first query couldn’t got finished). Without other choices, I came up with the following solution:

a) log in to mysql server
b) select * from db_A.tbl_data into outfile '/tmp/outfile_A';
c) select * from db_B.tbl_data into outfile '/tmp/outfile_B';
d) exit mysql console
e) cd /tmp; diff outfile_A outfile_B > diff
f) wc -l diff

if the result from f) is small, I can open up file diff with an editor to see what the differences are, otherwise I can use head diff and/or tail diff to take a glance at the differences. With this method, I got the results in less than 3 minutes, which is a huge improvement compared to the first method.

Categories: howto, mysql, Tip

How to get out from tmux session inside a tmux window

June 8, 2011 7 comments

Here’s how I got stuck:
1) Remote access to a linux serverA
2) Launch tmux, Ctrl-b c to create a new window
3) From the new window, remote access to serverB, then run tmux att under shell to attach an existing tmux session (on serverB)
4) Now if I use the shortcut key Ctrl-b d, I can only detach from the session on serverA, because the tmux session on server B also uses Ctrl-b as the prefix

The fix:
a) Press Ctrl-b n until the window containing the ssh session to server B becomes the current window
b) Press Ctrl-b :
c) Enter tmux command set -g prefix C-a
d) Press Ctrl-b d, tmux session on serverB is now detached, type exit to close the new window created in step 2, this step is optional though as the “nested” tmux session is detached from the ssh shell
e) Press Ctrl-a :
f) Enter tmux command set -g prefix C-b

Step c changes prefix to Ctrl-a so it won’t conflict with the tmux shortcut prefix from serverB. Step f changes the prefix back to Ctrl-b for serverA.

Categories: howto, Tip, tmux

Comparing huge text files with rdiff

June 7, 2011 Leave a comment

I happened to need to compare two huge data files (plain texts), each of which is near 2GBs in size. When I use the good old diff program on a Centos with 2GB of Ram and 512MB of SWAP, pretty quickly I got stuck with this error: diff: memory exhausted.

A little googling pointed me to the following solution

Categories: howto, linux, rdiff, Tip