Archive
Ubuntu 11.10 performance tuning
I recently upgraded my netbook’s OS to Ubuntu 11.10 and noticed that it’s taking longer than usual to launch programs that I use quite often. For example, it takes 7-8 seconds to launch Google Chrome, about 4 seconds to open terminal. Here’s what I did to make the os run faster without investing with a SSD disk (My root partition is using ext4 fs, skip step 1 through 3 if you are using file systems other than ext3 or ext4):
1. Add the following bold part to the root partition line in /etc/fstab
errors=remount-ro,noatime,data=writeback
2.
tune2fs -o journal_data_writeback /dev/sda5
/dev/sda5 is the root partition.
3. change the following line in /etc/default/grub
GRUB_CMDLINE_LINUX=”rootflags=data=writeback”
(bold part is added)
4. Run update-grub
5. Add the following line to /etc/sysctl.conf
vm.swappiness=10
NOTE: all the above changes require root privileges.
6. Reboot
The difference? It now takes about 3-4 seconds to launch Google Chrome from cold start.
A simple node.js rss parser using sax-js
The xml parser sax-js written by Issacs (the creator of npm, the de facto package manager of node.js) comes with a few examples that deal with local xml files. I couldn’t find one that can parse xml data from remote host (think RSS) therefore I decided to write one. In this example I borrowed codes heavily from both sax-js’s example code and node-rss source code.
The codes
cat saxrss.js
var sax=require('sax');
var http=require('http');
var callback=function(){};
exports.get_rss=function(host,port,path, cb) {
callback=cb
var parser = sax.parser(true)
var item = null
var currentTag = null
var items=[]
var cnt=0
parser.onclosetag = function (tagName) {
var tag_name=tagName.toLowerCase();
if (tag_name === 'item' || tag_name === 'entry') {
currentTag = item = null
cnt++
return
}
if (currentTag && currentTag.parent) {
var p = currentTag.parent
delete currentTag.parent
currentTag = p
}
}
parser.onopentag = function (tag) {
var tag_name=tag.name.toLowerCase()
if (tag_name !== 'item' && tag_name !== 'entry' && !item) return
if (tag_name === 'item') {
item = tag
items[cnt]={}
}
tag.parent = currentTag
tag.children = []
tag.parent && tag.parent.children.push(tag)
currentTag = tag
}
parser.ontext = function (text) {
if (currentTag) {
items[cnt][currentTag.name.toLowerCase()]=text
}
}
parser.onend = function () {
callback(items)
}
var body='';
http.get( { host:host, path:path, port:port }, function(res) {
res.addListener('end', function() {
parser.write(body).end()
});
res.setEncoding('utf8');
res.on('data', function(d) {
body+=d;
});
});
}
cat test1.js
var rss=require('./saxrss.js');
var host='feeds.finance.yahoo.com';
// to get finance headlines about stock AAPL
var path='/rss/2.0/headline?s=aapl®ion=US&lang=en-US';
rss.get_rss(host, 80, path, function(items) {
console.log(items);
});
To run
node test1.js
Required node modules:
sax
References:
[ UPDATE 2/20/2012 ]
With xml-simple module, the above example can be written as
// getting xml and convert to json object using xml-simple example
var http=require('http'), simplexml=require('xml-simple'), config= {host:'feeds.finance.yahoo.com', path:'/rss/2.0/headline?s=aapl®ion=US&lang=en-US', port:80}, body='';
http.get( config, function( res ) {
res.addListener('end', function() {
simplexml.parse(body, function(e, parsed) {
console.log(parsed.channel.item);
//console.log(JSON.stringify(parsed));
});
});
res.setEncoding('utf8');
res.on('data', function(d) {
body+=d;
});
});
To install xml-simple, simply npm install -g xml-simple.
tmux techniques by example
I am a big fan of tmux – a terminal multiplexer. Think of it as a text version of vnc client, with many more powerful features. In this post I will demo some of the tmux techniques that I use quite often.
Assumptions:
1) GNU version of tmux
2) Default shell is BASH
Preparation:
For demo’s purpose I make up a dummytask.sh to simulate the task(s) that we will be running in tmux windows:
#!/bin/bash taskname=$@ ans='' if [ -n "$taskname" ]; then while [ ! "$ans" == "q" -a ! "$ans" == "Q" ]; do read -e -n 1 -p "Running task $taskname, to exit, press q(Q). " ans done echo "Done task $taskname." else echo "Usage: $0 task." echo "Example1: $0 debugging" echo "Example2: $0 importing data" fi
Example: create a tmux session with session name mysess
tmux new-session -s mysess -d
If the option -d (detached) is omitted, you will be taken directly to the first window titled “0:bash” once the command is executed and any commands afterwards will be entered into that window. Therefore it’s a good habit to use option -d whenever creating a new session.
Example: create a new tmux session and change the first default window title to task1
[ type q, Enter, exit, Enter if the sess tmux session is currently attached ]
In the first example, tmux will create a first window titled “0:bash” (could be ksh, csh etc depending on default shell setting) by default, to change to something else, simply using -n (name) option:
tmux new-session -s sess -d -n task1
Example: create tmux session mysess if it has not been created yet
tmux list-session 2>&1 | grep -q "^mysess:" || tmux new-session -s sess -d
Notes: 2>&1 is to suppress error output “failed to connect to server: Connection refused”, which occurs when there are no tmux sessions running. -q is used to suppress the normal output of of grep. It won’t affect the result but using it makes the commands less distracting. Regular expression ^sess: is used to make sure it won’t match session name such as “sessionabc” by mistake. Logical operator || is just a shorthand form of if [ ! condititon ]; then … fi.
Example: create a new window title mywin in an existing tmux session mysess, if the window has not existed yet, create the session first if it hasn’t existed yet
#!/bin/bash sess=mysess wn=mywin tmux list-session 2>&1 | grep -q "^$sess" || tmux new-session -s $sess -d tmux list-window -t $sess 2>&1 | grep -q ": $wn \[" || tmux new-window -t $sess -n $wn
Example: run a script in mysess:mywin in the above example
#!/bin/bash sess=mysess wn=mywin tmux list-session 2>&1 | grep -q "^$sess" || tmux new-session -s $sess -d tmux list-window -t $sess 2>&1 | grep -q ": $wn \[" || tmux new-window -t $sess -n $wn tmux send-keys -t $sess:$wn "./dummytask.sh cooking" Enter
How do we know the above script is doing what we intended to do? Check out the next example.
Example: attach tmux session mysess
tmux a -t mysess
Example: run a script in the first window of the newly created tmux session
tmux new-session -s mysess -n mywin "bash dummytask.sh cooking"
This works but there’s a problem, once you exit the program by pressing q or Q, tmux session also terminates. This gets more annoying when a program crashes and you don’t get any debug info, a better handling of the task will be provided in the following example.
Example: run program in a tmux window and exit to bash shell inside the window if the program exits or crashes
#!/bin/bash sess=mysess wn=mywin # duplicate session or window handling code here # ... tmux send-keys -t $sess:$wn "./dummytask.sh cooking" Enter
Example: how to check if a tmux session is attached
Sometimes it’s desired to run certain command not in a tmux window, use the following code to detect if attempt is made to run some command inside a tmux window:
if [ "$TERM" = "screen" -a -n "$TMUX" ]; then
echo "This command should be run when tmux is not attached"
fi
Example: attach a tmux session with a specific window selected
#!/bin/bash sess=mysess tmux list-session 2>&1 | grep -q "^$sess" || tmux new-session -s $sess -d wn=win0 tmux list-window -t $sess 2>&1 | grep -q ": $wn \[" || tmux new-window -t $sess -n $wn tmux send-keys -t $sess:$wn "./dummytask.sh task 0" Enter wn=winX tmux list-window -t $sess 2>&1 | grep -q ": $wn \[" || tmux new-window -t $sess -n $wn tmux send-keys -t $sess:$wn "./dummytask.sh important mission" Enter wn=win1 tmux list-window -t $sess 2>&1 | grep -q ": $wn \[" || tmux new-window -t $sess -n $wn tmux send-keys -t $sess:$wn "./dummytask.sh another thing" Enter # here's the meat of this script, select window winX before attaching the session tmux select-window -t $sess:winX && tmux a -t $sess
Javascript function chaining how to – a simple demo
<html>
<head>
<title>Rico's Appbox Demos | A simple javascript function chaining demo</title>
<script type="text/javascript">
(function() {
if(!window['Foo']) window['Foo']={};
var self=window['Foo'];
var name=null;
var description=null;
var income=-999;
function init(options) {
name=options.name||'no name';
description=options.description||'default description';
return self;
}
function bark() {
alert( 'Hi my name is '+name+' and I do ' +description+'.' );
return self;
}
function showincome() {
alert( "Shhhh don't tell anybody that my income is "+income+'.' );
return self;
}
window['Foo']['init']=init;
window['Foo']['bark']=bark;
window['Foo']['showincome']=showincome;
})();
function say_it() {
var thename=document.getElementById('id-name').value,
thedesc=document.getElementById('id-desc').value;
return Foo.init({ name:thename, description:thedesc }).bark();
}
function say_it_more() {
say_it().showincome();
}
function start_over() {
Foo.init({}).bark().showincome();
}
</script>
</head>
<body>
<p><input id="id-name" value="the one and only"></input></p>
<p><input id="id-desc" value="a lot of things"></input></p>
<p><a href="#" onclick="say_it();">Say It</a></p>
<p><a href="#" onclick="say_it_more();">Say It More</a></p>
<p><a href="#" onclick="start_over();">Start Over</a></p>
</body>
</html>
For a live demo, please visit ricosappbox.appspot.com/demos/jschain.
So long, MySQL
I’ve been playing around with PostgreSQL recently and I am total impressed by the features that PostgreSQL supports. I would like to point out the create domain statement (which confirms with SQL standard) that instantly makes me a MySQL betrayer: (from http://www.postgresql.org/docs/8.4/interactive/sql-createdomain.html)
CREATE DOMAIN us_postal_code AS TEXT
CHECK(
VALUE ~ '^\\d{5}$'
OR VALUE ~ '^\\d{5}-\\d{4}$'
);
CREATE TABLE us_snail_addy (
address_id SERIAL PRIMARY KEY,
street1 TEXT NOT NULL,
street2 TEXT,
street3 TEXT,
city TEXT NOT NULL,
postal us_postal_code NOT NULL
);
The main reason: Data validation is made easy and powerful. With MySQL, data validation relies heavily on the application that interacts with the tables.
A mistake to avoid when calling jQuery.ajax
I can’t think of a reason why jQuery.ajax accepts both object and query string as the valid format the data parameter, for example,
$.ajax( {
url:'/someurl',
type:'POST',
data: { key1: val1, key2: val2 },
// alternatively, the following format is also accepted
// data: 'key1='+val1+'&key2='+val2,
success: function() {
alert(d);
}
} );
In the above example, the query string format is also accepted but should be discouraged to use, reason of which will be demonstrated with the following example. We need to submit data in the text fields as is to the server. When query string format is used, data is passed to the server without being encoded properly, as you can see from the first screenshot. For demonstration purpose, password field uses “text” as its input type.
index.html
<html>
<head>
<title>Rico's Appbox Demos | A mistake to avoid when using jQuery.ajax() function</title>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js"></script>
<script type="text/javascript">
function do_post( data ) {
var io='Sent from client:\nusername: '+$('#username').val()+'\npassword='+$('#password').val();
$.ajax( {
url: '/demos/jpost',
type:'POST',
data: data,
success: function(d) {
io+='\n\nReceived by server:\n'+d
alert(io);
}
});
return false;
}
function post_qs() {
do_post('username='+$('#username').val()+'&password='+$('#password').val());
}
function post_json() {
do_post({
username: $('#username').val(),
password: $('#password').val()
});
}
</script>
</head>
<body>
<p>
Username: <input id="username" type="text" value="This?is+ok">
</p>
<p>
Password: <input id="password" type="text" value="asdf%20test">
</p>
<p>
<a href="#" onclick="post_qs()">Post via query string format (BAD)</a><br />
<a href="#" onclick="post_json()">Post via json format (GOOD)</a>
</p>
</body>
</html>
Live demo is available at http://ricosappbox.appspot.com/demos/jpost.
The screenshots below are slightly different to the live demo result because they were created when I used php as the back-end to print the post data.
Screenshots:
When first link (using query string format) is clicked:
When the second link (using json format) is clicked:
Node.js: improve mysql query performance with mysql-pool and cluster
Prior testing:
npm install mysql -g npm install mysql-pool -g npm install express -g npm install cluster -g
Code that archives best result:
require.paths.push('/usr/local/lib/node_modules');
var express=require('express');
var cluster=require('cluster');
var MySQLPool = require("mysql-pool").MySQLPool;
var pool = new MySQLPool({
poolSize: 10,
user: 'node',
password: 'node',
database: 'testdb'
});
function symsearch(q, res) {
var query=pool.query(
"select symbol,company_name from symbols where symbol like ? limit 10",
['%'+q+'%'],
function selectcb(err, results, fields) {
if(err) {throw err;}
console.log("Searching Result for "+q);
res.send( results );
}
);
}
var app=express.createServer();
app.get('/sym/:q', function( req, res) {
var q=req.params.q || '';
if(q.length>0) symsearch(q, res);
});
cluster(app).listen(3000);
Benchmark command:
ab -n 10000 -c 250 http://localhost:3000/sym/<pattern>
Results:
With the js code above, I am able to achieve 1300/sec on a dual-core 2.9GHz Virtualbox vm (ubuntu 10.10 32bit). When the same test is conducted on Apache2 + php5, I am getting about 800/sec. If I opt-for the single connection mode (use ‘mysql’ instead of ‘mysql-pool’), the result is only a little better than apache + php combo. The use of node module cluster is to take advantage of the dual-core cpu (node would run with in single-core mode due to its single-threaded nature).
Rico's Tech Memo 