Archive for the ‘wget’ Category

A few wget tips

June 2, 2011 Leave a comment

In *nix world, wget is a very handy tool to download files from remote ftp or http servers. This post will present a few wget tips that I use quite often in my (mostly Bash) scripts:

1) use –quiet option to surpress download progress indicator
wget --quiet http://host_name_of_the_server/path/to/afile.doc

2) use -N to get file only when timestamp or size of the file downloaded has changed
wget -N http://host_name_of_the_server/path/to/somefile.dat
If file somefile.dat has been downloaded and file size and/or timestamp has not changed on the remote server, downloading of the file somefile.dat will be skipped to save bandwidth (and disk io).

3) when running wget under bash, one can take advantage of Bash curl braces expansion and do something like this
wget http://host_name_of_the_server/path/to/{file1.txt,file2.txt,file3.txt}
wget http://host_name_of_the_server/path/to/{file1,file2,file3}.txt
or even
wget http://host_name_of_the_server/path/to/file{1,2,3}.txt

Each of the above commands is equivalent to the following
wget http://host_name_of_the_server/path/to/file1.txt
wget http://host_name_of_the_server/path/to/file2.txt
wget http://host_name_of_the_server/path/to/file3.txt

With less code you get more done.

Categories: Bash, Tip, wget