Archive

Archive for the ‘rdiff’ Category

Comparing huge text files with rdiff

June 7, 2011 Leave a comment

I happened to need to compare two huge data files (plain texts), each of which is near 2GBs in size. When I use the good old diff program on a Centos with 2GB of Ram and 512MB of SWAP, pretty quickly I got stuck with this error: diff: memory exhausted.

A little googling pointed me to the following solution

http://beerpla.net/2008/05/12/a-better-diff-or-what-to-do-when-gnu-diff-runs-out-of-memory-diff-memory-exhausted/

Advertisements
Categories: howto, linux, rdiff, Tip