WARC (Web ARChive) is a file format for storing web crawls.
This warc library makes it very easy to work with WARC files.:
import warc
f = warc.open("test.warc")
for record in f:
print record['WARC-Target-URI'], record['Content-Length']
The documentation of the warc library is available at http://warc.readthedocs.org/.
This software is licensed under GPL v2. See LICENSE file for details.