Brunei Times E-paper Downloader

Update (2010/08/10): It seems that The Brunei Times has changed their online newspaper to use SWF files instead of JPG files so you can’t easily view the files anymore =(. The download script has been changed to download SWF files
——————————————————–
So I love the fact that The Brunei Times has a great looking website, started Twittering, had video news and most importantly has an E-Paper which is an exact replica of their actual newspaper available online. It’s great that they allow people to read the papers in such a way, but my distaste for Flash led me to be curious of how to get the actual images displayed for offline reading. While looking in Firefox’s web cache I found the URL of the pages:

Page 1 default view: http://www.bt.com.bn/e-paper/images/page1.jpg
Page 1 zoomed in: http://www.bt.com.bn/e-paper/images/page1_zoom.jpg

Armed with that knowledge I started building a script that would let me download these files onto my computer for reading at a later time and possibly even copying the files to my phone and reading it there.

Requirements:

Usage:

  • Double click the download script to download the news for the day. This will download the news into a folder of the given date
  • Open the folder and read the news

Screenshot of downloaded files

Using wget to resume downloads that require cookie credentials

I’ve been having an issue recently (not sure if it’s the Mac) but it has indeed become very apparent to me, that I would download things (applications / videos / ISOs / etc) in Firefox and the download would ‘finish’. It shows it as completed in the downloads window but in fact it is a truncated download. As seen in the screenshots below the SDK finished at 1.6GB when in fact the actual full file size is 2.1GB.

Misleading completed Firefox download

I initially tried using wget to download the file but got a ERROR 403: Forbidden error probably due to some cookie credentials. So not wanting to waste my 1.6GB download that took about 12 hours I resorted to looking into getting wget to work with cookies and was happy to see that the folks who created wget indeed had an option to load the cookies file. Sweet! Very sweet indeed. So next task was to get the cookies out of Firefox which was a bit more of an issue.

Export Cookies was known to work but it was not compatible with Firefox 3.5 so I had to download the Nightly Tester Tools to allow Export Cookies to work. Export the cookies (“Tools > Export cookies”) and in Terminal/Command line and enter the wget command to resume the download as follows:
wget --server-response --continue --load-cookies cookies.txt http://location/of/download

Download wget (Windows | Mac)