This last week has proved to be quite a week for the EPUB eBook format with announcements from Google, Sony and Project Gutenberg on their support for the EPUB format.

Project Gutenberg EPUB Books

Over at Project Gutenberg, Marcello Perathoner has been working hard to convert all the Gutenberg titles into the EPUB format. At this time these versions should be considered experimental, but after trying several different titles, they are all more than readable.

The books are converted where possible from the HTML version in the Gutenberg archives and for those titles without a HTML version, Marcello uses the plain .txt book. The plain .txt files at Gutenberg are notoriously inconsistent in their layout so converting these accurately is extremely difficult — I know this myself only too well. Perhaps it’s time Project Gutenberg embraced a Master Format.

What makes this special from the other news (see below) is that all the Gutenberg books go through a proofreading process and so the accuracy is very high. This is why so many other eBook project are based on the Gutenberg archives.

Google and Sony partner to release 500,000 Public Domain EPUB Books

Over the last few years Google has been scanning bo0ks by the million, making them available on their book search, but this is the first time they have any of them available to an eBook reader. All the titles are in the public domain (pre-1923 titles only) and once added to the current Sony Reader catalogue, brings the total available titles to around 600,000, far surpassing Amazon Kindle’s 240,000 catalogue. Amazon still use their own propriety eBook format and do no currently allow EPUB files to be read natively – conversion is needed first – but as the Google EPUB books all come DRM free, there are many tools out there that will allow you access to these on a Kindle or other reading platform.

As the press release states, “the publishing industry has more or less united on EPUB for e-book distribution”, so c’mon Amazon, let’s see some native EPUB support!

Don’t have a Sony Reader?

Although this announcement is aimed at Sony Reader owners, you don’t need to own a PRS-505 or PRS-700 to read these Google EPUB books. Just download the Sony “eBook Library” software to search and download. Paul Biba’s article over at TeleRead shows how to access the books without owning a Reader.

No access for non-U.S. based PRS-505 owners

Yet again, this is a U.S. only release, at least if you already own a PRS-505. You can only access the Google books via Sony’s “eBook Library” software, but in my UK version we don’t have access to the Sony Book Store as we are only pointed to the Waterstone website. When I tried to install the new version (2.5) it tells me it cannot install in my region! I believe it is because I already have my UK reader registered and if I were to install on a second computer I would then have access to these titles.

If there are any non U.S. PRS-505 users out there who have succeeded in accessing the archive, please let me know.

Google EPUB Quality?

As Google does not proof read their scanned books, relying completely on the OCR (Optical Character Recognition) software to produced the digital characters, the quality of the text themselves is not as good as archives like Project Gutenberg or websites that take their books from PG, such as Feedbooks.com. Then again, were are getting 500K free books with the potential to access another million Google books in the future.

All in all, this has been a good week for the EPUB format.

Over the last few weeks several people have emailed asking if epubBooks.com has been abandoned – the answer is a resounding No! Okay, I know I’ve not been very active recently so please accept my apologies for that.

The reason for such limited activity is that I am working very hard on developing the new site for epubbooks.com – yes, the blog and current resources will still be accessible. The new site will allow you to download all kinds of different EPUB books, including many from the Project Gutenberg archives.

If you don’t know what my ePub Books Project is, here is a short summary.

The project was started to provide free downloads of nicely formatted EPUB files, the majority of which will be taken from the Project Gutenberg archives. These will not just be plain TEXT files enclosed in an EPUB container, but fully converted to XML (of a TEI flavour) which are themselves converted using XSLT into professional quality EPUB files. Here are some of the features;

• Properly formatted and displayed Chapter Titles/Subtitles.
• Footnotes which are Linkable (forward and backward) for instant access
• Books with Illustrations will also be available.
• Text Formatting (italics, etc.)
• Nice indents for block quotes, letters of correspondence, epigraphs, etc.
• …and many more features

The new web application is the biggest project I’ve developed to date and so is naturally a challenge to my programming skills, which is why it’s taking some time to complete, however things are going very well.

The basic skeleton of the site up and running and I am now working on programming for usability. Of course it’s these less obvious items which are some of the hardest things to programme, so at this time can’t give an accurate launch date. Rest assured it will be sooner rather than later.

Thanks for your patience and understanding and do keep checking back regularly for any new updates.

My first suspicions that eBooks were going to be the future was way back in the day when all those CD’s were coming out and taking over our beloved Vinyl LP, but the real light-bulb moment was when I first discovered Project Gutenberg sometime at the end of the 1990’s.

If memory serves me correctly, I even considered trying to set up an eBook site back then. I believe what stopped me was that there were just not yet any decent reading devices available – reading from a computer monitor was, and still is, the most uncomfortable experience ever.

And so the years rolled by…

Then in 2004/05 I heard about the Sony Librie and immediately knew that the eBooks’ time was coming…and soon!

It took me a couple of years to get the project off the ground, but toward the end of 2006 I was seriously working out how to become a part of the eBook revolution. At this time I also started as the Project Gutenberg Newsletter editor which allowed me to get among, and learn from, those who’d been there right from the start.

I spent many months trying to figure out what eBook format would be best suited as my Master Format but after much research and some brain picking, I had my shortlist; Jon Noring’s BookX, TEI and the OPS (ePub) format.

At the time I actually rejected the ePub format as I felt trying to manage all those individual files would be just too much trouble — for my own tastes I still do. I liked the concept of BookX but felt my XML and DTD skills, which were non-existent at the time, would make it difficult.

TEI as a Master Format

I choose TEI not just because there is plenty of documentation, but also because Project Gutenberg was showing indications (albeit reluctant) that this could be an accepted format in their archives on a mass scale. Even if PG won’t accept it as a Master format, you know those PG volunteers are going to keep on producing TEI eBooks for the archives.

Another thing that really attracted me to TEI was that it utilises the ODD; One Document Does it all.

I’ve continued to keep eye out for alternate formats, even considering DTBook at one point, but I guess by sticking with the TEI format I can eventually make my files available for inclusion into the PG archives, so I stayed put.

In April 2007 I started teaching myself Perl and so work began on my pg2tei.pl script. The Gutenberg.org webmaster (Marcello Perathoner) kindly allowed me to use his gut2tei.pl script as a starting point and although the basic structure and a handful of routines remains the same, I’ve rewritten much of the original and added numerous new routines.

From this I created a sample base of around 70 PG eBooks in the TEI P4 format, converting over to TEI P5 when this was released in November 2007. I continue to make improvements and fix bugs wherever possible.

Alas the script is not fully automated, although it does catch most things. The biggest manual work required includes;

• <teiHeader> – Mostly automated but does need to be double-checked. The odd error happens so editing and adding changes for missing information is a must – luckily this normally takes just a minute or two, although the more awkward documents can take longer.
• Images – The inclusion of the <figure> tags are automatic but as none of the original TXT documents include filenames, I have to manually work out which image goes with which <figure> tag. This can be quite time consuming.
• Quote tags – This is probably the biggest consumption of time in the whole process. Although 99.99% of the <q> tags are correct, fixing that tiny percent can add many minutes to a conversion. Several times I’ve considered omitting the <q> tag mark-up, either leaving the original “double” and ‘single’ quotes in place or just replacing with a quote entity. However, I still feel the versatility this can offer makes them well worth the work.

The final output from this whole process produces what I call my Super-Lite TEI; creating a set of around 22 TEI tags (excluding the <teiHeader>, <front> and <back> sections) and no more than a dozen attributes.

In the final article of the ePub Books Project, I’ll talk about the plan to convert to the ePub format and the future of ePubBooks.com website.