A New York Times hacker, Derek Gottfrid, blogs about how he used Amazon Web Services to convert all the New York Times articles from 1851 to 1980, from scanned images to pdf. In his post, he says:
I then began some rough calculations and determined that if I used only four machines, it could take some time to generate all 11 million article PDFs. But thanks to the swell people at Amazon, I got access to a few more machines and churned through all 11 million articles in just under 24 hours using 100 EC2 instances, and generated another 1.5TB of data to store in S3. (In fact, it work so well that we ran it twice, since after we were done we noticed an error in the PDFs.)
Ain't it a beautiful thing? Just fire up 100 computers for 24 hours, then throw them away. And throwing them away is environmentally friendly...
0 comments:
Post a Comment