Amazon Transfers Data Faster than the Speed of Light?

Actually, Amazon doesn’t transfer data faster than the speed of light. But Amazon Web Services (AWS) does have a high-speed internal network that will move customer data faster than the internet. To the dilettantes in our midst, that would imply a transfer rate faster than the speed at which electromagnetic radiation travels, which in theory should be the same as the speed of light. And similarly, it should be the speed at which signals pass through the “ether of the internet”. However, this isn’t quite the case.

Data transfer: passing through the historical aether

Amazon’s breakthrough technology, providing both storage and processing services “in the cloud” is not particularly recent news. The beta release occurred over a year ago. The salient part of the story is here:

We are excited to announce the limited beta of AWS Import/Export, a new offering that accelerates moving large amounts of data into and out of AWS using portable storage devices for transport. AWS transfers your data directly on and off storage devices using Amazon’s high-speed internal network and bypassing the Internet. For significant data sets, AWS Import/Export is often faster than Internet transfer and more cost-effective than upgrading your connectivity.

AWS is now rolling out the Import/Export for Amazon S3, a premium storage solution designed for mission-critical and primary data storage. What are the portable storage devices referred to above? Any storage device with a USB or eSATA connector, that draws power from a U.S.-standard wall socket plug, 120 Volts at 60 Hertz. Amazon’s high-speed internal network is not electronic: it is internal combustion powered. Yes, the reference to the “Amazon high-speed internal network” is the Amazon internal network of vehicles, probably trucks. Really quite sensible of Amazon. The AWS Import/Export Calculator helps users decide the cost-equivalency between Amazon S3 data transfer charges and time versus directly loading data from an AWS Import/Export facility, at rates as fast as 500 Mbps.

If you have large amounts of data to load and an Internet connection with limited bandwidth, the time required to prepare and ship a portable storage device to AWS can be a small percentage of the time it would take to transfer your data over the internet. If loading your data over the Internet would take a week or more, you should consider using AWS Import/Export.

Why not combine old and new technologies to give the best possible service to your customers? AWS is offering free S3 data transfer in through November 2010. After that, the AWS Import/Export service should become yet more interesting as an option to increase price-performance for data storage in the clouds.

LATE BREAKING NEWS FLASH: Check out these AWS sticker photos! Delivered the old-fashioned way: by internal combustion, jet engine and/or freight train powered snail-mail. And if you want some too, you merely need to do as suggested by AWS Evangelist Jeff Barr and they WILL be yours. If only all wishes were so easily granted.

Transitory nature of information technology

Are we losing the means to preserve an enduring research trail? The premise is that due to the multiple forms of communication between academics and developers, and a lack of digital preservation standards, the steps leading to past scientific discovery and technological innovation will be lost.

Why is this re-creation, even documentation, so important?  First, for history of science and secondly, for innovators to be, coming through the pipeline. Not-yet-arrived scientists will want to study the development process. Sometimes what appears to be a flash of inspiration is preceded by months, or years, of reading, analysis or experiments. Documentation is important for understanding creative research design. Relatively easy access to successful examples from the past is a necessity.

The pace of innovation is wonderfully fast!

Representations like Alan Warburton‘s video, Format: A Brief History of Data Storage always makes me feel a frisson of delight, shiver of awe. It has great music too, Short Like Me by Beni (Kitsuné Maison).

More than information overload

Data deluge swamps science historians is an eyebrow-raising news story about the collected research materials of the world’s leading evolutionary biologist, William Hamilton, following his demise. This is more than a problem of information overload. When the British Library received Hamilton’s working papers, they were faced with assembling the contents of

• 200 crates of handwritten letters, draft typescripts and lab notes,
• 26 cartons of vintage floppy disks, reels of 9-track magnetic tape, and 80-column punch cards, but no devices that could read these archaic storage media

It was enough to convince me that we need better digital preservation and archival standards.