Advertise on ITWeb         Tue, 21 May, 05:41:09 AM

Moving truckloads of data to the cloud

The AWS Snowmobile can hold up to 100PB of data in its 45-foot shipping container pulled by a semi-trailer truck.

The AWS Snowmobile can hold up to 100PB of data in its 45-foot shipping container pulled by a semi-trailer truck.

Amazon Web Services (AWS) can now move truckloads of data to the cloud with the launch of its new Exabyte-scale data transfer service – Snowmobile. 

The new data transfer service was unveiled by CEO Andy Jassy at the AWS re:Invent Conference in Las Vegas last week and its purpose is to move extremely large amounts of data to AWS. Up to 100PB can be transferred per Snowmobile, which is a 45-foot-long, rugged shipping container pulled by a semi-trailer truck.

"You would not believe how many companies now have exabytes of data that they want to move to the cloud, but moving exabytes of data before was completely unreasonable and impossible," Jassy said during his keynote address to the 32 000 attendees of the conference.

AWS believes Snowmobile is a game-changer in the market because, as Jassy explained, moving an exabyte of data (1 000 petabytes or one million terabytes) using a 10Gbps dedicated connection would usually take 26 years. With Snowmobile, it would be possible in only six months (using about 10 Snowmobiles).

Local analysts, however, are not convinced this service is essential in the South African market, or anything particularly new.

Clinton Jacobs, IT sector senior analyst at BMI-TechKnowledge, says he is not sure how many companies locally would have this size requirement to need Snowmobile.

"Given exponential growth of rich content and big data, there are definitely use cases and requirements for faster data transfers. There are transfer technologies such as FASP, an IBM technology that a local company Montana Data Company provides, for data transfer as well as high-efficiency video coding for video."

Jon Tullett, IDC's research manager for IT services, Africa, says moving giant datasets into the cloud is still prohibitive in many instances as throughput is the constraint.

"As the saying goes, a jumbo jet full of hard disks has terrible latency but phenomenal throughput. In some big data applications, this doesn't actually help – if you're dealing with a lot of data that moves rapidly, latency is the constraint as much as throughput. But for large static datasets, for archival or long-term analysis, it's an important capability.

"I would question how many customers will need the scale of ingress offered by Snowmobile vs Snowball, but presumably it's enough for it to be more than a gimmick," he adds.

Snowmobile is the successor to AWS' 2015 launched petabyte-scale data transfer service "Snowball", a pre-packed hard drive that could store 50TB of data. A new version "Snowball Edge" was also unveiled at 2016 re:Invent. It can now hold twice as much data as the original Snowball – up to 100TB – and has four times the network speed.

Snowball Edge can transport 100TB of data in less than a week, according to AWS. One Snowmobile is the equivalent of using about 1 250 Snowball devices.

"Snowmobile is Snowball on steroids, basically – they used to ship you a bunch of disks in a NAS enclosure and you'd copy data on and ship it back. This is the same thing, only way bigger; it's not really a new concept," according to Tullett.

Cloud shifts 

Jacobs says BMI-T estimates the cloud services market in SA was in the region of R2.6 billion last year, with growth in excess of 20%. However, local adoption is still slower than global adoption.

"There is definitely a shift to cloud services, as companies are becoming more comfortable with the idea of not having their data within arm's reach, provided they have the security assurances and it provides a cost benefit," he explains.

"Amazon is the dominant force in public cloud and is expected to remain so; Amazon's ongoing infrastructure investment is very large and it's putting more resources into growing its capabilities than its competitors are," according to Tullett.

"But just because they have the largest slice of the pie doesn't mean there's not enough pie to go around. The pie's growing, and the other vendors have no shortage of opportunity, even if their share is likely to remain quite a bit smaller than Amazon's for the time being," he adds.

AWS has 38 availability zones across 14 geographic regions but none of the regions are Africa-based and there are no set plans to change this anytime soon.

Jassy told journalists in Las Vegas that Africa is not on the company's current disclosed growing regional roadmap, "but we have a lot of places that we plan to have regions over time − and it's not very hard to imagine one of those being a region in Africa".

In August 2015, AWS opened an office in Johannesburg to support its growing customer base in the country. The office is an addition to the Amazon Development Centre in Cape Town, which has been in operation since 2004.

Our comments policy does not allow anonymous postings. Read the policy here




Sponsors Message