Today, Etsy uses Amazon Elastic MapReduce for web log analysis and recommendation algorithms. Because AWS easily and economically processes enormous amounts of data, it’s ideal for the type of processing that Etsy performs. Etsy copies its HTTP server logs every hour to Amazon S3, and syncs snapshots of the production database on a nightly basis. The combination of Amazon’s products and Etsy’s syncing/storage operation provides substantial benefits for Etsy. As Dr. Jason Davis, lead scientist at Etsy, explains, “the computing power available with [Amazon Elastic MapReduce] allows us to run these operations over dozens or even hundreds of machines without the need for owning the hardware.”
Dr. Davis goes on to say, “Amazon Elastic MapReduce enables to focus on developing Hadoop-based analysis stack without worrying about the underlying infrastructure. As cycles shift between development and research, software and analysis requirements change and expand constantly, and [Amazon Elastic MapReduce] effectively eliminates half of scaling issues, allowing to focus on what is most important.”
Etsy has realized improved results and performance by architecting their application for the cloud, with robustness and fault tolerance in mind, while providing a market for users to buy and sell handmade items online
How Cloud Computing Helped Netflix Emerge as a Streaming Media Powerhouse
Netflix may be getting a lot of bad press in recent times due to its management’s ill-advised decision to raise subscription rates by almost 50% resulting in widespread customer dissatisfaction and a groveling apology by CEO Reed Hastings, but it was not long ago that it was considered the epitome of home entertainment.Netflix is another new-age company that owes its success to cloud computing, the same way that Zynga, the creators of Facebook game sensation Farmville, does (See: Zynga, the Latest Cloud Computing Success). And not surprisingly, for both of them, the cloud provider of choice is Amazon, perhaps the earliest player in the game.
Although Netflix began life as a DVD-by-mail service in 1997, it was with the introduction of the on-demand streaming service that it saw a huge expansion of its customer base. In fact, when it crossed 10 million subscribers in 2009, it “attributed the recent surge in subscribers to growing consumer recognition of the value and convenience offered by Netflix and increasingly more ways to instantly watch a growing library of movies and TV episodes from Netflix on PCs, Intel-based Macs and TVs.”
Not surprisingly, this model was soon adopted by many other providers like Fox’s Hulu, Amazon and even Google, who created a paid version of YouTube content. Now, running such a service required a level of flexibility, resource optimization and redundancy that traditional data centers were ill-equipped to provide (See: Virtualization: The Virtual Way to Real-World Benefits). That is why Netflix today relies almost exclusively on cloud services for its infrastructure.
This point was reiterated by Netflix Cloud Security Architect Jason Chan in his presentation “Practical Cloud Security” at the United Security Summit in San Francisco. During his presentation, Chan articulated the advantages that being on the cloud provided Netflix, advantages that were not possible with traditional IT infrastructure.
Chen explained that in a traditional data center, applications are long-lived, code is pushed to running systems, and it can be difficult to enforce deployment patterns such as patches. However, on the cloud, new versions are written which replace the old ones entirely with new instances, eliminating the need for patches. Also, while earlier repeatable tasks such as adding a user account, changing firewall configurations or forensic analysis required multiple steps and interfacing with multiple systems, “these tasks are a simple API call with cloud.” Moreover, with systems being added to groups that control the connectivity, “there’s no one chokepoint” like the traditional firewall.
“The key lesson learned is you have to leave the old ways behind,” However, that moving to the cloud did introduce some specific security concerns that had to be addressed. With Amazon having launched a similar service in direct competition with its own customer Netflix (See: Is Amazon’s Cloud Player a Game Changer in the Music Industry? ), this space should see some interesting developments in the near future.