The Amazon logo sits on a cart parked in the inbound goods zone at the Amazon.com Inc.centre in Poznan, Poland.Bartek Sadowski/Bloomberg
Like many Type A technology executives, Matt Wood, who's general manager for product strategy at Amazon Web Services (AWS), has done a lot of impressive things most ordinary people probably can't understand—even though he's just 36 years old.
After graduating from medical school in England, Wood went on to do a PhD in bioinformatics (applying software and computing power to biology—in his case, the formation of proteins). The fast-talking Brit then did some postdoc studies in Big Data at Cornell in New York State, before returning to the United Kingdom to work at the prestigious Wellcome Trust Sanger Institute.
Wood arrived in 2006, as scientists were finishing the Human Genome Project (a collaborative global research effort to map the DNA blueprint of humankind), and just as a quantum leap was occurring in how quickly computers could map out DNA. "It moved from being able to do one genome in 10 years to being able to do one in about 10 days," Wood says. "The flip side of that was although you could sequence more DNA, these machines generated just tons and tons of data." By tons of data, he means several hundred terabytes each week (one terabyte is 1,000 gigabytes; even an iPhone 6 with maximum augmented memory has just 128 gigs).
But the institute is situated in the bucolic English countryside, about 14 kilometres south of Cambridge (near villages with names like Ickleton, Great Chesterford and Little Walden), and there was no way to get more power onsite to add more servers to its jam-packed data centre. The little town of Hinxton couldn't really cope with the dark, satanic mill-like server farms increasingly required in 21st-century business and research.
So Wood fired off an e-mail to some of his friends at Amazon, outlining the problem. They asked: Had he not heard of AWS, the cloud computing arm of their almighty e-commerce behemoth? Wood signed up and Amazon gave him a credit for $300 worth of data storage and service, which he used to buy digital space for some of the most cutting-edge DNA research in the world (what he calls a "genomic assembly pipeline")—all of which could now be stored up in the so-called cloud. In those days, cloud storage was relatively novel, but it worked brilliantly.
"I was a convert, basically, from that moment," says Wood. He was so impressed that he joined the company in 2010 (after selling a start-up). He now helps develop the hundreds of sophisticated services and tools that AWS has layered on top of its basic cloud storage and computing functions.
Even 2010 now seems like it was a generation ago. Since then, the cloud computing business has exploded, and AWS is growing faster than any of its rivals (including Microsoft and Google, as well as more specialized competitors like Salesforce). AWS's revenue has soared from less than $1 billion (U.S.) the year Wood arrived to $7.8 billion in 2015, and it is climbing by about 70% a year.
Much of that expansion has been fuelled by a new wave of tech start-ups, such as Airbnb and Pinterest. But even massive corporations and security-conscious government agencies have signed up, including General Electric, Novartis, Boeing, NASA and the CIA, as AWS added more functions and analytics. As with Amazon's online retailing and delivery businesses, the exciting—and scary—thing is that the growth now seems to be feeding on itself.
*****
Anyone who has ever thrown documents, photos or videos into Dropbox has done basic cloud computing: You're storing things in servers that you don't own, quite safely and securely, even if you don't have a clue where those servers are. In the early 2000s, old-guard firms such as IBM, and newer ones such as Rackspace, were already renting digital space on servers to clients.
The next step was to integrate the data storage and manipulation, and move it all to the cloud. That would allow companies to get rid of servers, network hardware and software, and most IT staff, leaving only equipment that employees need to access the cloud services. It is now the reality in many companies. Employees arrive at work and can log in to their own desktop on any screen. Departmental and office-wide systems are in the cloud, too. All transactions are tracked, and the system can alert customer service reps if there are problems. Companies can build, test and run apps in secured digital environments, shift entire libraries of data, and grant or restrict access to individuals.
So why is AWS bigger than its rivals and growing so much faster? Amazon launched AWS in 2006, and it quickly moved far beyond just providing a lot of storage space. It soon built out an entire platform with more and more flexibility. In 2014, AWS developers, working from a road map that is 90% related to specific customer requests, created more than 500 new features. Last year, that number jumped to more than 700—about two per day.
Much of the drive to dominate comes from Amazon CEO Jeff Bezos, of course. His overriding goals and obsessions are now familiar: making things as simple and effortless as possible for customers, constantly searching for new ways to disrupt the status quo, and driving costs and margins as low as he can to squeeze competitors and keep expanding revenue.
Yet, like Apple with iPhones, AWS has also become much more than a generic provider. It is a megabrand—a community, even. Many of AWS's customers now flock to its annual re:Invent conference in Las Vegas. From about 3,000 attendees a few years ago, the event has grown to almost 20,000, with nearly 40,000 others nerding it up via live streaming. AWS executives, and clients like Airbnb, get up onstage and do Silicon Valley-style multimedia presentations, filled with the wonderful industry-disrupting and paradigm-shifting jargon of tech.
Sheer size also allows AWS to throw a lot of new products and features at the wall to see what might stick, and to learn from those that don't. This, too, is fairly standard procedure at Amazon. Over the years, it has had several failures, including A9, a search engine and advertising application launched in 2003, and BlockView, which was surpassed by Google's Street View.
Despite AWS's now massive size and success, Amazon remains secretive about many aspects of its operations. The company only began breaking out AWS's sales and profits separately in its financial statements in the first quarter of 2015. That was partly because it was becoming almost obvious to analysts just how fast AWS was growing. Its numbers used to be included in Amazon's "Other" category, and that had ballooned over the past several years (see chart).
But Amazon still keeps many other key details under wraps. A PR rep wouldn't allow me to see a data centre—saying she herself had never been inside one. AWS will only say its employees number "in the thousands," but won't say where they work or break down what they do. A LinkedIn search reveals that 9,654 people list AWS as their current employer, out of Amazon's total of 230,800 full- and part-time employees in the fourth quarter of last year. For more specifics on what AWS actually does, you need to ask its customers.
*****
Saeid Fard is a friend of mine in Vancouver who's in the tech business. He's president of Sokanu, which provides online career tests. When I asked him to suggest any start-ups that might be AWS clients, he said I could pick them at random: "Pretty much every start-up uses AWS."
Phil Menary is fairly typical. He's the vice-president of development at Waterloo-based Axonify, which provides large companies such as Walmart, Toys "R" Us, Toyota and Telus with customized, three-to-four-minute online training videos for their employees, on desktop and mobile devices.
At a tech start-up Menary worked at in the early 2000s, there was a fairly big data centre: dozens of servers, raised floors to hide multicoloured criss-crossing wires and a cooling system to ensure that the servers didn't overheat. At least one employee was assigned to monitor that physical infrastructure at all times, and there were several other dedicated IT staffers.
But when Axonify launched in 2012, it was based entirely on AWS cloud technology. All Axonify's back-end systems are in the cloud, and it relies on Amazon's data centres for delivery systems to keep latency (wait times) for videos below one second per click. Axonify has about 60 employees, but it doesn't even have a dedicated e-mail server on-site, and it doesn't have anyone taking care of physical infrastructure.
"We'd never go back to having our own physical data centres," Menary explains. Even if Axonify wanted dedicated servers of its own, "there are data centres you can rent space in—the Rackspaces of the world—where you're renting or leasing servers." But why bother, when you can do everything in the cloud? "Not to pick on Rackspace," he says, "but even in that model—with physical equipment in a data centre—we'd need another dedicated person."
One of the most useful AWS tools for Axonify is CloudFront, a global digital-content web service. Most of Axonify's data is stored on servers in a large AWS facility in northern Virginia. But so-called edge servers in CloudFront's cloud-computing centres around the world then shoot high-definition videos to clients, and Axonify is billed on a pay-as-you-go basis. Amazon has regional hubs in Oregon, northern California, São Paulo, Ireland, Frankfurt, Singapore, Sydney, Beijing, Seoul and Tokyo, with more sites being built in Ohio, the United Kingdom, India, China's remote Ningxia Hui Autonomous Region and Montreal. Axonify clients such as Johnson & Johnson or Walmart have employees scattered across continents. "As a global solutions provider, it's really important to maintain a sub-second user experience," says Menary.
Axonify also relies on AWS's EC2 core computing platform (the acronym stands for Elastic Compute Cloud). EC2 made it possible for the company to develop its own "drill-down analytics," as Menary calls them, to track and study millions of customer interactions. Axonify can resize its computing capacity almost instantly, too. If it looks like it will need more storage space or computing power, employees can click and buy it. Resizing can even be automated, through what Amazon calls autoscaling, although Menary prefers that his developers keep monitoring server and storage functions.
Safety is another advantage. Most of AWS's regions have several data centres, so AWS can back up data and functions, and shift things quickly in case of power outages or other disasters. Its clients can move their data around, too. "Our primary data centre is in northern Virginia, and our secondary is in Oregon," says Menary. "If there's a storm and it looks like Virginia's going to be out for three days, then we just turn on Oregon."
All this means that Menary's team of 17 developers can spend pretty much all of their time on Axonify's products. AWS's price is also right: "I suspect that our entire Amazon bill is less than one full-time equivalent salary," he says. "If you were trying to manage your own system, you'd have server costs plus a person."
One key to AWS's continued expansion, however, will be to keep hanging on to all of a client's computing business once it grows beyond the start-up stage. Shopify, the wildly successful, Ottawa-based e-commerce platform that provides software to stores and online sellers, joined the ranks of $1-billion "unicorns" when it IPO'd last year. Jean-Michel Lemieux, vice-president of engineering, says his company was completely in the cloud when it launched itself in 2006. But, as it grew, it felt it couldn't rely on just one cloud provider, so it divided tasks between AWS, the cloud service Heroku—which was bought by Salesforce in 2010—and servers of its own.
One AWS function Shopify relies on heavily is the Amazon Relational Database Service. Relational databases store information in categories, and Shopify uses the service to house data for non-core apps. Lemieux's team also uses the Amazon Redshift service for merchant analytics, taking anonymized customer sales data from Shopify users, and identifying patterns and trends.
Still, Lemieux says that if Shopify really wants to unleash the floodgates, it tends to rely on its own powerful servers, rather than the cloud.
Yet Amazon and Shopify are now also partners. The two companies have integrated their services. Shopify clients can take advantage of Amazon's vast retail distribution network, and the two companies have made it seamless for Amazon Webstore clients to set up online stores with Shopify.
*****
As AWS keeps gobbling up more of the cloud computing business, it touches off the same good-or-evil debate that Amazon did when it supplanted traditional booksellers in the 1990s and early 2000s. Is it driving us toward a better, more efficient future with far more choices for customers? Or is it just a greedy leviathan?
AWS was launched on March 14, 2006. In part, it arose out of Amazon's internal struggles with IT issues. Software developers and project managers were pulling their hair out as IT gatekeepers kept them at bay, guarding the company's relatively constrained servers and computing capacity.
But the division also grew out of demand from third-party booksellers who sold titles online through Amazon. In the early 2000s, they wanted to get a closer look inside Amazon.com and its bestseller rankings, to see which books customers were buying and which publishers they were buying from. Third-party sellers of other merchandise also wanted more information on products and prices. So did outside programmers who wanted to develop application programming interfaces (APIs) to let those third parties search through Amazon's e-commerce platform.
"All of those things gave us what turned out to be a blinding glimpse of the obvious," says Adam Selipsky, a management consultant with an MBA from Harvard Business School who joined Amazon in 2005, and who is now vice-president of marketing, sales and product management and support for AWS. There was a case for setting up a cloud-computing division. "In the early days we talked about it potentially being a $1-billion business, and people looked at us as if we had several heads," he says.
Selipsky began with a team of nine people. For several years, they tried to operate AWS like a scrappy start-up. One early marketing event was held in a loft in Seattle's Capitol Hill trendy club-and-coffeehouse district, with bottled water donated by a venture capital company. The rest of Amazon treated AWS as a side project. When AWS launched its own website in 2006, it still had a strip of Amazon's retail categories (books, CDs, etc.) running across the top. "In the first couple of years we were asked, what exactly does this have to do with selling books?" says Selipsky, sitting on the 20th floor of an Amazon corporate office in Seattle.
Today, those categories are gone from the AWS website and it is Amazon's fastest-growing division. The day before he and I spoke, The Seattle Times ran a front-page feature on the development boom in the city's gentrifying South Lake Union neighbourhood, fuelled mostly by Amazon, which is putting up several buildings nearby.
Demands on employees can be heavy. A long feature in The New York Times last August detailed Amazon's occasionally brutal work pace. The article prompted a memo to employees from Jeff Bezos, who encouraged them to read it, but also said "the article doesn't describe the Amazon I know or the caring Amazonians I work with every day." But the angry wife of a burned-out former Amazon manager then posted an open letter online, blasting Bezos.
There is also a local pejorative for hyper-aggressive Amazon employees: Am-hole. In South Lake Union, someone has gone around posting yellow Am-hole quizzes, with questions such as: "Think this neighbourhood started with your arrival? Can you barely resist snapping your fingers at service people in the area? Do you not realize that you are working for an updated version of Sears and Roebuck?"
Selipsky bristles at that kind of sniping. "A couple of things that I've seen written certainly don't resemble the company that I work at, and I wouldn't have stayed here for over 10 years if it did," he says tartly. "I had one colleague who said Amazon and AWS operate like many of the successful start-ups down in Silicon Valley. We have a big vision, broad ambitions, we work really hard, with a sense of passion, in order to build things the world has never seen before. Does that mean we work hard? Absolutely. People who do well here are people who have passion and want to work hard to achieve great things."
AWS essentially brought a successful e-commerce company's attention to customer experience to an industry—enterprise computing—that was infamously almost allergic to customer needs. It also brought Amazon's ambition and hard-driving work culture to a sector that was used to explaining why things couldn't get done.
Selipsky also has little time for the old-guard companies AWS has surpassed. They still don't publish their prices, which means that their customers have to negotiate their cloud-computing costs. That, too, is a legacy of the days when enterprise technology companies had a stranglehold on their clients, who were hopelessly enmeshed in their providers' systems. And as those systems got bigger and more elaborate, costs usually went up.
AWS publishes prices and usage rates on its website, though it may provide custom solutions priced differently to larger clients. And its guiding principle is the same as it is for all of Amazon's operations: low cost and enormous scale. "Every business at Amazon pretty much works that way," Selipsky says. "The amazingly high margins of technology companies have not been in the best interest of customers. This, at the end of the day, is the most powerful benefit of cloud computing—the ability to move faster and deliver more value, more quickly to your customers."
Those lower costs also flow through to consumers in all sorts of markets. Ever wonder why Netflix is so cheap compared with cable or satellite TV? It's an AWS client, and it now accounts for 36.5% of North American streaming traffic during peak periods. When NASA's Curiosity Rover began streaming images from Mars in 2012, it did so over AWS servers, which also powered NASA's mission control at the Jet Propulsion Laboratory in the California Institute of Technology in Pasadena. "We have the honour of supporting robots on the surface of Mars," says Selipsky.
How much bigger could AWS get? "We're really still at the beginning," he says.
Editor's note: Shopify and Amazon have made it seamless for Amazon Webstore clients to set up online stores with Shopify. An earlier version of this story incorrectly said the partnership was between Shopify and AWS (Amazon Web Services) clients.