On Wednesday last week, AP once again attended the annual AWS Summit in London. Every year, this event gets bigger, and this year there were 6000 attendees and 74 partners in the exhibitor hall. Here are some of the highlights from this year's event.
The morning keynote was kicked off by Gavin Jackson, Managing Director, Amazon Web Services UK and Dr. Werner Vogels, CTO, Amazon.com. They were joined on stage by Dave Perry, CTO, DVLA; Claire Dickson, CIO, BP; Dan Webb, VP Engineering, Deliveroo; and Paul Clarke, CTO, Ocado.
The keynote started off by talking about Amazon’s impressive growth. The AWS business now has a $14 Billion revenue run rate as of Q1 2017. Comparing this to other estimates I’ve seen about Microsoft Azure, I make AWS more than 5 times larger than their nearest competitor.
What’s more, AWS had year-on-year growth of 43% which makes them the fastest growing multi-billion dollar enterprise IT company in the world. And looking at the other companies on that list, most of the top 10 are running some or all of their business in AWS including ServiceNow, Workday, Saleforce and Adobe.
The number of AWS regions around the globe has increased again since last year, now with 16 Regions and 43 Availability Zones, including a UK region which went live at end of 2016, which is very important for some UK companies who require that their data stays within the UK, such as the DVLA, who came on stage to talk about it.
Werner's theme throughout the keynote was that AWS give you Super Powers. By making developers' lives easy they can get things done that they couldn’t on-premises, or which would take a lot more time.
- Forcefield - This referred to the deep set of Cloud Security tools (Networking, Identity, Compliance and Encryption). Many customers including financial services companies now say they are more secure in AWS than they could ever be on-premises. Werner went into some detail around AWS Shield, released at the end of last year, as a managed DDoS protection service that protects your web applications on AWS from DDoS attacks.
- SuperSonic speed - This talked about the agility you get with AWS and the ability to scale instantly. It also covered some of the newer instance types such as P2 and F1. There are now 11 families including several GPU based instance type for AI applications. BP Downstream CIO, Clare Dickson, took to the stage to explain how they can operate Faster, Cheaper and Better in the cloud.
- Invisibility - Covered the advantages of serverless architectures and services such as Lambda, Lambda Step functions, AWS Gateway, S3, DynomoDB and newer services such as [email protected] and Athena.
- Flight - Was all about DB Migration from on-premises proprietary databases, such as Oracle, to a Relational Database Service (RDS) such as RDS Aurora, MariaDB etc. Amazon said they have migrated over 28,000 databases so far.
- X-ray vision - This super power related to the various Data Analytics capabilties such as using Elastic Map Reduce (EMR), which is Amazon's Hadoop as a service, and also some of the newer services such as Athena and RedShift Spectrum. Athena and RedShift Spectrum have a lot of similarities, being ways of using ANSI standard SQL against data files stored in your S3 Data Lake, avoiding the need for any ETL (Extract Tranform and Load) processes. With RedShift Spectrum you need a RedShift cluster, but no need to load the data into RedShift. With Athena it is completley serverless, so you simply pay for the queries you run.
- Pre-cognition - This section was about predicting the future using Artiificial Intelligence (AI) and Machine Learning (ML). AWS now offers 3 AI services: Lex, Polly and Rekognition, which can do speech recognition, text-to-speech and image recognition purely through interacting with their APIs and requiring no servers or complex programming. Ocado CTO Paul Clarke took to the stage to give an impressive overview of their next generation warehouses which use Machine Learning to optimise the picking of groceries by swarms of thousands of robots.
- Immortality - Was all about building a business that lives forever. For example, GE are the only original company still in S&P500. Amazon said that they themselves would be out of business in 10-15 years if they stopped innovating. Startups everywhere, 'born in the cloud', are taking market share from long established businesses, and in some cases putting them out of business.
The rate of innovation at Amazon is phenomenal. With over 1000 new features released in 2016 and that number set to grow again in 2017 it would seem hard for competitors like IBM, Microsoft and Google to keep up.
Following the keynote, the Summit split into multiple parallel tracks with 45 different tech sessions, covering many of the AWS services, either for those wanted to get started, or for those already using them wanting to take things to the next level.
Many of the sessions I attended (and the keynote) had a heavy focus on 2 things: serverless application design and Artificial Intelligence.
Serverless Application Design
Serverless architectures have been pushed by Amazon for some time now. At the London Summit the Deep dive session was so popular that I along with several hundred others couldn’t get in :(
Serverless architectures have the advantage that there are no servers to manage which means no Operating Systems to patch, maintain or administer. You are just responsible for your code and Amazon will scale your application automatically for you without you having to create a load balancer and define scaling rules. And your application will be inherently highly available since there is a built in fault tolerant architecture under the covers. From a cost point of view their is no idle capacity to worry about (most apps are over provisioned) since you only pay for the compute time or storage you actually use. See https://aws.amazon.com/serverless/ for more.
AWS Services which allow you to build serverless applications include, Lamdba (run code on demand in AWS regions) and the newer [email protected] (run code at CloudFront edge locations), API Gateway (managed service for running APIs at scale), S3 (secure, durable, scalable object storage), DynamoDB (NoSQL DB for both document and key value storage), SNS (message queuing as a service), Kinesis (analyse streaming data) and the new service Athena (run SQL queries against files stored in S3 without requiring any ETL or loading into a database).
But what about the downside to serverless? Well the number one most obvious downside is that you are locking yourself into one vendor. Vendor lock-in is not a new thing in IT. Historically many companies were locked into hardware or software vendors. What is new with Amazon AWS, is that as an organisation, Amazon are also likely to compete with yours as they expand into more and more industries. If you are a food retailer running on AWS for example, seeing Amazon acquire Whole Foods for $13.4 billion, might feel like you are paying Amazon money to help them compete with you. See a recent article titled How to keep Amazon from eating your business, too.
One solution is to avoid Serverless entirely and instead adopt a Container based architecture. Through technologies like Docker, Kubernetes and Mesosphere you can choose to move your containers between private and pubic cloud vendors avoiding lock-in.
Artificial Intelligence (AI) has been a hot topic for several years now and as you would expect, had a big focus at this years AWS Summit. For those looking to get in to AI quickly and easily, there are 3 AI services with which you can interact simply though their APIs, without spinning up any servers:
- Amazon Lex, to build sophisticated text and voice chatbots (as used by Amazon Alexa devices)
- Amazon Polly, for turning text into lifelike speech in multiple languages (as used by Amazon Alexa devices)
- Amazon Rekognition, for deep learning-based image recognition.
Amazon said that the list of AI services will continue to grow.
For more advanced AI use cases, AWS has their P2 instances which are general purpose GPU instances optimised for AI. They have an AMI available which includes all the major engines such as TensorFlow, Caffe, Theano but they have aligned themselves with Apache MxNet which they say performs extremely well on the P2 instance, and it also supports multiple programming languages, both Declarative and Imperative.
In the exhibitor hall were 74 vendors with solutions in the AWS eco-system include 2 AP partners, AppDynamics and Turbonomic.
AppDynamics are the leader in Application Performance Management according to analysts such as Gartner and they have a solution which is designed to give you end to end visibility of your applications whether they are on-premises or with a cloud provider such as AWS. As well as monitoring every line of code to help you resolve issues quickly, it also tracks end-user experience and business metrics so your business can have clear, understandable correlations between the quality of your customers’ experiences with your applications and business outcomes.
Turbonomic gives you visibility of all application workloads, whether on-premises or in AWS or Azure. It can recommend or automate the movement of those workloads to free up resources which have been over-provisioned, reducing your public cloud bill or freeing up infrastructure for on-premises. It will also identify where extra resources are needed for critical workloads in order to ensure performance.