I was recently looking into API Gateways for a project. As I was reading up on AWS’ API Gateway, I realized that there were two options that I could potentially choose: REST APIs or HTTP APIs. The two services had similar names and provided similar features and left me feeling pretty confused about which option was more suitable for my use-case.
One can always count on AWS to create new products that end up confusing people even more. As I started digging deeper to understand the differences, I wondered why Amazon created an entirely new HTTP API service instead of improving the existing REST API service. This way the customers get all the benefits and don’t have to spend time trying to understand the differences and choosing the right service for themselves.
In this article, I’ll try and do a comparison between the two services, and hopefully, that will be useful for folks who are trying to decide between the two services.
With the growing popularity of Serverless, I wanted to explore how to to build a (Big) Data platform using Amazon’s serverless services. In this article, we will look into what is a data platform and the potential benefits of building a serverless data platform. We will also look at the architectures of some of the serverless data platforms being used in the industry.
What is a Data Platform?
Over the last decade, software applications have been generating more data than ever before. As computation and storage have become cheaper, it is now possible to process and analyze large amounts of data much faster and cheaper than before.
A lot of companies don’t fully utilize Spot Instances because they are worried about making their service unreliable. AWS Spot Fleet is a service that lets you create a fleet of EC2 instances that can help make your application more reliable and significantly cheaper to run. A fleet is composed of both Spot and On-Demand Instances. In this article, we will learn more about Spot Fleet, how they are different from Spot Instances and how they can be used to make your infrastructure cheaper and more reliable.
Before we go deeper into Spot Fleet, it is useful to understand how Spot instances work. AWS has three tiers of EC2 instances. They are:
On-Demand Instances: This is generally the default option when you create an EC2 instance. Most of the time, when you see any EC2 pricing online, it refers to the On-Demand pricing. On-Demand instances are the most expensive because you have no commitment to AWS. You can terminate the instance at any time and only pay for how long you use the instance.
Reserved Instances (RIs): RIs are a billing construct instead of being an actual EC2 instance tier. RIs can be significantly (40-60%) cheaper than On-Demand instances. However, in return for the discount, you have to make a long term commitment (1-year or 3-year).
Spot Instances: AWS has a lot of spare compute capacity (unused EC2 instances) which they offer as Spot Instances. Spot Instances are generally the cheapest option (50%-80% lower than On-Demand) amongst the three. However, since Spot Instance availability is predicated upon the demand, Amazon provides no guarantee that once you get a Spot Instance you can run it indefinitely. This is the biggest difference between On-Demand and Spot Instances.
Amazon recently announced that it’s Elastic Kubernetes Service (EKS) now supports running Kubernetes pods on AWS Fargate. Amazon is putting a lot of resources into their serverless products and this new feature allows us to run Kubernetes in a “serverless” mode. I am pretty excited about being able to run Kubernetes pods in Fargate as this reduces the overhead to run Kubernetes applications as there is no need to run any EC2 worker nodes anymore.
This article will cover the following topics:
- Differences between running a traditional EKS cluster and running EKS on AWS Fargate
- How to bring up a new EKS cluster which can run pods on Fargate
Amazon made a bunch of new product announcements in the recently concluded re:Invent conference. I was looking forward to seeing what new products and features they were going to introduce in their serverless services and there are quite a few announcements. In this article, I have tried to summarize some of the announcements which I thought were the most interesting.
Provisioned Concurrency for Lambda
Amazon introduced Provision Concurrency for Lambda functions. This feature allows you to keep your Lambda functions initialized and ready to respond to any requests in milliseconds. This new feature should help with the Cold Start problem in Lambda.
The pricing for Provisioned Concurrency is slightly different. More information on pricing is available here.