AWS Elasticache Deep Dive: What is it and when to use it
Introduction
AWS offers a broad selection of datastores and therefore, it is important to choose the right datastore for your application. Amazon Elasticache is a fully-managed in-memory data store that supports both Redis and Memcached. In-memory databases provide low latency lookups and high throughput.
Common use-cases for Elasticache are:
- Caching
- Real-time data analysis
- Leaderboards
- Chat rooms
Benefits
Fully Managed
AWS is responsible for managing the entire infrastructure including updates, configuration, and monitoring.
Scalable
AWS provides non-disruptive scaling for Elasticache. Reads can be scaled by adding more replicas. Writes and memory can be scaled with sharding.
Highly Available
Elasticache is reliable and highly available. Elasticache clusters are deployed across multiple AZs. AWS also provides automatic failovers with minimal downtime.
Amazon Elasticache Engines
Elasticache supports two different in-memory databases:
- Redis
- Memcached
The table below lists some of the differences between Redis & Memached:
| Redis | Memcached | |
|---|---|---|
| Primary use case | In-memory database & cache | Cache |
| Data model | In-memory key-value | In-memory key-value |
| Data structures | Strings, lists, sets, sorted sets, hashes, hyperloglog | Strings, objects |
| High availability & failover | Yes | No |
AWS Elasticache for Redis
Redis is the most popular in-memory database and is more widely adopted than Memcached. For the rest of this article, we will focus on Redis and how to use it with Elasticache.
Redis data structures
Redis provides a rich set of data structures. In a traditional key-value store, the value is limited to a simple string. However, in Redis, the value can hold complex data structures. These data structures enable Redis to be used for a variety of different applications and use-cases as we will see later in this article.
Some of these data structures are:
- Binary-safe string
- Lists
- Set
- Sorted set: similar to set, but every string element is associated with a score. The elements are sorted by their score.
- Hash
- Streams
This article provides a more in-depth explanation of the various data structures supported by Redis.
Redis Cluster mode
Elasticache makes it easier to scale Redis clusters. When creating a new Elasticache cluster, you have the option to choose from one of three cluster configurations:
- A single node
- Cluster mode disabled
- Cluster Mode enabled
The three modes differ primarily in reliability, availability, and scaling behavior.
The following table lists the key differences between the two cluster modes (a single node cluster with no replication should not be used for production workloads):
| Cluster Mode Disabled | Cluster Mode Enabled | |
|---|---|---|
| Replication | Yes (up to 5 replicas per node) | Yes (up to 5 replicas per node) |
| Data Partitioning | No (single shard) | Yes (up to 250 shards) |
| Scaling | Change node type (vertical scaling) | Add / remove shards and rebalance (horizontal scaling) |
| Performance | Throughput limited by 1 primary, 5 replicas | Throughput scales with number of shards |
| Multi-AZ | Optional with at least 1 replica | Required |
| Failover | Writes affected on entire dataset; Reads available | Writes affected on partial dataset (shard); Reads available |
Redis Sharding
In this section, we will take a closer look at how sharding (data partitioning) works in Elasticache.
If cluster mode is disabled, there is only one shard. The shard comprises the primary node along with the read replicas. Read replicas maintain a copy of the data from the cluster’s primary node.
Elasticache allows up to 250 shards for a Redis cluster that has Cluster-Mode enabled. Each shard has a primary node and up to 5 read replicas. Elasticache distributes the keyspace evenly across all the shards. When reading or writing data to the cluster, the client itself determines which shard to use based on the keyspace. This design pushes the work to the client and avoids any potential single points of failure.
Cluster Scaling
Elasticache clusters with Cluster-Mode enabled can also leverage online re-sharding and shard rebalancing dynamically with no downtime or application interruption.
Scaling activities that are supported are:
- Scale-out - Increasing read and write capacity by adding more shards.
- Scale-in - Reduce read and write capacity by removing shards
- Rebalancing - Move keyspaces among shards
A summary of different methods of scaling Redis clusters is provided in the table below:
| Vertical scaling | Horizontal scaling | |
|---|---|---|
| Action | All instances (including replicas) are changed | Individual shards are added or removed |
| Cost | Cluster-wide change | Incremental change |
| Impact | Clients reconnect | Clients stay connected |
| Support | Cluster-Mode enabled and disabled | Cluster-Mode enabled |
Use-cases & Patterns
In this section, we will look at some of the most popular use-cases for Redis.
Caching
Caching is the most common reason for using an in-memory key-value store. The two widely used caching patterns are:
-
Cache-aside (or lazy loading): In this pattern, the application first queries the cache to see if the data is available. If the data is available (cache hit), the cached data is returned. If the data isn’t available (cache miss), the database is queried for the data. The cache is then populated with the data retrieved from the database.
-
Write-through cache: In a write-through cache, the cache is proactively updated immediately following an update on the primary database.
Message Queues / Pub-Sub
It is possible to build message queues using Redis Streams. This article gives an overview of how Redis Streams help in facilitating message queues. If you are interested in a comparison between Amazon SQS and Redis Streams, check out this article.
Leaderboards
Leaderboards can be efficiently built using the Sorted Set data structure on Redis. This article provides an example of how to build a leaderboard using Elasticache.
Geospatial
Redis’ Geospatial data structure makes it easier to build location-based apps. This article provides an overview of how to build a bike-sharing application using the Geospatial data structure.
Conclusion
Amazon Elasticache is one of the most popular AWS services. It is helpful to understand the key concepts and features of Elasticache so that you can use it for the right use-case and maximize its performance.