How to fix Docker Rate Limiting error on AWS CodeBuild?
AWS CodeBuild is a managed service from Amazon Web Services (AWS) that allows developers to build applications without having to manage servers. This makes it easier for them to build, test and deploy their services.
CodeBuild is frequently used with Docker to build images that are eventually deployed using AWS Elastic Container Service (ECS) or AWS Elastic Kubernetes Service (EKS). One of the most common issues developers run into while using Docker with CodeBuild is being rate limited by Docker while trying to pull an image from Docker Hub. The error message looks something like this:
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
In this tutorial, we will look into why this error occurs and how you can solve it.
What is the reason for Docker’s Rate Limiting Error?
Docker has a rate limit of 100 pulls per 6 hours for anonymous users. This means that if you are using Docker Hub without authenticating, you can only pull 100 images in a 6 hour period. If you exceed this limit, you will get the error message mentioned above. This is done to prevent abuse of Docker Hub. You can read more about the policy on Docker’s website.
How to fix Docker’s Rate Limiting Error?
There are a couple of ways to fix this error:
Authenticate with Docker Hub
You create an account on Docker Hub and use the credentials from that account to access Docker in your CodeBuild builds. Docker provides a free plan that provides a much higher limit for accessing images. You can read more about the free plan here. We will not be covering this option in this tutorial.
Use AWS Public Registry
AWS provides a public registry that you can use to pull images from. If the image that you are looking for is available on the AWS public registry, you can use that to pull the image. You can read more about the AWS public registry here.
How to use DockerHub with AWS CodeBuild?
To use DockerHub with AWS CodeBuild, we will need to store the DockerHub credentials in AWS Secrets Manager. We will then use the credentials to authenticate with DockerHub and pull the image. Check out this tutorial to learn how to integrate AWS Secrets Manager with AWS CodeBuild.
buildspec.yaml should look something like this:
version: 0.2 env: secrets-manager: DOCKERHUB_PASS: "/docker/password" DOCKERHUB_USERNAME: "/docker/username" phases: pre_build: commands: - docker login --username $DOCKERHUB_USERNAME --password $DOCKERHUB_PASS build: commands: - echo "Building Docker Image..."" post_build: commands: - echo "Build finished"
In this tutorial, we looked at how to fix the Docker Rate Limiting error on AWS CodeBuild. We looked at two ways to fix this error:
- Authenticate with Docker Hub
- Use AWS Public Registry