AWS Boto3 is the Python SDK for AWS. Boto3 can be used to directly interact with AWS resources from Python scripts. In this tutorial, we will look at how we can use the Boto3 library to perform various operations on AWS Secrets Manager.

Table of contents

Prerequisites

  • Python3
  • Boto3: Boto3 can be installed using pip: pip install boto3
  • AWS Credentials: If you haven’t set up AWS credentials before, this resource from AWS is helpful.

What is AWS Secrets Manager?

AWS Secret Manager allows you to store sensitive data like passwords, API keys, certificates, and other secrets securely in the cloud. When you create a secret, you define what kind of information should be stored, how long it should last, and who has access to it. Secrets manager also provides additional features such as rotation of credentials, encryption at rest, and automatic expiration of credentials.

With Secrets Manager, you can replace hardcoded credentials in your code, including passwords. You can retrieve secrets programmatically with an API call to Secrets Manager.

AWS Secrets Manager vs Systems Manager Parameter Store?

Secrets Manager is specifically designed for confidential information (like database secrets, API keys) whereas the Parameter Store supports a much wider use case. Parameter store can be used for storing application configuration in addition to any secrets.

How to create a secret?

We will use the create_secret method to create a Secrets Manager secret. The API definition can be found [here][create-secet].

Some of the important parameters are:

  • To create a secret, you can provide the secret value to be encrypted in either the SecretString parameter or the SecretBinary parameter, but not both. The SecretString parameter will be in the string format whereas the SecretBinary parameter is in the binary format.
  • You can use your own KMS encryption key. By default, Secrets Manager uses the AWS managed key aws/secretsmanager.

Required permissions

  • To create the secret, secretsmanager:CreateSecret is needed.
  • kms:GenerateDataKey and kms:Decrypt permissions are needed if you are using your own KMS key.

[create-secret] https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.create_secret

We can use the following cuntion to create a secret:


def create_secret(name, secret_value):
    """
    Creates a new secret. The secret value can be a string or bytes.
    """
    secrets_client = boto3.client("secretsmanager")
    kwargs = {"Name": name}
    if isinstance(secret_value, str):
        kwargs["SecretString"] = secret_value
    elif isinstance(secret_value, bytes):
        kwargs["SecretBinary"] = secret_value
    response = secrets_client.create_secret(**kwargs)
    return response

The output of creating a string secret would be:

create_secret("my-test-secret-str", "test-secret-str")

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'VersionId': '490a3ce1-c3d1-496a-b65e-cde9b8a7631c',
 'ResponseMetadata': {'RequestId': '8af459a0-98f0-44d9-9690-c349920f2b56',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 05:35:32 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '167',
   'connection': 'keep-alive',
   'x-amzn-requestid': '8af459a0-98f0-44d9-9690-c349920f2b56'},
  'RetryAttempts': 0}}

The output of creating a bytes secret would be:

byte_secret = bytes("my-secret", 'utf-8')

create("my-byte-secret", byte_secret)

{'ARN': 'arn:aws:secretsmanager:us-west-1:734066626836:secret:my-byte-secret-zbklZO',
 'Name': 'my-byte-secret',
 'VersionId': '639692aa-5079-4e43-a6af-6559dbd8d97f',
 'ResponseMetadata': {'RequestId': 'f61fe943-a936-49fb-86c3-93bbc92abd31',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'x-amzn-requestid': 'f61fe943-a936-49fb-86c3-93bbc92abd31',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '159',
   'date': 'Sun, 28 Aug 2022 17:41:47 GMT'},
  'RetryAttempts': 0}}

How to retrieve a secret value?

To retrieve the value of a secret, we will use the get_secret_value method. By default, Secrets Manager returns the current version (AWSCURRENT) of the secret.

Required permissions

  • secretsmanager:GetSecretValue is required for accessing a secret
  • If secret is encrypted using a custom KMS key, kms:Decrypt permissions are required.

def get_secret_value(name, version=None):
    """Gets the value of a secret.

    Version (if defined) is used to retrieve a particular version of
    the secret.

    """
    secrets_client = boto3.client("secretsmanager")
    kwargs = {'SecretId': name}
    if version is not None:
        kwargs['VersionStage'] = version
    response = secrets_client.get_secret_value(**kwargs)
    return response

The output of this function would be:

get_secret_value("my-test-secret-str")

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'VersionId': '490a3ce1-c3d1-496a-b65e-cde9b8a7631c',
 'SecretString': 'test-secret-str',
 'VersionStages': ['AWSCURRENT'],
 'CreatedDate': datetime.datetime(2021, 2, 26, 21, 35, 32, 273000, tzinfo=tzlocal()),
 'ResponseMetadata': {'RequestId': '64163213-1f39-430a-9ad2-816983695e51',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 05:56:38 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '262',
   'connection': 'keep-alive',
   'x-amzn-requestid': '64163213-1f39-430a-9ad2-816983695e51'},
  'RetryAttempts': 0}}

If the secret is of the string format, then the secret will be in the SecretString key. If the secret is in the binary format, then the secret will be in the SecretsBinary key.

How to modify an existing secret?

To modify an existing secret, we will use the update_secret method. Similar to the create_secret method, we need to provide either a string or binary secret.


def update_secret(name, secret_value):
    """Updates the value of an existing secret"""
    secrets_client = boto3.client("secretsmanager")

    kwargs = {'SecretId': name}

    if isinstance(secret_value, str):
        kwargs["SecretString"] = secret_value
    elif isinstance(secret_value, bytes):
        kwargs["SecretBinary"] = secret_value

    response = secrets_client.update_secret(**kwargs)
    return response

Output of running the function to update the secret:


update_secret("my-test-secret-str", "test-secret-str-new-value")

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'VersionId': '4e746767-91d3-4abc-a6ee-15fbe092e3d9',
 'ResponseMetadata': {'RequestId': '5892aa73-b1b8-4242-952a-7b0bf21b3a26',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 06:12:49 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '167',
   'connection': 'keep-alive',
   'x-amzn-requestid': '5892aa73-b1b8-4242-952a-7b0bf21b3a26'},
  'RetryAttempts': 0}}

How to create a new version of the secret?

We will use the put_secret_value method to create a new version of an existing secret. The new secret would automatically become the latest version of the secret.


def update_secret_version(name, secret_value, versions=None):
    """Puts a value into an existing secret."""
    secrets_client = boto3.client("secretsmanager")

    kwargs = {'SecretId': name}
    if isinstance(secret_value, str):
        kwargs['SecretString'] = secret_value
    elif isinstance(secret_value, bytes):
        kwargs['SecretBinary'] = secret_value
    if versions is not None:
        kwargs['VersionStages'] = versions
    response = secrets_client.put_secret_value(**kwargs)
    return response

Now, to update the secret, we can run the function like follows:


update_secret_version("my-test-secret-str", "test-secret-str-new-version", versions=["new-version"])

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'VersionId': 'b9e6a990-0bb0-4724-af63-16c7c314c3d3',
 'VersionStages': ['new-version'],
 'ResponseMetadata': {'RequestId': '8bbf368c-0caf-4555-a2bb-d3ddde593259',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 06:27:30 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '199',
   'connection': 'keep-alive',
   'x-amzn-requestid': '8bbf368c-0caf-4555-a2bb-d3ddde593259'},
  'RetryAttempts': 0}}

Now, we can use the previous get_secret method with the version we want to retrieve:

get_secret_value("my-test-secret-str", version="new-version")

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'VersionId': 'b9e6a990-0bb0-4724-af63-16c7c314c3d3',
 'SecretString': 'test-secret-str-new-version',
 'VersionStages': ['new-version'],
 'CreatedDate': datetime.datetime(2021, 2, 26, 22, 27, 30, 931000, tzinfo=tzlocal()),
 'ResponseMetadata': {'RequestId': 'a77e4ecc-78c2-4143-96eb-00c14bfc2c08',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 06:31:14 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '275',
   'connection': 'keep-alive',
   'x-amzn-requestid': 'a77e4ecc-78c2-4143-96eb-00c14bfc2c08'},
  'RetryAttempts': 0}}

How to delete a secret?

We will use the delete_secret method to delete the secret we created previously. By default, any deleted secrets can be retrieved within 30 days of deletion. If you want to disable recovery, we can disable recovery.


def delete_secret(name, without_recovery=False):
    """Deletes the secret."""
    secrets_client = boto3.client("secretsmanager")
    response = secrets_client.delete_secret(
        SecretId=name, ForceDeleteWithoutRecovery=without_recovery)
    return response

We can delete the secret by running the function like:


delete_secret("my-test-secret-str", True)

{'ARN': 'arn:aws:secretsmanager:us-west-2:xxx:secret:my-test-secret-str-k4sx86',
 'Name': 'my-test-secret-str',
 'DeletionDate': datetime.datetime(2021, 2, 26, 22, 44, 11, 848000, tzinfo=tzlocal()),
 'ResponseMetadata': {'RequestId': '233ccfa4-4094-4660-8c96-9776e4d9630f',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Sat, 27 Feb 2021 06:44:11 GMT',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '148',
   'connection': 'keep-alive',
   'x-amzn-requestid': '233ccfa4-4094-4660-8c96-9776e4d9630f'},
  'RetryAttempts': 0}}

How to generate a random password?

We can also use the Secrets Manager API to generate a random password using the get_random_password method.

Important Parameters:

  • PasswordLength: The default password length is 32 characeters. It can be changed using this parameter.

Required permissions:

  • secretsmanager:GetRandomPassword

def generate_random_password(password_length):
    """Generates a random password"""
    secrets_client = boto3.client("secretsmanager")
    response = secrets_client.get_random_password(PasswordLength=password_length)
    return response

Output:

generate_random_password(32)

{'RandomPassword': '+$w&@L4QxApcG>TJpT;r-%yru06h._y#',
 'ResponseMetadata': {'RequestId': '7463078b-e4fa-4a30-af91-56877951e7c9',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'x-amzn-requestid': '7463078b-e4fa-4a30-af91-56877951e7c9',
   'content-type': 'application/x-amz-json-1.1',
   'content-length': '53',
   'date': 'Sun, 28 Aug 2022 20:44:20 GMT'},
  'RetryAttempts': 0}}