AWS Boto3 is the Python SDK for AWS. Boto3 can be used to directly interact with AWS resources from Python scripts. In this tutorial, we will look at how we can use the Boto3 library to perform various operations on AWS KMS.

Table of contents

Prerequisites

  • Python3
  • Boto3: Boto3 can be installed using pip: pip install boto3
  • AWS Credentials: If you haven’t set up AWS credentials before, this resource from AWS is helpful.
  • cryptopgraphy: We will be using the cryptography package to encrypt and decrypt data.

How to create a Customer Master Key?

A Customer Master Key (CMK) is used to encrypt data. However, the maximum size of data that can be encrypted using the master key is 4KB. CMKs are used to generate, encrypt, and decrypt data keys that can be used outside of AWS KMS to encrypt data.

AWS KMS supports two types of CMKs:

  • Symmetric CMK: 256-bit symmetric key that never leaves AWS KMS unencrypted By default, KMS creates a symmetric CMK.
  • Asymmetric CMK: AWS KMS generates a key pair where private key never leaves AWS KMS unencrypted.

The following function creates a new Customer Master Key:


def create_cmk(description="My Customer Master Key"):
    """Creates a KMS Customer Master Key

    Description is used to differentiate between CMKs.
    """

    kms_client = boto3.client("kms")
    response = kms_client.create_key(Description=description)

    # Return the key ID and ARN
    return response["KeyMetadata"]["KeyId"], response["KeyMetadata"]["Arn"]

The output of the above function should be something like:

('c98e65ee-95a5-409e-8f25-6f6732578798',
 'arn:aws:kms:us-west-2:xxxx:key/c98e65ee-95a5-409e-8f25-6f6732578798')

How to retrieve existing Customer Master Key?

CMKs are created, managed and stored within AWS KMS. The following snippet shows how to retrieve an existing CMK based on the description it was created with.


def retrieve_cmk(description):
    """Retrieve an existing KMS CMK based on its description"""

    # Retrieve a list of existing CMKs
    # If more than 100 keys exist, retrieve and process them in batches
    kms_client = boto3.client("kms")
    response = kms_client.list_keys()

    for cmk in response["Keys"]:
        key_info = kms_client.describe_key(KeyId=cmk["KeyArn"])
        if key_info["KeyMetadata"]["Description"] == description:
            return cmk["KeyId"], cmk["KeyArn"]

    # No matching CMK found
    return None, None

Output

retrieve_cmk("My Customer Master Key")

('c98e65ee-95a5-409e-8f25-6f6732578798',
 'arn:aws:kms:us-west-2:xxx:key/c98e65ee-95a5-409e-8f25-6f6732578798')

How to create a data key?

A data key is a unique symmetric data key that is used to encrypt data outside of AWS KMS. AWS returns both an encrypted and a plaintext version of the data key.

AWS recommends the following pattern to use the data key to encrypt data outside of AWS KMS:

- Use the GenerateDataKey operation to get a data key.
- Use the plaintext data key (in the Plaintext field of the response) to encrypt your data outside of AWS KMS. Then erase the plaintext data key from memory.
- Store the encrypted data key (in the CiphertextBlob field of the response) with the encrypted data.

The function below generates a data key and returns the encrypted as well as plaintext copy of the key.

import base64

def create_data_key(cmk_id, key_spec="AES_256"):
    """Generate a data key to use when encrypting and decrypting data"""

    # Create data key
    kms_client = boto3.client("kms")
    response = kms_client.generate_data_key(KeyId=cmk_id, KeySpec=key_spec)

    # Return the encrypted and plaintext data key
    return response["CiphertextBlob"], base64.b64encode(response["Plaintext"])

How to encrypt data?

Data can be encrypted client-side using the generated data key along with the cryptography package in Python. It is recommended to store the encrypted data key along with your encrypted data since that will be used to decrypt the data in the future.

from cryptography.fernet import Fernet

NUM_BYTES_FOR_LEN = 4

def encrypt_file(filename, cmk_id):
    """Encrypt JSON data using an AWS KMS CMK"""

    with open(filename, "rb") as file:
      file_contents = file.read()

    data_key_encrypted, data_key_plaintext = create_data_key(cmk_id)
    if data_key_encrypted is None:
        return

    # Encrypt the data
    f = Fernet(data_key_plaintext)
    file_contents_encrypted = f.encrypt(file_contents)

    # Write the encrypted data key and encrypted file contents together
    with open(filename + '.encrypted', 'wb') as file_encrypted:
        file_encrypted.write(len(data_key_encrypted).to_bytes(NUM_BYTES_FOR_LEN,
                                                              byteorder='big'))
        file_encrypted.write(data_key_encrypted)
        file_encrypted.write(file_contents_encrypted)

Next, let’s create a file called test_file with the following content:

hello, world
this file will be encrypted

After running the encrypt_file function on our input file, the contents of the encrypted file should look something like:

cat test_file.encrypted
00m0hj@٪`He.0at2N{[6~0| *H
             jF~P~r;,7H#+5##߯^ڍ"={Gӓ

How to decrypt a data key?

The decrypt function can be used to decrypt an encrypted data key. The decrypted data key can then be used to decrypt any data on the client side.


import base64

def decrypt_data_key(data_key_encrypted):
    """Decrypt an encrypted data key"""

    # Decrypt the data key
    kms_client = boto3.client("kms")
    response = kms_client.decrypt(CiphertextBlob=data_key_encrypted)

    # Return plaintext base64-encoded binary data key
    return base64.b64encode((response["Plaintext"]))

How to decrypt data?


from cryptography.fernet import Fernet

NUM_BYTES_FOR_LEN = 4

def decrypt_file(filename):
    """Decrypt a file encrypted by encrypt_file()"""

    # Read the encrypted file into memory
    with open(filename + ".encrypted", "rb") as file:
      file_contents = file.read()

    # The first NUM_BYTES_FOR_LEN tells us the length of the encrypted data key
    # Bytes after that represent the encrypted file data
    data_key_encrypted_len = int.from_bytes(file_contents[:NUM_BYTES_FOR_LEN],
                                            byteorder="big") \
                             + NUM_BYTES_FOR_LEN
    data_key_encrypted = file_contents[NUM_BYTES_FOR_LEN:data_key_encrypted_len]

    # Decrypt the data key before using it
    data_key_plaintext = decrypt_data_key(data_key_encrypted)
    if data_key_plaintext is None:
        return False

    # Decrypt the rest of the file
    f = Fernet(data_key_plaintext)
    file_contents_decrypted = f.decrypt(file_contents[data_key_encrypted_len:])

    # Write the decrypted file contents
    with open(filename + '.decrypted', 'wb') as file_decrypted:
      file_decrypted.write(file_contents_decrypted)

Output of running this function on the encrypted file:

decrypt_file("test_file)

cat test_file.decrypted
hello, world
this file will be encrypted