Introduction

AWS S3 supports the ability to tag objects. Each tag is a key-value pair. You can associate up to 10 tags with an object.

Some of the benefits of tagging objects in S3 are:

  • Categorizing your storage
  • Fine-grained access control: You can grant IAM user permissions to read-only objects with specific tags
  • Fine-grained lifecycle management: You can specify a tag-based filter in a lifecycle rule.

AWS s3api supports Object Tagging via the put-object-tagging command. However, this command only supports tagging one object at a time. In this article, we will look at how you can tag all objects within a S3 bucket.

Table of contents

List all objects in the bucket

First, we will use the list-object command to filter and list all the objects we want to tag.


aws s3api list-objects --bucket my-test-bucket --prefix my-prefix/

This command will return all the objects in the bucket my-test-bucket with my-prefix/ prefix.

Output:

{
    "Contents": [
        {
            "Key": "prefix/my-file.json",
            "LastModified": "2019-12-03T19:03:26+00:00",
            "ETag": "\"c416452ad7f219e0bac6adc89d3a9285\"",
            "Size": 8440,
            "StorageClass": "STANDARD",
            ...
        },

    ....
    ]
}

Retrieve object key

Next, we will use the --query flag to only retrieve the object key as text. Let’s modify the command as follows:


aws s3api list-objects --bucket my-test-bucket --prefix my-prefix/ --query 'Contents[].[Key]' --output text

Things to note:

  • 'Contents[].[Key]': We only want the Key field so we are filtering on that field.
  • --output text: We want the output in a text format so its easier to pipe that output

Output:

prefix/my-file.json
prefix/report.csv

Add tags to objects

Now, we will use the output of the previous command and pipe it to the xargs command. xargs supports reading streams of data from standard input and pass it as input for another command. You can see examples of how to use xargs here.

We will use the put-object-tagging command to tag individual objects.

This is what the complete command looks like:


aws s3api list-objects --bucket my-test-bucket --prefix my-prefix/ --query 'Contents[].[Key]' --output text | xargs -n 1 aws s3api put-object-tagging --bucket my-test-bucket --tagging 'TagSet=[{Key=foo,Value=bar}]' --key

Things to note:

  • We are adding the tag foo=bar to our objects
  • --key gets passed as input to the xargs command
  • We can parallelise the command by using -P flag

We can verify that our objects were tagged by running the following command:


aws s3api get-object-tagging --bucket my-test-bucket --key prefix/my-file.json

{
"TagSet": [
    {
    "Key": "foo",
    "Value": "bar"
    }
  ]
}

Conclusion

You can use the following command to tag multiple objects at once:


aws s3api list-objects --bucket my-test-bucket --prefix my-prefix/ --query 'Contents[].[Key]' --output text | xargs -n 1 aws s3api put-object-tagging --bucket my-test-bucket --tagging 'TagSet=[{Key=foo,Value=bar}]' --key