Introduction

AWS Simple Storage Service (S3) is one of the most widely used object storage solutions. With S3, you can store files, photos, videos, applications, websites, databases, backups, logs, and many other types of data online without worrying about performance issues, security breaches, or hardware failures.

Although Amazon S3 is highly reliable, by default, it does not protect against accidental data loss or malicious destruction caused by authorized users. To protect yourself from these scenarios, you must leverage advanced S3 features like Object Versioning and Delete Markers.

Table of contents

What is AWS S3 Object Versioning?

S3 Object Versioning allows you to maintain multiple versions of an object. When you change the object, S3 creates a new version for you and stores it so you can revert to previous versions if required.

When you update an existing object on S3, the only way to revert to an older version is to delete it and recreate it from scratch. However, with S3 Object Versioning, you can create multiple versions of the same object in a bucket, which makes it easy for you to roll back in case of changes or errors in your code.

AWS S3 Object Versioning

What is an AWS S3 Delete Marker?

Delete Markers are a feature of versioning-enabled S3 buckets. When you delete an object in a versioning-enabled bucket, the object isn’t deleted permanently. Instead, AWS creates a placeholder (or marker) for the object. This marker is referred to as the Delete Marker. This marker becomes the current version of the object. The Delete Marker makes AWS S3 behave as if the object has been deleted.

The following figure shows that a simple DELETE does not remove the specified object. Instead, Amazon S3 inserts a delete marker.

AWS S3 Delete Marker

How do Delete Markers work?

If you make a DELETE request for an object in an S3 bucket with versioning enabled, Amazon S3 will not permanently delete the object. Instead, S3 will insert a delete marker in that bucket, and that marker will become the current version of the object with a new ID.

A delete marker has the following properties:

  • a key name (or key) and version ID like any other object.
  • It does not have data associated with it.
  • It is not associated with an access control list (ACL) value.
  • It does not retrieve anything from a GET request because it has no data; you get a 404 error.
  • The only operation you can use on a delete marker is DELETE, and only the bucket owner can issue such a request.  

If you try to get an object and its current version is a delete marker, Amazon S3 responds with the following:

  • A 404 (Object not found) error
  • A response header: x-amz-delete-marker: true

Examples

Deleting an object from a versioning enabled bucket

aws s3api delete-object --bucket abhisheks-test-bucket --key sample.csv

Output:

{
    "DeleteMarker": true,
    "VersionId": "ydLmU4m94aHrEK8KWRZkmwcDfN7RY2hh"
}

Retrieving object from S3

aws s3api get-object --bucket abhisheks-test-bucket --key sample.csv sample.csv

Output:

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Retrieving all object versions from S3

We can retrieve all versions of a particular object.

aws s3api list-object-versions --bucket abhisheks-test-bucket

Output:

{
    "Versions": [
        {
            "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
            "Size": 0,
            "StorageClass": "STANDARD",
            "Key": "sample.csv",
            "VersionId": "Lk7Eolts1.71QbJPdkD3EuWtuya2RUlJ",
            "IsLatest": false,
            "LastModified": "2022-10-04T22:19:37+00:00",
            "Owner": {
                "DisplayName": "abhishekray07",
                "ID": "873f2dd9943352346a1d0414e31c98eecf45e8997265ca9f639ab07d05dc87c6"
            }
        }
    ],
    "DeleteMarkers": [
        {
            "Owner": {
                "DisplayName": "abhishekray07",
                "ID": "873f2dd9943352346a1d0414e31c98eecf45e8997265ca9f639ab07d05dc87c6"
            },
            "Key": "sample.csv",
            "VersionId": "ydLmU4m94aHrEK8KWRZkmwcDfN7RY2hh",
            "IsLatest": true,
            "LastModified": "2022-10-04T22:20:39+00:00"
        },
    ]
}

Querying an older version of an object

We can retrieve an earlier (non-current) version of an object by specifying the version ID in the request.

aws s3api get-object --bucket abhisheks-test-bucket --key sample.csv --version-id Lk7Eolts1.71QbJPdkD3EuWtuya2RUlJ sample.csv

Output:


{
    "AcceptRanges": "bytes",
    "LastModified": "2022-10-04T22:19:37+00:00",
    "ContentLength": 0,
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
    "VersionId": "Lk7Eolts1.71QbJPdkD3EuWtuya2RUlJ",
    "ContentType": "binary/octet-stream",
    "Metadata": {}
}

Manging delete markers

Deleting a delete marker

You can delete a delete marker by specifying the version ID in the delete object request. Removing a delete marker makes an older version current.

aws s3api delete-object --bucket abhisheks-test-bucket --key sample.csv --version-id ydLmU4m94aHrEK8KWRZkmwcDfN7RY2hh

Output:

{
    "DeleteMarker": true,
    "VersionId": "ydLmU4m94aHrEK8KWRZkmwcDfN7RY2hh"
}

Configuring lifecycle to clean up expired delete markers automatically

An expired object delete marker is one where all object versions are deleted and only a single delete marker remains. If the lifecycle policy is set to delete current versions, or the ExpiredObjectDeleteMarker action is explicitly set, Amazon S3 removes the expired object’s delete marker.