How to copy files recursively using AWS S3 cp?
AWS S3 cp provides the ability to:
- Copy a local file to S3
- Copy S3 object to another location locally or in S3
If you want to copy multiple files or an entire folder to or from S3, the
--recursive flag is necessary. The official description of the recursive flag is:
Command is performed on all files or objects under the specified directory or prefix.
How to use the
If you want to copy all the files in a folder recursively named
my-local-folder to an S3 bucket named
my-s3-bucket, the command you would use is:
aws s3 cp my-local-folder s3://my-s3-bucket/ --recursive
If you want to download all the files from this S3 bucket to your local folder, the command you would use is:
aws s3 cp s3://my-s3-bucket/ . --recursive
Is there a way to preview the changes made by AWS S3 cp?
You can use the
--dry-run flag to generate the list of files copied.
aws s3 cp --dryrun . s3://my-s3-bucket --recursive
How to download multiple files with include and exclude flags?
AWS S3 CLI supports include and exclude filters to specify paths to include and exclude when copying files. These parameters perform pattern matching to exclude or include a particular file or object. The following pattern symbols are supported:
*: Matches everything
?: Matches any single character
[sequence]: Matches any character in the sequence
[!sequence]: Matches any character not in sequence
When there are multiple filters, the filters that appear later in the command take precedence.
If you want to upload all files ending with .jpg you would use the following command:
aws s3 cp . s3://my-bucket/ --recursive --exclude "*" --include "*.jpg"