Rename and Move S3 Object using Python Boto3

Overview

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.

Amazon S3 provides management features so that you can optimize, organize, and configure access to your data to meet your specific business, organizational, and compliance requirements.

To access S3 or any other AWS services we need SDK The SDK is composed of two key Python packages: Botocore (the library providing the low-level functionality shared between the Python SDK and the AWS CLI) and Boto3 (the package implementing the Python SDK itself).

Prerequisite

1. If you’re using AWS CLI need to install the same. Update python and install the Boto3 library in your system.

2. If you’re using some AWS Services like AWS Lambda, Glue, etc need to import the Boto3 package

Sample Code

There is no direct command available to rename or move objects in S3 from Python SDK. Using AWS CLI, we have direct commands available, see the below example for the same.

    aws s3 --recursive mv s3://<bucketname>/<folder_name_from>/<old_file_name> s3://<bucket>/<folder_name_to>/<new_file_name>

Using Python:

We need to copy files from source location to destination location and then delete the file(object) from the source location.

    import boto3

    client = boto3.client('s3')
    response = client.list_objects_v2(Bucket= your_bucket_name, Prefix = old_key_name)

    source_key = response["Contents"][0]["Key"]

    copy_source = {'Bucket': your_bucket_name, 'Key': source_key}

    client.copy_object(Bucket = your_dest_bucket_name, CopySource = copy_source, Key = old_key_name + your_destination_file_name)

    client.delete_object(Bucket = your_bucket_name, Key = source_key)

This process works to rename objects as well.

You can move — or rename — an object granting public read access through the ACL (Access Control List) of the new object. To do this, you have to pass the ACL to the copy_from method.

References

1. https://medium.com/plusteam/move-and-rename-objects-within-an-s3-bucket-using-boto-3-58b164790b78

2. https://www.stackvidhya.com/copy-move-files-between-buckets-using-boto3/

3. https://niyazierdogan.wordpress.com/2018/09/19/aws-s3-multipart-upload-with-python-and-boto3/

4. https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-examples.html