How Efficiently Download Folders with Folder Structure from Amazon S3 Bucket: A Step-by-Step Guide"

Introduction:
Amazon S3 (Simple Storage Service) is a popular cloud storage solution that allows users to store and retrieve any amount of data at any time. this blog will walk you through a step-by-step guide on how to efficiently zip folders with their folder structure from an Amazon S3 bucket using the Boto3 Python library.

Although S3 does not have a concept of folders, the key of each file has folder information as prefixes. Each folder level is delimited by ‘/’ and followed by the file name. (i.e., folder1/sub1/image.png)

This blog is trying in the AWS lambda function to zip the entire folder and upload to a bucket so anyone easily downloads from there.

Prerequisites:
Before we begin, make sure you have the following prerequisites:
1. An AWS account with appropriate access rights to the desired S3 bucket.
2. Import boto3 and required libraries
3. Basic knowledge of Python and Boto3.

The above code import required packages, create an s3 client for working with s3 services and select the bucket name and folder name.

Create an im-memory zip file for inserting folders.

in response get all objects inside the folder

by using for loop to fetch all keys and get the object of that keys.

Now in line No.29 Most important work happening normally developers give their base as input it flattered the folder structure because names are going to duplicate. so if you give a key it creates a folder structure that presents in the key.

Normally keys for files are long it creating many empty folders by using the proper key folder structure create as required.

Other Scenarios

Business requirements might be zipping files and folders same time that time few changes in the above code fulfill the above requirements

The above code is now more versatile and dynamic in just sending the path of folders and files to lambda function by sqs or API it process folders and files and converts them into zip without changing folder structure.

Even you can folder structure by giving full path or key of one file like

code creates all folder and sub folders.

Conclusion :

this blog demonstrated code on how to zip folders with their folder structure from an Amazon S3 bucket using the Boto3 Python library inside the AWS lambda function. By utilizing Boto3's powerful S3 functionalities, you can efficiently manage and compress data in the cloud, making it easier to share and organize your files. Start implementing this knowledge in your Python projects to enhance your Amazon S3 experience. Happy zipping!