Upload Large Folders to Cloudflare R2

Cloudflare R2 object storage has a limitation: the web interface only allows uploading folders containing fewer than 100 files. To upload folders with more than 100 files, you typically need to set up Cloudflare Workers or use the S3 API with custom code.

Rclone makes this process easy.

Step 1 - Install Rclone

Rclone ↗ is a command-line tool for managing files on cloud storage. Rclone works well for uploading multiple files from your local machine or copying data from other cloud storage providers.

brew install rclone

Enter fullscreen mode

Exit fullscreen mode

Windows: Download the installer from rclone.org/install/#windows

Step 2 - Create Cloudflare API Keys

• From your Cloudflare R2 dashboard, click the Manage button.

• Create a new user API token:

• Enter a Token Name (e.g. r2-upload-token)

• For Permission, select Object Read & Write

• Under Specify buckets, choose the bucket(s) you want to allow access to or allow all.

After creation, you will receive: Access Key ID, Secret Access Token, Endpoint (e.g., https://.r2.cloudflarestorage.com)

⚠️ Save these credentials immediately because you won’t be able to see the secret key again.

Step 3 - Configure Rclone

Run the configuration command:

rclone config

Enter fullscreen mode

Exit fullscreen mode

• Select new remote

• Enter name of new remote (you'll use this later)

• Select storage > 4 (Amazon S3 Compliant Storage Providers)

• Select provider > 7 (Cloudflare)

• env_auth > 1 (Enter AWS credentials in the next step)

• Enter access_key_id

• Enter secret_access_key

• Select region - auto (Leave empty or enter 1)

• Enter endpoint

• Select No for Edit Advanced Config > n (n for No)

• Keep this remote > y

• Quit config > q

Step 4: Upload Your Folder

Use the rclone copy command:

rclone copy -vv :/

Enter fullscreen mode

Exit fullscreen mode

Example:

rclone copy -vv /Users/Dev/project/images my-rclone-remote:images/apparels

Enter fullscreen mode

Exit fullscreen mode

The -vv flag gives verbose output so you can watch the upload progress. rclone also skips any file that has already been uploaded.

Step 5: Verify the Upload

List the bucket and count the files:

rclone ls : | wc -l

Enter fullscreen mode

Exit fullscreen mode