Problem Description
I'm experiencing an issue where images uploaded through Django (using boto3) to SeaweedFS's S3 API are corrupted, while uploads through S3 Browser desktop app work correctly. The uploaded files are 55 bytes larger than the original and contain a Content-Encoding: aws-chunked
header, making the images unopenable.
Environment Setup
- Storage: SeaweedFS with S3 API
- Proxy: nginx (handling SSL)
- Framework: Django
- Storage Client: boto3
- Upload Method: Using Django's storage backend with PrivateStorage
Issue Details
When uploading through S3 Browser desktop app:
- File size matches original
- Image opens correctly
- No corruption issues
When uploading through Django/boto3:
- File size increases by 55 bytes
- Response includes
Content-Encoding: aws-chunked
- Image becomes corrupted and unopenable
- First bytes contain unexpected data (
100000
) - Last bytes end with
.x-amz-checksum-
Example of Corrupted File
Original file size: 12345 bytes
Uploaded file size: 12400 bytes (+55 bytes)
First bytes: 100000...
Last bytes: ...x-amz-checksum-crc32:SJJ2UA==
Attempted Solutions
- Tried different upload methods:
# Method 1: Using ContentFile storage.save(path, ContentFile(file_content)) # Method 2: Using Django File object storage.save(path, File(file)) # Method 3: Direct boto3 upload client.upload_fileobj(f, bucket_name, path)
Questions
- Is this a known issue with SeaweedFS's S3 API implementation?
- Is there a way to disable the
aws-chunked
encoding in boto3? - Are there specific headers or configurations needed in the nginx proxy to handle binary uploads correctly?
Additional Information
- Django version: 5.1.7
- boto3 version: 1.37.23
Any help or insights would be greatly appreciated