最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

python - Image upload corruption with SeaweedFS S3 API - Stack Overflow

programmeradmin2浏览0评论

Problem Description

I'm experiencing an issue where images uploaded through Django (using boto3) to SeaweedFS's S3 API are corrupted, while uploads through S3 Browser desktop app work correctly. The uploaded files are 55 bytes larger than the original and contain a Content-Encoding: aws-chunked header, making the images unopenable.

Environment Setup

  • Storage: SeaweedFS with S3 API
  • Proxy: nginx (handling SSL)
  • Framework: Django
  • Storage Client: boto3
  • Upload Method: Using Django's storage backend with PrivateStorage

Issue Details

  1. When uploading through S3 Browser desktop app:

    • File size matches original
    • Image opens correctly
    • No corruption issues
  2. When uploading through Django/boto3:

    • File size increases by 55 bytes
    • Response includes Content-Encoding: aws-chunked
    • Image becomes corrupted and unopenable
    • First bytes contain unexpected data (100000)
    • Last bytes end with .x-amz-checksum-

Example of Corrupted File

Original file size: 12345 bytes
Uploaded file size: 12400 bytes (+55 bytes)
First bytes: 100000...
Last bytes: ...x-amz-checksum-crc32:SJJ2UA==

Attempted Solutions

  1. Tried different upload methods:
    # Method 1: Using ContentFile
    storage.save(path, ContentFile(file_content))
    
    # Method 2: Using Django File object
    storage.save(path, File(file))
    
    # Method 3: Direct boto3 upload
    client.upload_fileobj(f, bucket_name, path)
    

Questions

  1. Is this a known issue with SeaweedFS's S3 API implementation?
  2. Is there a way to disable the aws-chunked encoding in boto3?
  3. Are there specific headers or configurations needed in the nginx proxy to handle binary uploads correctly?

Additional Information

  • Django version: 5.1.7
  • boto3 version: 1.37.23

Any help or insights would be greatly appreciated

发布评论

评论列表(0)

  1. 暂无评论