I am trying to use s3fs.S3FileSystem with asynchronous operations.

I can successfully read and write files asynchronously using:

import s3fs

bucket_client = s3fs.S3FileSystem(asynchronous=True)

async with await bucket_client.open_async("path/to/file.xml", "rb") as f:
    xml = await f.read()

However, I am having trouble executing ls and glob operations asynchronously.

Issue 1: ls raises RuntimeError

When I try:

await bucket_client.ls("path/to/folder")

I get the following error:

RuntimeError: Loop is not running

Issue 2: ls raises TypeError

I also attempted using an async context manager:

async with bucket_client:
    bucket_client.ls("path/to/folder/**/*.xml")

But this raises:

TypeError: 'S3FileSystem' object does not support the asynchronous context manager protocol

Question:

• How can I properly run ls and glob asynchronously with s3fs.S3FileSystem?
• Is there a correct way to integrate this within an async workflow without running into event loop issues?

I am trying to use s3fs.S3FileSystem with asynchronous operations.

I can successfully read and write files asynchronously using:

import s3fs

bucket_client = s3fs.S3FileSystem(asynchronous=True)

async with await bucket_client.open_async("path/to/file.xml", "rb") as f:
    xml = await f.read()

However, I am having trouble executing ls and glob operations asynchronously.

Issue 1: ls raises RuntimeError

When I try:

await bucket_client.ls("path/to/folder")

I get the following error:

RuntimeError: Loop is not running

Issue 2: ls raises TypeError

I also attempted using an async context manager:

async with bucket_client:
    bucket_client.ls("path/to/folder/**/*.xml")

But this raises:

TypeError: 'S3FileSystem' object does not support the asynchronous context manager protocol

Question:

• How can I properly run ls and glob asynchronously with s3fs.S3FileSystem?
• Is there a correct way to integrate this within an async workflow without running into event loop issues?

• python
• s3fs

1 Answer 1

ok.... _ls and _glob are async