I’m encountering an error in Dremio when executing multiple queries via its REST API. After firing several queries, I get the following error:

"IndexWriter is closed"

Additionally: My current database and views stop appearing in the UI. After some time, API requests also fail to execute. What causes the IndexWriter is closed error in Dremio? Is this related to resource limits (e.g., memory/threads)?

I’m encountering an error in Dremio when executing multiple queries via its REST API. After firing several queries, I get the following error:

"IndexWriter is closed"

Additionally: My current database and views stop appearing in the UI. After some time, API requests also fail to execute. What causes the IndexWriter is closed error in Dremio? Is this related to resource limits (e.g., memory/threads)?

• dremio
• indexwriter

1 Answer 1

This error occurs because Dremio caches query results and logs metadata by default when executing queries via API. If too many queries are fired in succession:

1. Cache & Logs Fill Up Storage

   • Dremio stores query results in its distributed cache (accelerations) and maintains internal indexes (IndexWriter) for metadata.

   • If storage (disk/memory) gets full, the IndexWriter fails with this error, and metadata (databases/views) may disappear.

2. APIs Stop Responding

   • When storage is exhausted, Dremio’s internal services (e.g., catalog management) may freeze, causing API failures.

How to Fix It

1. Clear Cache Manually (Temporary Fix) - Restart Dremio to force cache cleanup.

2. Adjust Cache Settings (Permanent Fix) - Reduce cache expiration time

3. Increase Storage Monitoring

4. Clear Cache Manually (Temporary Fix) - Restart Dremio to force cache cleanup.

5. Adjust Cache Settings (Permanent Fix) - Reduce cache expiration time

6. Increase Storage Monitoring