I am using the tantivy rust library for searching. I have one large search index, with many documents that needs to be shared across all users.

However, some users also have some private data in the same structure that needs to be searchable. This data is sensitive, volatile and I do not want to write it to the existing tantivy index.

Is there any way to write temporarily to a tantivy index, or compose a search across multiple indexes?

In a perfect world, I want to add some documents to a tantivy index temporarily without persisting to disk, or showing it to any other users than the current user. However, I have not been able to find any methods that would allow me to do this.

My other alternative is to create an entirely new index in a RamDirectory, however the scores for my large index and my small ram-based index will be completely different, and thus hard to compare.