I have an application that receives a lot of tasks to be executed in the background from AWS SQS.
I noticed that a db logging mechanism is implemented, and it may add around 30 log entries via JPA/Hibernate per task.
Also, it creates a new transaction for each Log ( I don't like it, yet the biz requires every Log to be visible immediately upon query ).
I have been searching for a way to buffer the insertions and make a batch insertion requiring much less transactions.
Here are my research results
- Spring / Hibernate caching works only for reading from DB, not buffering insertions ( maybe I am mistaken ).
- Hibernate batch operations, set the batch size to a suitable size, but I have no control over the query, and not sure if it will periodically flush or can even allow this on a specific entity not all statements.
- Manually create a buffer and manage it, then flush the buffer to DB periodically via background thread. Also the query must go through the buffer to get any logs yet not inserted.
Is there a way except the manual buffer ?