最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

database - How to ingest ILP over HTTP without external dependencies? - Stack Overflow

programmeradmin0浏览0评论

I am using QuestDB and I can ingest data with ILP over HTTP using the client libraries, so there is an available HTTP endpoint in QuestDB for ingestion.

I would like to be able to ingest with no dependencies, just using curl or any other HTTP client, but when I try doing this:

curl -X PUT http://localhost:9000 --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

I get Method not supported

I am using QuestDB and I can ingest data with ILP over HTTP using the client libraries, so there is an available HTTP endpoint in QuestDB for ingestion.

I would like to be able to ingest with no dependencies, just using curl or any other HTTP client, but when I try doing this:

curl -X PUT http://localhost:9000 --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

I get Method not supported

Share Improve this question asked Nov 20, 2024 at 17:11 Javier RamirezJavier Ramirez 4,0321 gold badge27 silver badges36 bronze badges
Add a comment  | 

1 Answer 1

Reset to default 0

The url and port for ingestion are indeed (by default) http://localhost:9000, but the endpoint for ingestion is /write, so we have to send the data like this:

 curl -X POST http://localhost:9000/write --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\n'

It is worth noting that when using the client libraries, they take care of things like buffering rows (by number of rows or by frequency) or reconnecting in case of any transient failures, but when using a standalone HTTP client we won't be getting any of that.

At the very least, it is a good idea to write the data in batches, unless we have very low ingestion volume. Otherwise, the server will experience a lot of overhead due to small transactions. For reference, the official libraries have a default of 75K rows per batch (or 1000 milliseconds, whatever comes first).

Sending multiple rows in one go is as easy as separating them with newlines, as in:

 curl -X POST http://localhost:9000/write --data-binary $'trades,symbol=ETH-USD,side=sell price=2615.54,amount=0.00044 1646762637609765000\ntrades,symbol=BTC-USD,side=sell price=39269.98,amount=0.001 1646762637710419000'
``
发布评论

评论列表(0)

  1. 暂无评论