最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

jdbc - Kafka Connect JdbcSourceConnector fails loading from Clickhouse - Stack Overflow

programmeradmin2浏览0评论

I am trying to integrate Kafka Connect with Clickhouse. What I need is to load data from Clickhouse tables into Kafka topics. I am using clickhouse-jdbc-0.8.0.jar. Here is my configuration for connector:

"config": {
        "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
        "incrementing.column.name": "id",
        "connection.password": "password",
        "tasks.max": "1",
        "automit.interval": "10000",
        "batch.max.rows": "1000",
        "table.whitelist": "table_to_load_from",
        "mode": "incrementing",
        "key.converter.schemas.enable": "false",
        "topic.prefix": "clickhouse-",
        "connection.user": "user",
        "poll.interval.ms": "10000",
        "value.converter.schemas.enable": "false",
        "name": "clickhouse-source-connector",
        "connection.url": "jdbc:clickhouse://<ip>:8123",
        "value.converter": ".apache.kafka.connect.json.JsonConverter",
        "key.converter": ".apache.kafka.connect.json.JsonConverter"
    }

My tasks within this connector are failing with java.sql.SQLFeatureNotSupportedException: setAutoCommit = false not supported. I suppose this is because JdbcSourceConnector forces some params about DB commits, while Clickhouse driver can't understand it.

I tried setting autoCommit: "true" and auto.offset.reset: "earliest" but it didn't help.

So I wonder if it is even possible to load data from Clickhouse with JdbcSourceConnector. Or the only way is to write my own source connector?

发布评论

评论列表(0)

  1. 暂无评论