I am trying to integrate Kafka Connect with Clickhouse. What I need is to load data from Clickhouse tables into Kafka topics. I am using clickhouse-jdbc-0.8.0.jar. Here is my configuration for connector:
"config": {
"connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
"incrementing.column.name": "id",
"connection.password": "password",
"tasks.max": "1",
"automit.interval": "10000",
"batch.max.rows": "1000",
"table.whitelist": "table_to_load_from",
"mode": "incrementing",
"key.converter.schemas.enable": "false",
"topic.prefix": "clickhouse-",
"connection.user": "user",
"poll.interval.ms": "10000",
"value.converter.schemas.enable": "false",
"name": "clickhouse-source-connector",
"connection.url": "jdbc:clickhouse://<ip>:8123",
"value.converter": ".apache.kafka.connect.json.JsonConverter",
"key.converter": ".apache.kafka.connect.json.JsonConverter"
}
My tasks within this connector are failing with java.sql.SQLFeatureNotSupportedException: setAutoCommit = false not supported
. I suppose this is because JdbcSourceConnector forces some params about DB commits, while Clickhouse driver can't understand it.
I tried setting autoCommit: "true"
and auto.offset.reset: "earliest"
but it didn't help.
So I wonder if it is even possible to load data from Clickhouse with JdbcSourceConnector. Or the only way is to write my own source connector?