最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

databricks - Delta Live Tables - Source data for the APPLY CHANGES must be a streaming query - Stack Overflow

programmeradmin0浏览0评论

Use Case

I am ingesting data using Fivetran, which syncs data from an Oracle database directly into my Databricks table. Fivetran manages the creation, updates, and inserts on these tables. As a result, my source is a static table in the Bronze layer.

Goal

I want to use Delta Live Tables (DLT) to stream data from the Bronze layer to the Silver and Gold layers.

Implementation

I have a SQL notebook with the following code:

CREATE OR REFRESH STREAMING TABLE cdc_test_silver;  

APPLY CHANGES INTO live.cdc_test_silver  
FROM lakehouse_poc.bronze.cdc_test  
KEYS (ID)  
SEQUENCE BY ModificationTime;

The objective is to create the Silver Delta Live Table using the Bronze Delta Table as the source.

Issue Encountered

I am receiving the following error:

Source data for the APPLY CHANGES target 'lakehouse_poc.bronze.cdc_test_silver' must be a streaming query.

Question

How can I handle this issue and successfully stream data from Bronze to Silver using Delta Live Tables?

发布评论

评论列表(0)

  1. 暂无评论