Is there any way to stop or pause an already running optmize on delta table started using

val deltaTable = DeltaTable.forPath(spark, deltaTablePath) 
deltaTable.optimize().executeCompaction()

Is there any way to stop or pause an already running optmize on delta table started using

val deltaTable = DeltaTable.forPath(spark, deltaTablePath) 
deltaTable.optimize().executeCompaction()

• apache-spark
• databricks
• delta

1 Answer 1

To stop or pause a running OPTIMIZE operation on a Delta table (started via executeCompaction()), there is no direct API in Delta Lake 2.x or earlier. If the OPTIMIZE is running as a Spark job, you can kill the associated Spark job.