Alpha 0.1.0

DataOps411

Data pipeline operations. Right now.

Edition of

All releases

Integrations

Airflow Databricks provider 7.24.0

Released · Edition of

The current package adds an agent operator and carries forward the 7.23 change to query-lineage dependencies.

Dependencies
Version 7.24.0 requires Airflow 2.11.0 or later and databricks-sql-connector 4.4.0 or later. Apache Airflow
Upgrade
When upgrading across 7.23.0, update consumer filters for the AIRFLOW integration value and read query errors from the errorMessage facet. Apache Airflow
Version fallback
Dropping the OpenLineage provider below 2.16.0 with this Databricks provider suppresses query-lineage events. The provider logs the skipped emission. Apache Airflow

Milestone history

  1. Version 7.22.0 added downstream workflow notifications when COPY INTO writes a Unity table.

    Apache Airflow
  2. Version 7.23.0 moved query-lineage emission onto the OpenLineage public API.

    Apache Airflow
  3. Version 7.24.0 added the agent invocation hook and operator.

    Apache Airflow