[`airflow`] Update oudated `AIR301`, `AIR302` rules #17123

Lee-W · 2025-04-01T15:05:05Z

Summary

Some of the migration rules has been changed during Airflow 3 development. The following are new AIR302 rules. Corresponding AIR301 has also been removed.

airflow.sensors.external_task_sensor.ExternalTaskMarker → airflow.providers.standard.sensors.external_task.ExternalTaskMarker
airflow.sensors.external_task_sensor.ExternalTaskSensor → airflow.providers.standard.sensors.external_task.ExternalTaskSensor
airflow.sensors.external_task_sensor.ExternalTaskSensorLink → airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink
airflow.sensors.time_delta_sensor.TimeDeltaSensor → airflow.providers.standard.sensors.time_delta.TimeDeltaSensor
airflow.operators.dagrun_operator.TriggerDagRunLink → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
airflow.operators.dagrun_operator.TriggerDagRunOperator → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator
airflow.operators.python_operator.BranchPythonOperator → airflow.providers.standard.operators.python.BranchPythonOperator
airflow.operators.python_operator.PythonOperator → airflow.providers.standard.operators.python.PythonOperator
airflow.operators.python_operator.PythonVirtualenvOperator → airflow.providers.standard.operators.python.PythonVirtualenvOperator
airflow.operators.python_operator.ShortCircuitOperator → airflow.providers.standard.operators.python.ShortCircuitOperator
airflow.operators.latest_only_operator.LatestOnlyOperator → airflow.providers.standard.operators.latest_only.LatestOnlyOperator
airflow.sensors.date_time_sensor.DateTimeSensor → airflow.providers.standard.sensors.DateTimeSensor
airflow.operators.email_operator.EmailOperator → airflow.providers.smtp.operators.smtp.EmailOperator
airflow.operators.email.EmailOperator → airflow.providers.smtp.operators.smtp.EmailOperator
airflow.operators.bash.BashOperator → airflow.providers.standard.operators.bash.BashOperator
airflow.operators.EmptyOperator → airflow.providers.standard.operators.empty.EmptyOperator

closes: #17103

Test Plan

The test fixture has been updated and checked after each change and later reorganized in the latest commit

github-actions · 2025-04-01T15:11:31Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+6 -2 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+6 -2 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ airflow-core/tests/unit/utils/test_dot_renderer.py:103:22: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
+ airflow-core/tests/unit/utils/test_dot_renderer.py:83:22: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
- dev/airflow_perf/dags/perf_dag_1.py:41:10: AIR302 Import path `airflow.operators.bash_operator` is moved into `standard` provider in Airflow 3.0;
+ dev/airflow_perf/dags/perf_dag_1.py:41:10: AIR302 `airflow.operators.bash_operator.BashOperator` is moved into `standard` provider in Airflow 3.0;
- dev/airflow_perf/dags/perf_dag_1.py:48:12: AIR302 Import path `airflow.operators.bash_operator` is moved into `standard` provider in Airflow 3.0;
+ dev/airflow_perf/dags/perf_dag_1.py:48:12: AIR302 `airflow.operators.bash_operator.BashOperator` is moved into `standard` provider in Airflow 3.0;
+ performance/src/performance_dags/performance_dag/performance_dag.py:111:13: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;
+ providers/ydb/tests/system/ydb/example_ydb.py:111:23: AIR302 `airflow.operators.python.PythonOperator` is moved into `standard` provider in Airflow 3.0;

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
AIR302	8	6	2	0	0

…#17123

crates/ruff_linter/src/rules/airflow/rules/moved_to_provider_in_3.rs

Lee-W · 2025-04-04T12:21:17Z

@ntBre Would be nice if we can get a quick review on this one 🙂 Thanks 🙏

ntBre · 2025-04-04T12:53:55Z

Will do! Feel free to ping me or request my review on any of these. There's a chance I could have missed them before taking over from Dhruv. I assigned myself to a few last night :)

ntBre

Thanks! Just a few nits/questions

crates/ruff_linter/src/rules/airflow/rules/moved_to_provider_in_3.rs

crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs

…irflow 3 change * ``airflow.operators.bash.BashOperator`` → ``airflow.providers.standard.operators.bash.BashOperator`` * ``airflow.operators.EmptyOperator`` → ``airflow.providers.standard.operators.empty.EmptyOperator``

… cases

* ``airflow.operators.email_operator.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator`` * ``airflow.operators.email.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator``

…be recategorized as AIR302 due to rule change * airflow.sensors.external_task_sensor.ExternalTaskMarker * airflow.sensors.external_task_sensor.ExternalTaskSensor * airflow.sensors.external_task_sensor.ExternalTaskSensorLink * airflow.sensors.time_delta_sensor.TimeDeltaSensor * airflow.operators.dagrun_operator.TriggerDagRunLink * airflow.operators.dagrun_operator.TriggerDagRunOperator * airflow.operators.python_operator.BranchPythonOperator * airflow.operators.python_operator.PythonOperator * airflow.operators.python_operator.PythonVirtualenvOperator * airflow.operators.python_operator.ShortCircuitOperator * airflow.operators.latest_only_operator.LatestOnlyOperator * airflow.operators.bash_operator.BashOperator * airflow.operators.branch_operator.BaseBranchOperator

…be recategorized as AIR302 due to rule change * airflow.sensors.date_time_sensor.DateTimeSensor

* airflow.sensors.external_task_sensor.ExternalTaskMarker → airflow.providers.standard.sensors.external_task.ExternalTaskMarker * airflow.sensors.external_task_sensor.ExternalTaskSensor → airflow.providers.standard.sensors.external_task.ExternalTaskSensor * airflow.sensors.external_task_sensor.ExternalTaskSensorLink → airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink * airflow.sensors.time_delta_sensor.TimeDeltaSensor → airflow.providers.standard.sensors.time_delta.TimeDeltaSensor * airflow.operators.dagrun_operator.TriggerDagRunLink → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink * airflow.operators.dagrun_operator.TriggerDagRunOperator → airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator * airflow.operators.python_operator.BranchPythonOperator → airflow.providers.standard.operators.python.BranchPythonOperator * airflow.operators.python_operator.PythonOperator → airflow.providers.standard.operators.python.PythonOperator * airflow.operators.python_operator.PythonVirtualenvOperator → airflow.providers.standard.operators.python.PythonVirtualenvOperator * airflow.operators.python_operator.ShortCircuitOperator → airflow.providers.standard.operators.python.ShortCircuitOperator * airflow.operators.latest_only_operator.LatestOnlyOperator → airflow.providers.standard.operators.latest_only.LatestOnlyOperator * airflow.sensors.date_time_sensor.DateTimeSensor → airflow.providers.standard.sensors.DateTimeSensor

Lee-W · 2025-04-07T01:47:18Z

I think this one is ready again. Thanks!

ntBre

Thanks!

* main: (42 commits) [playground] New default program (#17277) [red-knot] Add `--python-platform` CLI option (#17284) [red-knot] Allow ellipsis default params in stub functions (#17243) [red-knot] Fix stale syntax errors in playground (#17280) Update Rust crate clap to v4.5.35 (#17273) Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061) [ci] Fix pattern for code changes (#17275) [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123) [docs] fix formatting of "See Style Guide" link (#17272) [red-knot] Support stub packages (#17204) ruff_annotate_snippets: address unused code warnings [red-knot] Add a couple more tests for `*` imports (#17270) [red-knot] Add 'Format document' to playground (#17217) Update actions/setup-node action to v4.3.0 (#17259) Update actions/upload-artifact action to v4.6.2 (#17261) Update actions/download-artifact action to v4.2.1 (#17258) Update actions/setup-python action to v5.5.0 (#17260) Update actions/cache action to v4.2.3 (#17256) Update Swatinem/rust-cache action to v2.7.8 (#17255) Update actions/checkout action to v4.2.2 (#17257) ...

…#17123

…#17123 (#48634)

* main: (222 commits) [playground] New default program (#17277) [red-knot] Add `--python-platform` CLI option (#17284) [red-knot] Allow ellipsis default params in stub functions (#17243) [red-knot] Fix stale syntax errors in playground (#17280) Update Rust crate clap to v4.5.35 (#17273) Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061) [ci] Fix pattern for code changes (#17275) [`airflow`] Update oudated `AIR301`, `AIR302` rules (#17123) [docs] fix formatting of "See Style Guide" link (#17272) [red-knot] Support stub packages (#17204) ruff_annotate_snippets: address unused code warnings [red-knot] Add a couple more tests for `*` imports (#17270) [red-knot] Add 'Format document' to playground (#17217) Update actions/setup-node action to v4.3.0 (#17259) Update actions/upload-artifact action to v4.6.2 (#17261) Update actions/download-artifact action to v4.2.1 (#17258) Update actions/setup-python action to v5.5.0 (#17260) Update actions/cache action to v4.2.3 (#17256) Update Swatinem/rust-cache action to v2.7.8 (#17255) Update actions/checkout action to v4.2.2 (#17257) ...

Lee-W marked this pull request as draft April 1, 2025 15:05

Lee-W changed the title ~~Update air302 303 rules~~ [airflow] update oudated AIR302, AIR303 rules Apr 1, 2025

Lee-W mentioned this pull request Apr 1, 2025

Airflow 3.0 deprecations #14626

Open

2 tasks

Lee-W force-pushed the update-AIR302-303-rules branch from 2c0eaf2 to 117f1ae Compare April 1, 2025 15:28

Lee-W mentioned this pull request Apr 2, 2025

Organize ruff rules as required changes for Airflow 3.0 and suggested changes apache/airflow#48560

Closed

6 tasks

Lee-W force-pushed the update-AIR302-303-rules branch from 117f1ae to b7be1f4 Compare April 2, 2025 10:02

Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 2, 2025

docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…

cc95258

…#17123

Lee-W force-pushed the update-AIR302-303-rules branch 3 times, most recently from 914825c to 37057cd Compare April 3, 2025 02:58

Lee-W changed the title ~~[airflow] update oudated AIR302, AIR303 rules~~ [airflow] update oudated AIR301, AIR302 rules Apr 3, 2025

Lee-W marked this pull request as ready for review April 3, 2025 02:59

Lee-W commented Apr 3, 2025

View reviewed changes

crates/ruff_linter/src/rules/airflow/rules/moved_to_provider_in_3.rs Show resolved Hide resolved

ntBre self-assigned this Apr 3, 2025

ntBre approved these changes Apr 4, 2025

View reviewed changes

Lee-W added 12 commits April 7, 2025 09:06

test(AIR301): remove DummyOperator and EmptyOperator from AIR301 test…

c4cf780

… cases

test(AIR302): add test cases for DummyOperator and EmptyOperator

baeb1f9

feat(AIR302): extend the following rules

016624d

* ``airflow.operators.email_operator.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator`` * ``airflow.operators.email.EmailOperator`` → ``airflow.providers.smtp.operators.smtp.EmailOperator``

test(AIR301): clean up unused variable in test fixtures

5351a3d

feat(AIR302): remove the following checks from AIR301 as they should …

1f320d5

…be recategorized as AIR302 due to rule change * airflow.sensors.date_time_sensor.DateTimeSensor

test(AIR301): clean up unused test cases

7cd6cef

test: reorg test cases

a252339

fix(AIR302): fix S3KeySensor path error

8f37995

style: add missed style

44e5cb6

Lee-W added 3 commits April 7, 2025 09:06

test: reorganize test cases

16212f5

refactor(AIR302): refactor match condition as suggested

ee057f6

docs: remove unnecesay TODO

eaa3656

Lee-W force-pushed the update-AIR302-303-rules branch from 37057cd to eaa3656 Compare April 7, 2025 01:46

Lee-W requested a review from ntBre April 7, 2025 01:47

MichaReiser added rule Implementing or modifying a lint rule preview Related to preview mode features labels Apr 7, 2025

ntBre changed the title ~~[airflow] update oudated AIR301, AIR302 rules~~ [airflow] Update oudated AIR301, AIR302 rules Apr 7, 2025

ntBre approved these changes Apr 7, 2025

View reviewed changes

ntBre merged commit 1e9e423 into astral-sh:main Apr 7, 2025
22 checks passed

Lee-W added a commit to astronomer/airflow that referenced this pull request Apr 8, 2025

docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…

eb27b06

…#17123

Lee-W mentioned this pull request Apr 8, 2025

docs(newsfragments): mark implemented AIR302 rules as Done apache/airflow#48634

Merged

Lee-W added a commit to apache/airflow that referenced this pull request Apr 8, 2025

docs(newsfragments): check AIR303 rules implemented in astral-sh/ruff…

bcddef3

…#17123 (#48634)

BrewTestBot mentioned this pull request Apr 10, 2025

ruff 0.11.5 Homebrew/homebrew-core#219299

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[`airflow`] Update oudated `AIR301`, `AIR302` rules #17123

[`airflow`] Update oudated `AIR301`, `AIR302` rules #17123

Lee-W commented Apr 1, 2025 •

edited

Loading

github-actions bot commented Apr 1, 2025 •

edited

Loading

Lee-W commented Apr 4, 2025

ntBre commented Apr 4, 2025

ntBre left a comment

Lee-W commented Apr 7, 2025

ntBre left a comment

[airflow] Update oudated AIR301, AIR302 rules #17123

[airflow] Update oudated AIR301, AIR302 rules #17123

Conversation

Lee-W commented Apr 1, 2025 • edited Loading

Summary

Test Plan

github-actions bot commented Apr 1, 2025 • edited Loading

ruff-ecosystem results

Linter (stable)

Linter (preview)

Lee-W commented Apr 4, 2025

ntBre commented Apr 4, 2025

ntBre left a comment

Choose a reason for hiding this comment

Lee-W commented Apr 7, 2025

ntBre left a comment

Choose a reason for hiding this comment

[`airflow`] Update oudated `AIR301`, `AIR302` rules #17123

[`airflow`] Update oudated `AIR301`, `AIR302` rules #17123

Lee-W commented Apr 1, 2025 •

edited

Loading

github-actions bot commented Apr 1, 2025 •

edited

Loading

`ruff-ecosystem` results