Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
0
Release date:
April 27, 2022
Previous version:
2.3.0rc2
(released April 27, 2022)
Magnitude:
0
Diff Delta
Contributors:
0 total committers
Commits:
0 Commits in this Release
Could not find any commits associated with this release.
Browse Other Releases
Latest Pending
Unreleased π
2.3.2
Released June 1, 2022
0 Ξ
2.3.2rc2
Released June 1, 2022
151,344 Ξ
2.3.2rc1
Released May 30, 2022
2,803 Ξ
2.3.1
Released May 21, 2022
0 Ξ
2.3.1rc1
Released May 21, 2022
9,764 Ξ
2.3.0
Released April 27, 2022
0 Ξ
2.3.0rc2
Released April 27, 2022
103 Ξ
2.3.0rc1
Released April 27, 2022
11,225 Ξ
2.3.0b1
Released April 14, 2022
130,178 Ξ
2.2.5
Released March 30, 2022
0 Ξ
Top Contributors in 2.3.0
Could not determine top contributors for this release.
Directory Browser for 2.3.0
We haven't yet finished calculating and confirming the files and directories changed in this release. Please check back soon.
Release Notes Published
New Features
- Add dynamic task mapping (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-42+milestone%3A%22Airflow+2.3.0%22)
- New Grid View replaces Tree View (#18675)
- Templated
requirements.txt
in Python Operators (#17349) - Allow reuse of decorated tasks (#22941)
- Move the database configuration to a new section (#22284)
- Add
SmoothOperator
(#22813) - Make operator's
execution_timeout
configurable (#22389) - Events Timetable (#22332)
- Support dag serialization with custom
ti_deps
rules (#22698) - Support log download in task log view (#22804)
- support for continue backfill on failures (#22697)
- Add
dag-processor
cli command (#22305) - Add possibility to create users in LDAP mode (#22619)
- Add
ignore_first_depends_on_past
for scheduled jobs (#22491) - Update base sensor operator to support XCOM return value (#20656)
- Add an option for run id in the ui trigger screen (#21851)
- Enable JSON serialization for connections (#19857)
- Add REST API endpoint for bulk update of DAGs (#19758)
- Add queue button to click-on-DagRun interface. (#21555)
- Add
list-import-errors
toairflow dags
command (#22084) - Store callbacks in database if
standalone_dag_processor
config is True. (#21731) - Add LocalKubernetesExecutor (#19729)
- Add
celery.task_timeout_error
metric (#21602) - Airflow
db downgrade
cli command (#21596) - Add
ALL_SKIPPED
trigger rule (#21662) - Add
db clean
CLI command for purging old data (#20838) - Add
celery_logging_level
(#21506) - Support different timeout value for dag file parsing (#21501)
- Support generating SQL script for upgrades (#20962)
- Add option to compress Serialized dag data (#21332)
- Branch python operator decorator (#20860)
- Add Audit Log View to Dag View (#20733)
- Add missing StatsD metric for failing SLA Callback notification (#20924)
- Add
ShortCircuitOperator
configurability for respecting downstream trigger rules (#20044) - Allow using Markup in page title in Webserver (#20888)
- Add Listener Plugin API that tracks TaskInstance state changes (#20443)
- Add context var hook to inject more env vars (#20361)
- Add a button to set all tasks to skipped (#20455)
- Cleanup pending pods (#20438)
- Add config to warn public deployment exposure in UI (#18557)
- Log filename template records (#20165)
- Added windows extensions (#16110)
- Showing approximate time until next dag_run in Airflow (#20273)
- Extend config window on UI (#20052)
- Add show dag dependencies feature to CLI (#19985)
- Add cli command for 'airflow dags reserialize` (#19471)
- Add missing description field to Pool schema(REST API) (#19841)
- Introduce DagRun action to change state to queued. (#19353)
- Add DAG run details page (#19705)
- Add role export/import to cli tools (#18916)
- Adding
dag_id_pattern
parameter to the/dags
endpoint (#18924)
Improvements
- Show schedule_interval/timetable description in UI (#16931)
- Added column duration to DAG runs view (#19482)
- Enable use of custom conn extra fields without prefix (#22607)
- Initialize finished counter at zero (#23080)
- Improve logging of optional provider features messages (#23037)
- Meaningful error message in resolve_template_files (#23027)
- Update ImportError items instead of deleting and recreating them (#22928)
- Add option
--skip-init
to db reset command (#22989) - Support importing connections from files with ".yml" extension (#22872)
- Support glob syntax in
.airflowignore
files (#21392) (#22051) - Hide pagination when data is a single page (#22963)
- Support for sorting DAGs in the web UI (#22671)
- Speed up
has_access
decorator by ~200ms (#22858) - Add XComArg to lazy-imported list of Airflow module (#22862)
- Add more fields to REST API dags/dag_id/details endpoint (#22756)
- Don't show irrelevant/duplicated/"internal" Task attrs in UI (#22812)
- No need to load whole ti in current_state (#22764)
- Pickle dag exception string fix (#22760)
- Better verification of Localexecutor's parallelism option (#22711)
- log backfill exceptions to sentry (#22704)
- retry commit on MySQL deadlocks during backfill (#22696)
- Add more fields to REST API get DAG(dags/dag_id) endpoint (#22637)
- Use timetable to generate planned days for current year (#22055)
- Disable connection pool for celery worker (#22493)
- Make date picker label visible in trigger dag view (#22379)
- Expose
try_number
in airflow vars (#22297) - Add generic connection type (#22310)
- Add a few more fields to the taskinstance finished log message (#22262)
- Pause auto-refresh if scheduler isn't running (#22151)
- Show DagModel details. (#21868)
- Add pip_install_options to PythonVirtualenvOperator (#22158)
- Show import error for
airflow dags list
CLI command (#21991) - Pause auto-refresh when page is hidden (#21904)
- Default args type check (#21809)
- Enhance magic methods on XComArg for UX (#21882)
- py files don't have to be checked
is_zipfiles
in refresh_dag (#21926) - Fix TaskDecorator type hints (#21881)
- Add 'Show record' option for variables (#21342)
- Use DB where possible for quicker
airflow dag
subcommands (#21793) - REST API: add rendered fields in task instance. (#21741)
- Change the default auth backend to session (#21640)
- Don't check if
py
DAG files are zipped during parsing (#21538) - Switch XCom implementation to use
run_id
(#20975) - Action log on Browse Views (#21569)
- Implement multiple API auth backends (#21472)
- Change logging level details of connection info in
get_connection()
(#21162) - Support mssql in airflow db shell (#21511)
- Support config
worker_enable_remote_control
for celery (#21507) - Log memory usage in
CgroupTaskRunner
(#21481) - Modernize DAG-related URL routes and rename "tree" to "grid" (#20730)
- Move Zombie detection to
SchedulerJob
(#21181) - Improve speed to run
airflow
by 6x (#21438) - Add more SQL template fields renderers (#21237)
- Simplify fab has access lookup (#19294)
- Log context only for default method (#21244)
- Log trigger status only if at least one is running (#21191)
- Add optional features in providers. (#21074)
- Better multiple_outputs inferral for @task.python (#20800)
- Improve handling of string type and non-attribute
template_fields
(#21054) - Remove un-needed deps/version requirements (#20979)
- Correctly specify overloads for TaskFlow API for type-hinting (#20933)
- Introduce notification_sent to SlaMiss view (#20923)
- Rewrite the task decorator as a composition (#20868)
- Add "Greater/Smaller than or Equal" to filters in the browse views (#20602) (#20798)
- Rewrite DAG run retrieval in task command (#20737)
- Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-130% (#20722)
- Make native environment Airflow-flavored like sandbox (#20704)
- Better error when param value has unexpected type (#20648)
- Add filter by state in DagRun REST API (List Dag Runs) (#20485)
- Prevent exponential memory growth in Tasks with custom logging handler (#20541)
- Set default logger in logging Mixin (#20355)
- Reduce deprecation warnings from www (#20378)
- Add hour and minute to time format on x-axis of all charts using nvd3.lineChart (#20002)
- Add specific warning when Task asks for more slots than pool defined with (#20178)
- UI: Update duration column for better human readability (#20112)
- Use Viewer role as example public role (#19215)
- Properly implement DAG param dict copying (#20216)
ShortCircuitOperator
push XCom by returning python_callable result (#20071)- Add clear logging to tasks killed due to a Dagrun timeout (#19950)
- Change log level for Zombie detection messages (#20204)
- Better confirmation prompts (#20183)
- Only execute TIs of running DagRuns (#20182)
- Check and run migration in commands if necessary (#18439)
- Log only when Zombies exists (#20118)
- Increase length of the email and username (#19932)
- Add more filtering options for TI's in the UI (#19910)
- Dynamically enable "Test Connection" button by connection type (#19792)
- Avoid littering postgres server logs with "could not obtain lock" with HA schedulers (#19842)
- Renamed
Connection.get_hook
parameter to make it the same as inSqlSensor
andSqlOperator
. (#19849) - Add hook_params in SqlSensor using the latest changes from PR #18718. (#18431)
- Speed up webserver boot time by delaying provider initialization (#19709)
- Configurable logging of
XCOM
value in PythonOperator (#19378) - Minimize production js files (#19658)
- Add
hook_params
inBaseSqlOperator
(#18718) - Add missing "end_date" to hash components (#19281)
- More friendly output of the airflow plugins command + add timetables (#19298)
- Add sensor default timeout config (#19119)
- Update
taskinstance
REST API schema to include dag_run_id field (#19105) - Adding feature in bash operator to append the user defined env variable to system env variable (#18944)
- Duplicate Connection: Added logic to query if a connection id exists before creating one (#18161)
Bug Fixes
- Use inherited 'trigger_tasks' method (#23016)
- In DAG dependency detector, use class type instead of class name (#21706)
- Fix tasks being wrongly skipped by schedule_after_task_execution (#23181)
- Fix X-Frame enabled behaviour (#23222)
- Allow
extra
to be nullable in connection payload as per schema(REST API). (#23183) - Fix
dag_id
extraction for dag level access checks in web ui (#23015) - Fix timezone display for logs on UI (#23075)
- Include message in graph errors (#23021)
- Change trigger dropdown left position (#23013)
- Don't add planned tasks for legacy DAG runs (#23007)
- Add dangling rows check for TaskInstance references (#22924)
- Validate the input params in connection
CLI
command (#22688) - Fix trigger event payload is not persisted in db (#22944)
- Drop "airflow moved" tables in command
db reset
(#22990) - Add max width to task group tooltips (#22978)
- Add template support for
external_task_ids
. (#22809) - Allow
DagParam
to hold falsy values (#22964) - Fix regression in pool metrics (#22939)
- Priority order tasks even when using pools (#22483)
- Do not clear XCom when resuming from deferral (#22932)
- Handle invalid JSON metadata in
get_logs_with_metadata endpoint
. (#22898) - Fix pre-upgrade check for rows dangling w.r.t. dag_run (#22850)
- Fixed backfill interference with scheduler (#22701)
- Support conf param override for backfill runs (#22837)
- Correctly interpolate pool name in
PoolSlotsAvailableDep
statues (#22807) - Fix
email_on_failure
withrender_template_as_native_obj
(#22770) - Fix processor cleanup on
DagFileProcessorManager
(#22685) - Prevent meta name clash for task instances (#22783)
- remove json parse for gantt chart (#22780)
- Check for missing dagrun should know version (#22752)
- Fixes
ScheduleInterval
spec (#22635) - Fixing task status for non-running and non-committed tasks (#22410)
- Do not log the hook connection details even at DEBUG level (#22627)
- Stop crashing when empty logs are received from kubernetes client (#22566)
- Fix bugs about timezone change (#22525)
- Fix entire DAG stops when one task has end_date (#20920)
- Use logger to print message during task execution. (#22488)
- Make sure finalizers are not skipped during exception handling (#22475)
- update smart sensor docs and minor fix on
is_smart_sensor_compatible()
(#22386) - Fix
run_id
k8s and elasticsearch compatibility with Airflow 2.1 (#22385) - Allow to
except_skip
None onBranchPythonOperator
(#20411) - Fix incorrect datetime details (DagRun views) (#21357)
- Remove incorrect deprecation warning in secrets backend (#22326)
- Remove
RefreshConfiguration
workaround for K8s token refreshing (#20759) - Masking extras in GET
/connections/<connection>
endpoint (#22227) - Set
queued_dttm
when submitting task to directly to executor (#22259) - Addressed some issues in the tutorial mentioned in discussion #22233 (#22236)
- Change default python executable to python3 for docker decorator (#21973)
- Don't validate that Params are JSON when NOTSET (#22000)
- Add per-DAG delete permissions (#21938)
- Fix handling some None parameters in kubernetes 23 libs. (#21905)
- Fix handling of empty (None) tags in
bulk_write_to_db
(#21757) - Fix DAG date range bug (#20507)
- Removed
request.referrer
from views.py (#21751) - Make
DbApiHook
useget_uri
from Connection (#21764) - Fix some migrations (#21670)
- [de]serialize resources on task correctly (#21445)
- Add params
dag_id
,task_id
etc toXCom.serialize_value
(#19505) - Update test connection functionality to use custom form fields (#21330)
- fix all "high" npm vulnerabilities (#21526)
- Fix bug incorrectly removing action from role, rather than permission. (#21483)
- Fix relationship join bug in FAB/SecurityManager with SQLA 1.4 (#21296)
- Use Identity instead of Sequence in SQLAlchemy 1.4 for MSSQL (#21238)
- Ensure
on_task_instance_running
listener can get at task (#21157) - Return to the same place when triggering a DAG (#20955)
- Fix task ID deduplication in
@task_group
(#20870) - Add downgrade to some FAB migrations (#20874)
- Only validate Params when DAG is triggered (#20802)
- Fix
airflow trigger
cli (#20781) - Fix task instances iteration in a pool to prevent blocking (#20816)
- Allow depending to a
@task_group
as a whole (#20671) - Use original task's
start_date
if a task continues after deferral (#20062) - Disabled edit button in task instances list view page (#20659)
- Fix a package name import error (#20519) (#20519)
- Remove
execution_date
label when get cleanup pods list (#20417) - Remove unneeded FAB REST API endpoints (#20487)
- Fix parsing of Cloudwatch log group arn containing slashes (#14667) (#19700)
- Sanity check for MySQL's TIMESTAMP column (#19821)
- Allow using default celery command group with executors subclassed from Celery-based executors. (#18189)
- Move
class_permission_name
to mixin so it applies to all classes (#18749) - Adjust trimmed_pod_id and replace '.' with '-' (#19036)
- Pass custom_headers to send_email and send_email_smtp (#19009)
- Ensure
catchup=False
is used in example dags (#19396) - Edit permalinks in OpenApi description file (#19244)
- Navigate directly to DAG when selecting from search typeahead list (#18991)
- [Minor] Fix padding on home page (#19025)
Doc only changes
- Update doc for DAG file processing (#23209)
- Replace changelog/updating with release notes and
towncrier
now (#22003) - Fix wrong reference in tracking-user-activity.rst (#22745)
- Remove references to
rbac = True
from docs (#22725) - Doc: Update description for executor-bound dependencies (#22601)
- Update check-health.rst (#22372)
- Stronger language about Docker Compose customizability (#22304)
- Update logging-tasks.rst (#22116)
- Add example config of
sql_alchemy_connect_args
(#22045) - Update best-practices.rst (#22053)
- Add information on DAG pausing/deactivation/deletion (#22025)
- Add brief examples of integration test dags you might want (#22009)
- Run inclusive language check on CHANGELOG (#21980)
- Add detailed email docs for Sendgrid (#21958)
- Add docs for
db upgrade
/db downgrade
(#21879) - Update modules_management.rst (#21889)
- Fix UPDATING section on SqlAlchemy 1.4 scheme changes (#21887)
- Update TaskFlow tutorial doc to show how to pass "operator-level" args. (#21446)
- Fix doc - replace decreasing by increasing (#21805)
- Add another way to dynamically generate DAGs to docs (#21297)
- Add extra information about time synchronization needed (#21685)
- Update debug.rst docs (#21246)
- Replaces the usage of
postgres://
withpostgresql://
(#21205) - Fix task execution process in
CeleryExecutor
docs (#20783)
Misc/Internal
- Bring back deprecated security manager functions (#23243)
- Replace usage of
DummyOperator
withEmptyOperator
(#22974) - Deprecate
DummyOperator
in favor ofEmptyOperator
(#22832) - Remove unnecessary python 3.6 conditionals (#20549)
- Bump
moment
from 2.29.1 to 2.29.2 in /airflow/www (#22873) - Bump
prismjs
from 1.26.0 to 1.27.0 in /airflow/www (#22823) - Bump
nanoid
from 3.1.23 to 3.3.2 in /airflow/www (#22803) - Bump
minimist
from 1.2.5 to 1.2.6 in /airflow/www (#22798) - Remove dag parsing from db init command (#22531)
- Update our approach for executor-bound dependencies (#22573)
- Use
Airflow.Base.metadata
in FAB models (#22353) - Limit docutils to make our documentation pretty again (#22420)
- Add Python 3.10 support (#22050)
- [FEATURE] add 1.22 1.23 K8S support (#21902)
- Remove pandas upper limit now that SQLA is 1.4+ (#22162)
- Patch
sql_alchemy_conn
if old postgres scheme used (#22333) - Protect against accidental misuse of XCom.get_value() (#22244)
- Order filenames for migrations (#22168)
- Don't try to auto generate migrations for Celery tables (#22120)
- Require SQLAlchemy 1.4 (#22114)
- bump sphinx-jinja (#22101)
- Add compat shim for SQLAlchemy to avoid warnings (#21959)
- Rename
xcom.dagrun_id
toxcom.dag_run_id
(#21806) - Deprecate non-JSON
conn.extra
(#21816) - Bump upper bound version of
jsonschema
to 5.0 (#21712) - Deprecate helper utility
days_ago
(#21653) - Remove
:type
lines nowsphinx-autoapi
supports type hints (#20951) - Silence deprecation warning in tests (#20900)
- Use
DagRun.run_id
instead ofexecution_date
when updating state of TIs (UI & REST API) (#18724) - Add Context stub to Airflow packages (#20817)
- Update Kubernetes library version (#18797)
- Rename
PodLauncher
toPodManager
(#20576) - Removes Python 3.6 support (#20467)
- Add deprecation warning for non-json-serializable params (#20174)
- Rename TaskMixin to DependencyMixin (#20297)
- Deprecate passing execution_date to XCom methods (#19825)
- Remove
get_readable_dags
andget_editable_dags
, andget_accessible_dags
. (#19961) - Remove postgres 9.6 support (#19987)
- Removed hardcoded connection types. Check if hook is instance of DbApiHook. (#19639)
- add kubernetes 1.21 support (#19557)
- Add FAB base class and set import_name explicitly. (#19667)
- Removes unused state transitions to handle auto-changing view permissions. (#19153)
- Chore: Use enum for
__var
and__type
members (#19303) - Use fab models (#19121)
- Consolidate method names between Airflow Security Manager and FAB default (#18726)
- Remove distutils usages for Python 3.10 (#19064)
- Removing redundant
max_tis_per_query
initialisation on SchedulerJob (#19020) - Remove deprecated usage of
init_role()
from API (#18820) - Remove duplicate code on dbapi hook (#18821)