Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
0
Release date:
September 15, 2022
Previous version:
2.4.0rc1
(released September 15, 2022)
Magnitude:
0
Diff Delta
Contributors:
0 total committers
Commits:
0 Commits in this Release
Could not find any commits associated with this release.
Browse Other Releases
Latest Pending
Unreleased π
2.4.3rc1
Released November 11, 2022
1,366 Ξ
2.4.2
Released October 19, 2022
0 Ξ
2.4.2rc1
Released October 19, 2022
2,362 Ξ
2.4.1
Released September 27, 2022
0 Ξ
2.4.1rc1
Released September 27, 2022
1,444 Ξ
2.4.0
Released September 15, 2022
0 Ξ
2.4.0rc1
Released September 15, 2022
1,254 Ξ
2.4.0b1
Released September 8, 2022
128,567 Ξ
2.3.4
Released August 19, 2022
0 Ξ
2.3.4rc1
Released August 19, 2022
6,437 Ξ
Top Contributors in 2.4.0
Could not determine top contributors for this release.
Directory Browser for 2.4.0
We haven't yet finished calculating and confirming the files and directories changed in this release. Please check back soon.
Release Notes Published
New Features
- Add Data-aware Scheduling (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-48+milestone%3A%22Airflow+2.4.0%22)
- Add
@task.short_circuit
TaskFlow decorator (#25752) - Make
execution_date_or_run_id
optional intasks test
command (#26114) - Automatically register DAGs that are used in a context manager (#23592, #26398)
- Add option of sending DAG parser logs to stdout. (#25754)
- Support multiple
DagProcessors
parsing files from different locations. (#25935) - Implement
ExternalPythonOperator
(#25780) - Make execution_date optional for command
dags test
(#26111) - Implement
expand_kwargs()
against a literal list (#25925) - Add trigger rule tooltip (#26043)
- Add conf parameter to CLI for airflow dags test (#25900)
- Include scheduled slots in pools view (#26006)
- Add
output
property toMappedOperator
(#25604) - Add roles delete command to cli (#25854)
- Add Airflow specific warning classes (#25799)
- Add support for
TaskGroup
inExternalTaskSensor
(#24902) - Add
@task.kubernetes
taskflow decorator (#25663) - Add a way to import Airflow without side-effects (#25832)
- Let timetables control generated run_ids. (#25795)
- Allow per-timetable ordering override in grid view (#25633)
- Grid logs for mapped instances (#25610, #25621, #25611)
- Consolidate to one
schedule
param (#25410) - DAG regex flag in backfill command (#23870)
- Adding support for owner links in the Dags view UI (#25280)
- Ability to clear a specific DAG Run's task instances via REST API (#23516)
- Possibility to document DAG with a separate markdown file (#25509)
- Add parsing context to DAG Parsing (#25161)
- Implement
CronTriggerTimetable
(#23662) - Add option to mask sensitive data in UI configuration page (#25346)
- Create new databases from the ORM (#24156)
- Implement
XComArg.zip(*xcom_args)
(#25176) - Introduce
sla_miss
metric (#23402) - Implement
map()
semantic (#25085) - Add override method to TaskGroupDecorator (#25160)
- Implement
expand_kwargs()
(#24989) - Add parameter to turn off SQL query logging (#24570)
- Add
DagWarning
model, and a check for missing pools (#23317) - Add Task Logs to Grid details panel (#24249)
- Added small health check server and endpoint in scheduler(#23905)
- Add built-in External Link for
ExternalTaskMarker
operator (#23964) - Add default task retry delay config (#23861)
- Add clear DagRun endpoint. (#23451)
- Add support for timezone as string in cron interval timetable (#23279)
- Add auto-refresh to dags home page (#22900, #24770)
Improvements
- Add more weekday operator and sensor examples #26071 (#26098)
- Add subdir parameter to dags reserialize command (#26170)
- Update zombie message to be more descriptive (#26141)
- Only send an
SlaCallbackRequest
if the DAG is scheduled (#26089) - Promote
Operator.output
more (#25617) - Upgrade API files to typescript (#25098)
- Less
hacky
double-rendering prevention in mapped task (#25924) - Improve Audit log (#25856)
- Remove mapped operator validation code (#25870)
- More
DAG(schedule=...)
improvements (#25648) - Reduce
operator_name
dupe in serialized JSON (#25819) - Make grid view group/mapped summary UI more consistent (#25723)
- Remove useless statement in
task_group_to_grid
(#25654) - Add optional data interval to
CronTriggerTimetable
(#25503) - Remove unused code in
/grid
endpoint (#25481) - Add and document description fields (#25370)
- Improve Airflow logging for operator Jinja template processing (#25452)
- Update core example DAGs to use
@task.branch
decorator (#25242) - Update DAG
audit_log
route (#25415) - Change stdout and stderr access mode to append in commands (#25253)
- Remove
getTasks
from Grid view (#25359) - Improve taskflow type hints with ParamSpec (#25173)
- Use tables in grid details panes (#25258)
- Explicitly list
@dag
arguments (#25044) - More typing in
SchedulerJob
andTaskInstance
(#24912) - Patch
getfqdn
with more resilient version (#24981) - Replace all
NBSP
characters bywhitespaces
(#24797) - Re-serialize all DAGs on
airflow db upgrade
(#24518) - Rework contract of try_adopt_task_instances method (#23188)
- Make
expand()
error vague so it's not misleading (#24018) - Add enum validation for
[webserver]analytics_tool
(#24032) - Add
dttm
searchable field in audit log (#23794) - Allow more parameters to be piped through via
execute_in_subprocess
(#23286) - Use
func.count
to count rows (#23657) - Remove stale serialized dags (#22917)
- AIP45 Remove dag parsing in airflow run local (#21877)
- Add support for queued state in DagRun update endpoint. (#23481)
- Add fields to dagrun endpoint (#23440)
- Use
sql_alchemy_conn
for celery result backend whenresult_backend
is not set (#24496)
Bug Fixes
- Have consistent types between the ORM and the migration files (#24044, #25869)
- Disallow any dag tags longer than 100 char (#25196)
- Add the dag_id to
AirflowDagCycleException
message (#26204) - Properly build URL to retrieve logs independently from system (#26337)
- For worker log servers only bind to IPV6 when dual stack is available (#26222)
- Fix
TaskInstance.task
not defined beforehandle_failure
(#26040) - Undo secrets backend config caching (#26223)
- Fix faulty executor config serialization logic (#26191)
- Show
DAGs
andDatasets
menu links based on role permission (#26183) - Allow setting
TaskGroup
tooltip via function docstring (#26028) - Fix RecursionError on graph view of a DAG with many tasks (#26175)
- Fix backfill occasional deadlocking (#26161)
- Fix
DagRun.start_date
not set during backfill with--reset-dagruns
True (#26135) - Use label instead of id for dynamic task labels in graph (#26108)
- Don't fail DagRun when leaf
mapped_task
is SKIPPED (#25995) - Add group prefix to decorated mapped task (#26081)
- Fix UI flash when triggering with dup logical date (#26094)
- Fix Make items nullable for
TaskInstance
related endpoints to avoid API errors (#26076) - Fix
BranchDateTimeOperator
to betimezone-awreness-insensitive
(#25944) - Fix legacy timetable schedule interval params (#25999)
- Fix response schema for
list-mapped-task-instance
(#25965) - Properly check the existence of missing mapped TIs (#25788)
- Fix broken auto-refresh on grid view (#25950)
- Use per-timetable ordering in grid UI (#25880)
- Rewrite recursion when parsing DAG into iteration (#25898)
- Find cross-group tasks in
iter_mapped_dependants
(#25793) - Fail task if mapping upstream fails (#25757)
- Support
/
in variable get endpoint (#25774) - Use cfg default_wrap value for grid logs (#25731)
- Add origin request args when triggering a run (#25729)
- Operator name separate from class (#22834)
- Fix incorrect data interval alignment due to assumption on input time alignment (#22658)
- Return None if an
XComArg
fails to resolve (#25661) - Correct
json
arg help inairflow variables set
command (#25726) - Added MySQL index hint to use
ti_state
onfind_zombies
query (#25725) - Only excluded actually expanded fields from render (#25599)
- Grid, fix toast for
axios
errors (#25703) - Fix UI redirect (#26409)
- Require dag_id arg for dags list-runs (#26357)
- Check for queued states for dags auto-refresh (#25695)
- Fix upgrade code for the
dag_owner_attributes
table (#25579) - Add map index to task logs api (#25568)
- Ensure that zombie tasks for dags with errors get cleaned up (#25550)
- Make extra link work in UI (#25500)
- Sync up plugin API schema and definition (#25524)
- First/last names can be empty (#25476)
- Refactor DAG pages to be consistent (#25402)
- Check
expand_kwargs()
input type before unmapping (#25355) - Filter XCOM by key when calculating map lengths (#24530)
- Fix
ExternalTaskSensor
not working with dynamic task (#25215) - Added exception catching to send default email if template file raises any exception (#24943)
- Bring
MappedOperator
members in sync withBaseOperator
(#24034)
Misc/Internal
- Add automatically generated
ERD
schema for theMetaData
DB (#26217) - Mark serialization functions as internal (#26193)
- Remove remaining deprecated classes and replace them with
PEP562
(#26167) - Move
dag_edges
andtask_group_to_dict
to corresponding util modules (#26212) - Lazily import many modules to improve import speed (#24486, #26239)
- FIX Incorrect typing information (#26077)
- Add missing contrib classes to deprecated dictionaries (#26179)
- Re-configure/connect the
ORM
after forking to run a DAG processor (#26216) - Remove cattrs from lineage processing. (#26134)
- Removed deprecated contrib files and replace them with
PEP-562
getattr (#26153) - Make
BaseSerialization.serialize
"public" to other classes. (#26142) - Change the template to use human readable task_instance description (#25960)
- Bump
moment-timezone
from0.5.34
to0.5.35
in/airflow/www
(#26080) - Fix Flask deprecation warning (#25753)
- Add
CamelCase
to generated operations types (#25887) - Fix migration issues and tighten the CI upgrade/downgrade test (#25869)
- Fix type annotations in
SkipMixin
(#25864) - Workaround setuptools editable packages path issue (#25848)
- Bump
undici
from5.8.0 to 5.9.1
in /airflow/www (#25801) - Add custom_operator_name attr to
_BranchPythonDecoratedOperator
(#25783) - Clarify
filename_template
deprecation message (#25749) - Use
ParamSpec
to replace...
in Callable (#25658) - Remove deprecated modules (#25543)
- Documentation on task mapping additions (#24489)
- Remove Smart Sensors (#25507)
- Fix
elasticsearch
test config to avoid warning on deprecated template (#25520) - Bump
terser
from4.8.0 to 4.8.1
in /airflow/ui (#25178) - Generate
typescript
types from restAPI
docs (#25123) - Upgrade utils files to
typescript
(#25089) - Upgrade remaining context file to
typescript
. (#25096) - Migrate files to
ts
(#25267) - Upgrade grid Table component to
ts.
(#25074) - Skip mapping against mapped
ti
if it returns None (#25047) - Refactor
js
file structure (#25003) - Move mapped kwargs introspection to separate type (#24971)
- Only assert stuff for mypy when type checking (#24937)
- Bump
moment
from2.29.3 to 2.29.4
in/airflow/www
(#24885) - Remove "bad characters" from our codebase (#24841)
- Remove
xcom_push
flag fromBashOperator
(#24824) - Move Flask hook registration to end of file (#24776)
- Upgrade more javascript files to
typescript
(#24715) - Clean up task decorator type hints and docstrings (#24667)
- Preserve original order of providers' connection extra fields in UI (#24425)
- Rename
charts.css
tochart.css
(#24531) - Rename
grid.css
tochart.css
(#24529) - Misc: create new process group by
set_new_process_group
utility (#24371) - Airflow UI fix Prototype Pollution (#24201)
- Bump
moto
version (#24222) - Remove unused
[github_enterprise]
from ref docs (#24033) - Clean up
f-strings
in logging calls (#23597) - Add limit for
JPype1
(#23847) - Simply json responses (#25518)
- Add min attrs version (#26408)
Doc only changes
- Add url prefix setting for
Celery
Flower (#25986) - Updating deprecated configuration in examples (#26037)
- Fix wrong link for taskflow tutorial (#26007)
- Reorganize tutorials into a section (#25890)
- Fix concept doc for dynamic task map (#26002)
- Update code examples from "classic" operators to taskflow (#25845, #25657)
- Add instructions on manually fixing
MySQL
Charset problems (#25938) - Prefer the local Quick Start in docs (#25888)
- Fix broken link to
Trigger Rules
(#25840) - Improve docker documentation (#25735)
- Correctly link to Dag parsing context in docs (#25722)
- Add note on
task_instance_mutation_hook
usage (#25607) - Note that TaskFlow API automatically passes data between tasks (#25577)
- Update DAG run to clarify when a DAG actually runs (#25290)
- Update tutorial docs to include a definition of operators (#25012)
- Rewrite the Airflow documentation home page (#24795)
- Fix
task-generated mapping
example (#23424) - Add note on subtle logical date change in
2.2.0
(#24413) - Add missing import in best-practices code example (#25391)