Upgrading from Apache Airflow 2.10.3 to version 3.0.6 over the holidays gave one developer a chance to explore the latest updates and improvements. The transition highlighted several key changes that make airflow more streamlined, scalable, and developer-friendly. This guide shares the main takeaways from that experience, focusing on the most impactful updates and how they change workflow management.

Simplified Imports and Improved Developer Experience

One of the first noticeable improvements is the shift to SDK-first imports. In Airflow 2.x, creating workflows required importing various objects from multiple modules, which could be confusing and repetitive. Airflow 3 consolidates these into a more unified SDK, making DAG creation more intuitive. Instead of importing decorators and models separately, developers now import from a single SDK surface, reducing cognitive load and increasing consistency across projects.

This change may seem small but pays off in larger codebases. It simplifies the process of writing and maintaining DAGs, especially for new team members. Overall, the new import structure helps developers focus more on building workflows rather than managing import complexity.

Decoupling DAG Code from Metadata Database

Another major architectural shift is the separation of DAG and task code from the metadata database. In older versions, the code was more tightly linked to the database, which sometimes led to accidental dependencies and less flexibility. Airflow 3 intentionally decouples these components, promoting a cleaner separation of concerns.

This change results in fewer accidental dependencies on database objects, making the system more robust and easier to scale. It also creates clearer boundaries between workflow orchestration and execution, which benefits both development and operations teams. For those who prefer lightweight and modular architectures, this update represents a solid move forward.

In practice, this means the platform becomes safer to operate at scale, with API-driven interactions that improve reliability and future-proof the system.

Enhanced DAG Versioning for Better Traceability

One of the standout features in Airflow 3 is DAG versioning. Each DAG run is now tied to a specific version of the DAG, which greatly improves the accuracy of historical data. In Airflow 2.x, even minor changes like renaming tasks could make past runs look inconsistent or confusing in the user interface. Debugging older runs was often a mental exercise, trying to match current code to previous executions.

With DAG versioning, each run executes against the exact DAG definition it started with. This means historical runs are more trustworthy and easier to interpret. It also simplifies debugging by removing the guesswork of whether code has changed since the run. Overall, versioning makes workflow evolution safer and more transparent, boosting confidence in managing complex pipelines.

This change is especially valuable for teams that need to audit or trace past executions without ambiguity, making Airflow a more reliable tool for production environments.

New UI: Modern Look with Some Rough Edges

The redesigned user interface in Airflow 3 offers a more modern look, but the experience can vary. Some users and clients found the new layout disorienting, mainly because buttons and workflow views moved around or changed location. While the new UI aims to be more user-friendly and visually appealing, it still feels a bit rough around the edges.

For some, the transition requires a learning curve as familiar workflows and navigation paths shift. Different deployment setups and versions may also affect how smooth the UI feels. Despite these challenges, the new interface is a step toward a more polished experience, but it might still need some refinement before feeling fully intuitive.

Overall, the UI updates are promising, and future iterations are likely to smooth out current issues. For now, users should expect some adjustment time but can look forward to a more modern interface in the near future.

In summary, migrating to Airflow 3 brings meaningful improvements in developer experience, system architecture, and workflow management. While some areas like the UI are still evolving, the core changes set a strong foundation for more scalable and reliable workflow orchestration. Teams considering the upgrade should prepare for some adjustments but can benefit from the long-term advantages these updates offer.

Inspired by

• https://www.infoworld.com/article/4145050/migrating-from-apache-airflow-v2-to-v3.html