As data has been growing in size and demand for analyses has been increasing, the software engineering world had to focus on implementing well performing solutions that could serve the demand. This has lead to many different tools, unable to offer interoperability (or offering interoperability at a major performance cost) and effectively locking developers and companies into specific vendors. As the data engineering world has matured, proper solutions emerged that were focused on interoperability, breaking vendor barriers and empowering people with ownership and control of their data and their tools. We will see now the modern data engineering tools shifted from monolithic approaches to composable systems where developers are free to store, analyse and exchange data combining the tools that are best suited for each context, need and purpose and evolving their stack as their needs evolve.
Alessandro has been working in the Python Open Source field for 20 years and has been focusing on Data Engineering for the past 10 years. During the years he has been the core developer of the TurboGears2 web framework, a core developer of the Ming MongoDB ORM used at Sourceforge.net, and a contributor of the Apache Arrow and Substrait projects. Recently Alessandro has been Senior Director of Open Source Engineering at Voltron Data, where he led development efforts on Apache Arrow, ADBC, Substrait and Ibis and currently works at Posit where he has been focusing on improving support for Python software in Posit products. Alessandro is the author of the Crafting Test-Driven Software with Python and Modern Python Standard Library Cookbook and is currently working on the 5th edition of the Expert Python Programming book.