AWS announced that Aurora can now query data stored in S3-based data lakes without copying it into the database first. The feature uses Apache Iceberg table formats and lets transactional applications pull from analytical stores in near real-time. For data teams, the promise is simpler: fewer overnight batch jobs, fewer reconciliation headaches, and less infrastructure to maintain.
We have watched companies in the DACH region spend months building custom ETL pipelines to keep their production databases and analytics layers in sync. A Frankfurt insurance company we work with has three full-time engineers dedicated to pipeline maintenance alone. When something breaks at 2am, the on-call rotation is brutal. Tooling that reduces that overhead frees those engineers for work that actually moves the business forward.
The hiring implication follows directly. Companies adopting zero-ETL patterns will still need data engineers, but the role shifts. Configuration and pipeline scripting become less central. Architectural thinking rises in importance: deciding which data stays where, how access patterns should be optimised, and how to govern queries that span transactional and analytical domains. More job specifications now list Iceberg, Delta Lake, or Hudi as required skills alongside traditional database experience.
For candidates, the shift is gradual but worth tracking. Engineers who have spent careers building batch pipelines may find those skills less differentiated over the coming years. The premium is moving toward people who can design systems that span data stores coherently, with clean governance and observability built in. Companies hiring today are already asking for that profile, even if they phrase it differently in the job description.
Prompted by reporting from BigDATAwire.