Data Community Austria

by Jan 26, 2024

Unleashing the Potential of Metadata-Driven ELT Framework in Synapse Analytics and Microsoft Fabric

Today, I had the opportunity to speak at the Data Community Austria Day 2024, a full day of sessions dedicated to data enthusiasts and professionals. The event was held at the Jufa Hotel in Wien and attracted more than 300 attendees.

My session was about ā€œUnleashing the Potential of Metadata-Driven ELT Framework in Synapse Analytics and Microsoft Fabric.ā€ I explained how metadata-driven pipelines can help you ingest and transform data with less code, reduced maintenance, and greater scalability than writing code or pipelines for every data source and destination. I also demonstrated how to leverage the medallion architecture (Landingzone, Bronze, Silver, Gold) and Delta tables to build a lakehouse in Microsoft Fabric, a new data platform that combines the best of data lakes and data warehouses.

The audience was very engaged and asked insightful questions. We had a lively discussion about the benefits and challenges of metadata-driven ELT frameworks, the features and capabilities of Synapse Analytics and Microsoft Fabric, and the best practices and tips for implementing them in real-world scenarios.

I want to thank the organizers and sponsors of Data Community Austria Day 2024 for inviting me to speak and for hosting such a wonderful event. I also want to thank all the attendees for their interest and participation. It was a pleasure to share my knowledge and experience with you and to learn from you as well.

 

My presentation and the code I used can be found on my Github, link below

Data Community Austria 2024

Feel free to leave a comment

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

18 + 5 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.