ADF: Get Metadata Activity stopped working

by ErwinJul 1, 2019

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

Invalid Template

After debugging my Pipeline I found out the output.structure couldn’t be evaluated.

Field List

I don’t know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

After rerunning my Pipeline the error was gone.

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

0 Comments

Azure Synapse Analytics

Azure Synapse Analytics Insights for all Azure Synapse provides a breathtaking view of your data across data warehouses and big data analytics systems. Bringing these two worlds together into a single service is challenging as it requires unifying similar...

Calculate the Last Day of the Month using SQL

Calculate the Last DayToday I needed to calculate the last day of the previous month for a Customer.Ever heard from the function EOMONTH? Searching on the web I came across a function I never heard from before EOMONTH, this function can be used as of SQL Server 2012....

My Sessions at DataSaturdayNL 2019

DataSaturday NL 2019Date: 5 th October Location: Pathé Leidsche Rijn Data Saturday Holland is the new name for SQL Saturday Holland. A new start a new location and wow what a top location, Pathé Leidsche Rijn. A cinema with 7 screens and ranging from 180 to 668 seats....

Microsoft Build 2022 Book of News

Microsoft Build 2022MSBuild 2022 has just started. A lot of new Announcement will be released or demonstrated the upcoming days. Like every year or event most of these announcement are added to the Book of News.The book of news can be downloaded from the link below...

Goodbye 2022, Hello 2023

Goodbye 2022Recap It's that time of year again to reflect on the past year. Also think it's really good, to see what you've done in the past year. It is also the time again to traditionally bake Oliebollen on this day, a Dutch Tradition that we do on New Year's Eve....

Azure Purview Public Preview Starts billing

Billing for Azure Purview(Public Preview)As of January 20th 2021 0:00 UTC Azure Purview will starts billing.Preview From January 20 ,2021 Azure Purview will start billing. During the Public Preview, you will only be billed if you exceed the 4 capacity units for Azure...

Are you using Azure DevOps?

Azure DevOpsMore and more users are starting with Azure DevOps. Azure DevOps can be used for up to 5 users free of charge and is therefore a great start to start a project. Build and Release I will not describe what Azure DevOps can do, because that is too much to...

How to enable Microsoft Fabric

Microsoft FabricI got some questions from customers that didn’t know how to enable Microsoft Fabric and that they only see Power BI Items and not the new announced Experiences. In this short blog I will explain how you can easily enable Microsoft Fabric.How to enable...

Microsoft Fabric pricing (Preview)

Microsoft Fabric PricingMicrosoft Fabric pricing in Public Preview is announced as of the 1 st of June.These are currently the Pay as You go pricing, later this year the Azure Reservation will follow. OneLake storage pricing is comparable to Azure ADLS (Azure Data...

Rerun Pipeline activities in Azure Data Factory

Rerun Pipeline activities in ADF! As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a...

Discover more from Erwin | Data & Intelligence

Subscribe to get the latest posts sent to your email.