ADF: Get Metadata Activity stopped working

by ErwinJul 1, 2019

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

Invalid Template

After debugging my Pipeline I found out the output.structure couldn’t be evaluated.

Field List

I don’t know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

After rerunning my Pipeline the error was gone.

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

0 Comments

Submit a Comment Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Azure Data Factory: Save a pipeline as a Template

Saving a PipelineAnother great improvement in Azure Data Factory. Saving you template!How can you save your template? First of all you need to connect your Azure Data Factory to a GIT integration. Both Azure DevOps GIT and GitHub are supported. Please follow this...

Microsoft Build 2022 Book of News

Microsoft Build 2022MSBuild 2022 has just started. A lot of new Announcement will be released or demonstrated the upcoming days. Like every year or event most of these announcement are added to the Book of News.The book of news can be downloaded from the link below...

Goodbye 2020 Hello 2021

Goodbye 2020 Started to work for InSpark Last year was certainly an eventful year. Started with a new job at InSpark and after 10 weeks we all know what happened, the first intelligent lockdown. The Netherlands was partially locked, but our office was immediately...

Custom comments in Azure Synapse Analytics

Add custom comments to your Azure DevOps and Github commitsFinally Finally and secretly hidden, we can now add a Comment to our commits in Azure Synapse Analytics and Azure Data Factory to Azure Dev Ops. How do you activate this custom comment option in your existing...

Rerun Pipeline activities in Azure Data Factory

Rerun Pipeline activities in ADF! As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a...

How to use concurrency in Azure Synapse pipelines?

How to prevent concurrent pipeline execution?Concurrency This week I had a discussion with a colleague about how we can now make sure that a Pipeline does not start when it's already started. He then indicated, have you ever thought of the concurrency option? I've...

Book of News Microsoft Ignite 2019

Microsoft Ignite 2019MSIgnite 2019 has almost come to an end. A lot of new Annoucement have been released and presented. Would you also like to know which announcements have been made?Microsoft has released during MSIgnite a "Book of News". This book contains all new...

Proud to be an ADF Hackathon winner!

Azure Data Factory HackathonAround the clock During the event 'Around the clock Azure SQL and Azure Data Factory' event on Feb 3, 2021, they kicked-off the Azure Data Factory (ADF) Hackathon. Recording of this event can be found here.Winner I submitted a ADF Pipeline...

Azure Synapse Analytics Code Repository has arrived

Azure Synapse Analytics Code repository‎I just opened my Azure Synapse Analytics Workspace and got a great surprise, the option Git Configuration is available as of today‎. After a long wait, today the Git Configuration option became available in Azure Synapse...

Microsoft Fabric pricing (Preview)

Microsoft Fabric PricingMicrosoft Fabric pricing in Public Preview is announced as of the 1 st of June.These are currently the Pay as You go pricing, later this year the Azure Reservation will follow. OneLake storage pricing is comparable to Azure ADLS (Azure Data...