ADF: Get Metadata Activity stopped working

by ErwinJul 1, 2019

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

Invalid Template

After debugging my Pipeline I found out the output.structure couldn’t be evaluated.

Field List

I don’t know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

After rerunning my Pipeline the error was gone.

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

0 Comments

Parameterize Linked Services in ADF

Parameterize Linked Services For my Azure Data Factory solution I wanted to Parameterize properties in my Linked Services. Not all properties are Parameterized by default through the UI. But there's another way to achieve this. Linked Service Open your existing...

Restore a accidentally deleted Azure SQL Database

Help I deleted my Logical ServerOOPS Have you ever experienced that you accidentally deleted your Logical Server in Azure? Because, for example, you made your Pipeline wrong. Surely. And of course you didn't have a backup in your storage either.Well I must confess...

New Microsoft Azure Certifications

Microsoft Certification by Solution Area Handy overview of the new Microsoft Azure Certifications. More details can be found here Feel free to leave a comment

Calculate Workingdays including Holidays with T-SQL

Calculate Workingdays between 2 Date columnsRecently I have been getting some questions from my customers, can I calculate the number of workdays between 2 dates? Of course my answer was, yes you can. But I do want certain closing dates and holidays of our company not...

Calculate the Last Day of the Month using SQL

Calculate the Last DayToday I needed to calculate the last day of the previous month for a Customer.Ever heard from the function EOMONTH? Searching on the web I came across a function I never heard from before EOMONTH, this function can be used as of SQL Server 2012....

Azure Synapse Analytics

Azure Synapse Analytics Insights for all Azure Synapse provides a breathtaking view of your data across data warehouses and big data analytics systems. Bringing these two worlds together into a single service is challenging as it requires unifying similar...

SQLBits session: Microsoft Purview Data Policy App

SQLBits 2023 Thanks everyone for visiting my session during SQLBits. It's great to see such a full room and that so many people have started using Microsoft Purview. SLIDES The slides can be downloaded via the link below, so that you can view them again at...

Custom comments in Azure Synapse Analytics

Add custom comments to your Azure DevOps and Github commitsFinally Finally and secretly hidden, we can now add a Comment to our commits in Azure Synapse Analytics and Azure Data Factory to Azure Dev Ops. How do you activate this custom comment option in your existing...

Azure Purview March Updates

Azure Purview updatesAnnouncements Last week during SQLBITS, quite a few new updates were announced. I would like to include you in these announcements.March updates Support for SAP Business Warehouse (Preview) Blogpost:...

10 Days of Azure Synapse Analytics

10 Days of Azure Synapse AnalyticsFor the next 10 days, every day a different topic is explained about Azure Synapse Analytics. The shortest and easiest way to see how Azure Synapse Analytics can help you, to make decisions within your data landscape.Day 1...

Discover more from Erwin | Data & Intelligence

Subscribe to get the latest posts sent to your email.