ERWIN & BUSINESS ANALYTICS

ADF: Get Metadata Activity stopped working

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

 

 

Invalid Template

After debugging my Pipeline I found out the output.structure couldn’t be evaluated.

ADF METADATA Error

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Field List

I don’t know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

ADF METADATA Error field list

 

 

 

 

 

 

 

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

ADF METADATA Structure

 

 

 

 

 

 

 

 

After rerunning my Pipeline the error was gone.ADF METADATA working

 

 

 

 

 

 

 

 

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

 

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

nine + 10 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Azure Data Factory: Save a pipeline as a Template

Saving a PipelineAnother great improvement in Azure Data Factory. Saving you template!How can you save your template? First of all you need to connect your Azure Data Factory to  a GIT integration. Both Azure DevOps GIT and GitHub are supported. Please follow this...

Collection of all ADF Mapping Data Flow videos

ADF Mapping FlowDid you use the Dataflow preview functionality in Azure Data Factory? This has recently be renamed to Mapping Data Flows.  All video's which the ADF team has created, are collected. Start Here: ADF Data Flow: Overview ADF Data Flow: Data Flow...

LEAP 2018

Lead Enterprise Architect ProgramFrom the 4th of December until the 6th of December I've joined the LEAP (Lead Enterprise Architect Program) 2018 Data/AI from Microsoft NL. Three  days on the Microsoft Campus in Redmond joining business and technical sessions from a...

Rerun Pipeline activities in Azure Data Factory

Rerun Pipeline activities in ADF! As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a...

Updated competency exams and certifications Data Platform and Data Analytics for 2020

Retiring and new exams and certifications as of June 30 2020 A lot of Exams and certifications for Data Platform and Data Analytics are retiring on June 30 2020. All retired Exams and certifications will remain eligible for competency attainment and renewal until June...

Calculate the Last Day of the Month using SQL

Calculate the Last DayToday I needed to calculate the last day of the previous month for a Customer.Ever heard from the function EOMONTH? Searching on the web I came across a function I never heard from before EOMONTH, this function can be used as of SQL Server 2012....

Data Factory Pricing

Data Factory pricingAre you also having problems to understand the Pricing Model for Azure Data Factory? After some research on the internet I came across an article which I wanted to share with you. ADFV2 Pricing ExamplesFeel free to leave a comment

New Microsoft Azure Certifications

Microsoft Certification by Solution Area Handy overview of the new Microsoft Azure Certifications. More details can be found here Feel free to leave a comment

Yeah My website is Finally Live

Yeah, my first built website is LIVEThe last couple of weeks I've been working hard to design my website and finally the moment is there. My website live and I'm really proud.  Building my own website The last couple of weeks/months I've been working hard... to...

My First Blog Post SQLSatHolland 790

Yeah, my first blog is LIVEAfter a good talk with Reza Rad from RADACAD during SQLSatHolland, I decided to start my first blog!Knowledge sharing is very important, it gives me a lot of energy. But it also gives others people in the community energies to pick up new...