ERWIN & BUSINESS ANALYTICS

ADF: Get Metadata Activity stopped working

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

 

 

ADF METADATA

Invalid Template

After debugging my Pipeline I found out the output.structure couldn’t be evaluated.

ADF METADATA Error

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Field List

I don’t know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

ADF METADATA Error field list

 

 

 

 

 

 

 

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

ADF METADATA Structure

 

 

 

 

 

 

 

 

After rerunning my Pipeline the error was gone.ADF METADATA working

 

 

 

 

 

 

 

 

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

 

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

twenty + twelve =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Speaking(Virtual) at SQL Saturday #963 Denmark

SQL Saturday 963 DenmarkPASS SQLSaturday is a free training event for professionals who use the Microsoft data platform. These community events offer content across data management, cloud and hybrid architecture, analytics, business intelligence, AI, and more.My first...

A new year with a new job

Changing jobsAfter almost 11 years and 4 months I have decided to leave Axians and to start a new adventure in the new year. On January 2, 2020 I will start my day as Lead Data and AI at InsparkThe past years have flown by. I started at Eniac BI, which was...

Yeah My website is Finally Live

Yeah, my first built website is LIVEThe last couple of weeks I've been working hard to design my website and finally the moment is there. My website live and I'm really proud.  Building my own website The last couple of weeks/months I've been working hard... to...

Data Factory Pricing

Data Factory pricingAre you also having problems to understand the Pricing Model for Azure Data Factory? After some research on the internet I came across an article which I wanted to share with you. ADFV2 Pricing ExamplesFeel free to leave a comment

Collection of all ADF Mapping Data Flow videos

ADF Mapping FlowDid you use the Dataflow preview functionality in Azure Data Factory? This has recently be renamed to Mapping Data Flows.  All video's which the ADF team has created, are collected. Start Here: ADF Data Flow: Overview ADF Data Flow: Data Flow...

Parameterize Linked Services in ADF

Parameterize Linked Services For my Azure Data Factory solution I wanted to Parameterize properties in my Linked Services. Not all properties are Parameterized by default through the UI. But there's another way to achieve this.    Linked Service Open your existing...

New Microsoft Azure Certifications

Microsoft Certification by Solution Area Handy overview of the new Microsoft Azure Certifications. More details can be found here Feel free to leave a comment

Azure Data Factory: Save a pipeline as a Template

Saving a PipelineAnother great improvement in Azure Data Factory. Saving you template!How can you save your template? First of all you need to connect your Azure Data Factory to  a GIT integration. Both Azure DevOps GIT and GitHub are supported. Please follow this...

Updated competency exams and certifications Data Platform and Data Analytics for 2020

Retiring and new exams and certifications as of June 30 2020 A lot of Exams and certifications for Data Platform and Data Analytics are retiring on June 30 2020. All retired Exams and certifications will remain eligible for competency attainment and renewal until June...