Rerun Pipeline activities in Azure Data Factory

by Mar 7, 2019

Rerun Pipeline activities in ADF!

As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.
Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a lot of time if many different activities are created within one pipeline. Another nice step forward, I’m curious what else is coming in the next months.

 

Visualized

Besides that you can rerun your Pipeline in Azure Data Factory in a easy way, you also have the possibilities to see your run,  visualized in the Azure Data Factory Monitoring. This is a big improvement in my opinion.

Rerun a Pipeline

If you want to partially rerun a Pipeline, follow the steps below:
Select the Pipeline which has failed, go to the view activity runs and select the activity which failed.

Click on the Rerun Icon

 

 

 

 

You need to confirm that you want to rerun this activity.

The Pipeline will start and will first skip all the activities(the grey new icons in the upper right corner of each activity) in the Pipeline before your selected Activity.
Your Pipeline will now finalize all the activities from your newly defined starting point.

 

What else is new?

Monitor Rerun History

You can now view all the history reruns by clicking on the toggle to ‘View All Rerun History’.

By clicking on the red marked action, you can see all the History from an particular Pipeline run.

 

Thanks for reading.

 

 

Updated 10th of March:

 

Found a video on Channel9 which explains how to  “Rerun activities inside your Azure Data Factory pipelines”

https://channel9.msdn.com/Shows/Azure-Friday/Rerun-activities-inside-your-Azure-Data-Factory-pipelines?ocid=player

Feel free to leave a comment

1 Comment

  1. curious_dev
    curious_dev on April 27, 2020 at 9:43 am

    How can we automate this rerun activity process. If an activity fails and the pipeline is retriggered, it should start from the fail activity automatically.
    Looking for help on how to implement this using Python

    Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Restore a accidentally deleted Azure SQL Database

Help I deleted my Logical Server​OOPS Have you ever experienced that you accidentally deleted your Logical Server in Azure?  Because, for example, you made your Pipeline wrong. Surely. And of course you didn't have a backup in your storage either.Well I must confess...

Service Health Alerts in Azure

Maintenance and Service Healths in AzureI get often questions, is there no maintenance in Azure. Like every data center, the Azure data center also needs maintenance. You can even be notified when a specific service, in a specific data center, has scheduled...

Azure Data Factory: New functionalities and features

New functionalities and featuresLast week, a number of great new functionalities and features were added within Azure Data Factory. I would like to take you in some details in the blog below:Customer key With this new functionality you can add extra security to your...

Data Factory Pricing

Data Factory pricingAre you also having problems to understand the Pricing Model for Azure Data Factory? After some research on the internet I came across an article which I wanted to share with you. ADFV2 Pricing ExamplesFeel free to leave a comment

LEAP 2018

Lead Enterprise Architect ProgramFrom the 4th of December until the 6th of December I've joined the LEAP (Lead Enterprise Architect Program) 2018 Data/AI from Microsoft NL. Three  days on the Microsoft Campus in Redmond joining business and technical sessions from a...

Azure Synapse Analytics

Azure Synapse Analytics  Insights for all Azure Synapse provides a breathtaking view of your data across data warehouses and big data analytics systems. Bringing these two worlds together into a single service is challenging as it requires unifying similar...

Updated competency exams and certifications Data Platform and Data Analytics for 2020

Retiring and new exams and certifications as of June 30 2020 A lot of Exams and certifications for Data Platform and Data Analytics are retiring on June 30 2020. All retired Exams and certifications will remain eligible for competency attainment and renewal until June...

Yeah My website is Finally Live

Yeah, my first built website is LIVEThe last couple of weeks I've been working hard to design my website and finally the moment is there. My website live and I'm really proud.  Building my own website The last couple of weeks/months I've been working hard... to...

How to check your SQL Server Quota in Azure?

Azure Subscription Usages for SQL Server  Last week we reached our Logical server Quota in Azure. By default you're only allowed to add 20 Logical Servers, but we wanted to have some more for testing purposes.Microsoft Support You can submit a support ticket trough...

Azure Synapse Analytics Code Repository has arrived

Azure Synapse Analytics Code repository‎I just opened my Azure Synapse Analytics Workspace and got a great surprise, the option Git Configuration is available as of today‎.    After a long wait, today the Git Configuration option became available in Azure Synapse...

Discover more from Erwin | Data & Intelligence

Subscribe to get the latest posts sent to your email.