Rerun Pipeline activities in Azure Data Factory

Rerun Pipeline activities in ADF!

As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.
Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a lot of time if many different activities are created within one pipeline. Another nice step forward, I’m curious what else is coming in the next months.

 

Visualized

Besides that you can rerun your Pipeline in Azure Data Factory in a easy way, you also have the possibilities to see your run,  visualized in the Azure Data Factory Monitoring. This is a big improvement in my opinion.

Rerun a Pipeline

If you want to partially rerun a Pipeline, follow the steps below:
Select the Pipeline which has failed, go to the view activity runs and select the activity which failed.

Click on the Rerun Icon

 

 

 

 

You need to confirm that you want to rerun this activity.

The Pipeline will start and will first skip all the activities(the grey new icons in the upper right corner of each activity) in the Pipeline before your selected Activity.
Your Pipeline will now finalize all the activities from your newly defined starting point.

 

What else is new?

Monitor Rerun History

You can now view all the history reruns by clicking on the toggle to ‘View All Rerun History’.

By clicking on the red marked action, you can see all the History from an particular Pipeline run.

 

Thanks for reading.

 

 

Updated 10th of March:

 

Found a video on Channel9 which explains how to  “Rerun activities inside your Azure Data Factory pipelines”

https://channel9.msdn.com/Shows/Azure-Friday/Rerun-activities-inside-your-Azure-Data-Factory-pipelines?ocid=player

Feel free to leave a comment

1 Comment

  1. curious_dev

    How can we automate this rerun activity process. If an activity fails and the pipeline is retriggered, it should start from the fail activity automatically.
    Looking for help on how to implement this using Python

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

five × one =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Service Health Alerts in Azure

Maintenance and Service Healths in AzureI get often questions, is there no maintenance in Azure. Like every data center, the Azure data center also needs maintenance. You can even be notified when a specific service, in a specific data center, has scheduled...

New Microsoft Azure Certifications

Microsoft Certification by Solution Area Handy overview of the new Microsoft Azure Certifications. More details can be found here Feel free to leave a comment

Yeah My website is Finally Live

Yeah, my first built website is LIVEThe last couple of weeks I've been working hard to design my website and finally the moment is there. My website live and I'm really proud.  Building my own website The last couple of weeks/months I've been working hard... to...

Calculate the Last Day of the Month using SQL

Calculate the Last DayToday I needed to calculate the last day of the previous month for a Customer.Ever heard from the function EOMONTH? Searching on the web I came across a function I never heard from before EOMONTH, this function can be used as of SQL Server 2012....

Azure Data Factory: Save a pipeline as a Template

Saving a PipelineAnother great improvement in Azure Data Factory. Saving you template!How can you save your template? First of all you need to connect your Azure Data Factory to  a GIT integration. Both Azure DevOps GIT and GitHub are supported. Please follow this...

Control my Nest Thermostat on my Domoticz Server running on a Synology DiskStation

Control your Nest Thermostat  in DomoticzNormally I always write Azure related, but today I'm writing about something different, on how I can control my Nest Thermostat on my Domoticz Server (Home automation). A while ago I have upgraded my  Nest Thermostat to login...

Parameterize Linked Services in ADF

Parameterize Linked Services For my Azure Data Factory solution I wanted to Parameterize properties in my Linked Services. Not all properties are Parameterized by default through the UI. But there's another way to achieve this.    Linked Service Open your existing...

Are you using Azure DevOps?

Azure DevOpsMore and more users are starting with Azure DevOps. Azure DevOps can be used for up to 5 users free of charge and is therefore a great start to start a project.    Build and Release I will not describe what Azure DevOps can do, because that is too much to...

A new year with a new job

Changing jobsAfter almost 11 years and 4 months I have decided to leave Axians and to start a new adventure in the new year. On January 2, 2020 I will start my day as Lead Data and AI at InsparkThe past years have flown by. I started at Eniac BI, which was...

Collection of all ADF Mapping Data Flow videos

ADF Mapping FlowDid you use the Dataflow preview functionality in Azure Data Factory? This has recently be renamed to Mapping Data Flows.  All video's which the ADF team has created, are collected. Start Here: ADF Data Flow: Overview ADF Data Flow: Data Flow...