Azure Data Factory updates June
Azure Data Factory updates
There have been quite a few updates in Azure Data Factory and Azure Synapse Analytics in the last few days.
Below is a summary of these updates:
Time-To-Live (TTL) on Integration Runtime with managed virtual network enabled
The new TTL functionality, currently in Preview, will save you a lot of time. Instead of provision compute for every copy activity execution, with this new option the provisioned compute will be reserved. You can set this new option on every Integration Runtime with Managed Virtual Network enabled. Very handy and will advise you to do this direclty. This option is also available in Azure Synapse Analytics.
More can be found on the blog below:
CI/CD improvement using Global Parameters in Azure Data Factory
We used to have a PowerShell script to add the Global parameters to our CI/CD pipelines. Now we can include these as Override Parameters in our Deployment/Release Pipeline. Very handy and easy.
Deprecated:
New:
If you are using the arm-template-parameters-definition.json file to customize your parameters make sure you add the global parameters to this file. Otherwise you cannot override the parameters in the Deployment/Release Pipeline.
When we have created a Global Parameters Environment, we have now have the option to override this parameter option during the release:
-default_properties_Environment_value “Development“
More can be found on the blog below:
CI/CD improvement using Global Parameters in Azure Data Factory
Public Preview of the SAP CDC solution in Azure Data Factory
The new SAP ODP connector leverages SAP Operational Data Provisioning (ODP) framework, which is an established best practice for data integration within SAP landscapes. ODP provides access to a wide range of sources across all major SAP applications and comes with built-in CDC capabilities. In combination with the predefined data flow templates to process and update the changed records to any sink, this makes SAP data integration into Azure very much straight forward.
More can be found on the blog below:
Public Preview of the SAP CDC solution in Azure Data Factory
Azure Data Factory studio preview experience
You can now choose whether to enable preview experiences/functionality in your Azure Data Factory.
There are 2 ways how you can enable this preview experience:
When opening the portal you will see banner, just click on the Open settings to learn more and opt in. link
Or open the setting pane on the right top and enable the toggle Azure Data Factory Studio preview update.
Current Preview Updates
Currenlty the following options are available as preview features:
Dataflow data experimental view
Configuration panel
The configuration panel will only have Data Preview that will automatically refresh when changes are made to transformations.
Before:
After:
Transformation settings
A pop-up will now appear when you want to change your settings
Data preview
Data preview now includes Elapsed time (seconds) to show how long your data preview took to load.
Pipeline experimental view
Adding activities
You now have the option to add an activity using the Add button in the bottom right corner of an activity in the pipeline editor canvas. Clicking the button will open a drop-down list of all activities that you can add.
Iteration & conditionals container view
And the last one is really awesome, you now view the activities in the contained iteration and conditional activities like For each, Switch, Until and Switch
More can be found on the detailed page below:
Azure Data Factory studio preview experience
Please visit the blog below for all the latest update on Synapse Analytics in June 2022:
Azure Synapse Analytics June Update 2022
If you have any questions left, please leave them in the comments below.
0 Comments