Azure Synapse Analytics Code Repository has arrived

by Nov 25, 2020

Azure Synapse Analytics Code repository

‎I just opened my Azure Synapse Analytics Workspace and got a great surprise, the option Git Configuration is available as of today‎.

 

 

After a long wait, today the Git Configuration option became available in Azure Synapse Analytics.

The setup isn’t much different from Azure Data Factory, which can be found in this link.

The difference is that we no longer use an adf_publish branch but a workspace_publish branch. Which makes sense if you want to use both Azure Services side by side. In this blog I do quick walkthrough with the Azure Dev Ops Configuration enabled.

Azure Synapse GIT Config

Once we have configured everything, we can walk through the Git Configuration options within Azure Synapse Analytics. I’m sure there will be a lot of them, but below is a list of the ones I noticed first.

Synapse live

After you published your code, it will be available in Synapse Live, like in Azure Data Factory you develop everything in Azure DevOps branches.

Synapse Live

Notebooks

After creating a Notebook, we have the option Commit, after you have committed it will be directly saved within your current working branch.

Azure Synapse Notebooks  Azure Synapse Notebooks

SQL Scripts

Like Notebooks,  After creating a SQL Script, we can Commit, after you have committed it will be directly saved within your current working branch.

Azure Synapse SQL Script

Azure Synapse SQL Scripts

Pipelines

Also here we have now a Commit option.

Azure Synapse Pipelines

Workspace_publish

Beside the Notebooks and the SQL Scripts we can also store the Credentials and Spark Job definitions in Azure Dev Ops

Azure Synapse Publish

Differences

So as we can see the main differences between Azure Data Factory and Azure Synapse Analytics are:

Workspace_publish branch instead of adf_publish branch.

Commit instead of Save.

Azure Data Factory Pipelines

By moving your code from Azure Data Factory to Azure Synapse Analytics in Azure Dev Ops your Azure Data Factory Configuration will be available in Azure Synapse Analytics.

I added my ADF code to Azure Synapse in Azure Dev Ops and it looks the same.

Azure Synapse Dev Ops Example

After Refreshing the Azure Synapse Analytics Workspace, in the Data Hub we see the Integration Datasets(ADF DataSets) and the Linked Storage accounts.

Azure Synapse Data Example

And in the Integrate Hub, we see all our Pipelines. And the same is working for our triggers

Azure Synapse PipeLine Example

It looks like that we can reuse our code quite easily. I haven’t tested everything yet but I wanted to share this with you as quick as possible. I’m sure a easier way to migrate from Azure Data Factory to Azure Synapse will be on his way, you can use above as a start.

Integration Runtimes

Does everything work as in Azure Data Factory, NO at this moment you can’t use the Azure SSIS Integration Runtime and the shared Self Hosted Integration Runtime? But hopefully this will take not that long before it will arrive.

Thank you for reading, this was a quick overview of the first changes I discovered. Please feel free to leave comment if you have discovered more.

Do you want to become more familiar with the various possibilities of Azure Synapse Analytics, please read the following articles which I published a while ago:

✅ Creating your Azure Synapse Analytics Workspace

✅ Exploring the new Azure Synapse Analytics Studio

Creating an Apache Spark Pool

Creating a SQL Pool

Integration with Power BI

Feel free to leave a comment

1 Comment

  1. Devops Training In Hyderabad

    thanks for sharing such valuable information with us. it’s hard to come by well-informed people in this particular topic, but you explained this really good..

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

19 + twelve =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

ADF: Get Metadata Activity stopped working

Meta Data ActivityToday my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.    Invalid Template After debugging my Pipeline I found out the...

How to check your SQL Server Quota in Azure?

Azure Subscription Usages for SQL Server  Last week we reached our Logical server Quota in Azure. By default you're only allowed to add 20 Logical Servers, but we wanted to have some more for testing purposes.Microsoft Support You can submit a support ticket trough...

Working from Home: How do I get my energy and focus back on track?

Working from home It has now been exactly 7 weeks since I started working from home. In the beginning this went with full energy, but now gradually the energy and the focus on the work is starting to drain.I'm the only one in here? I'm sure, I'm not. I regularly hear...

Azure Data Factory: New functionalities and features

New functionalities and featuresLast week, a number of great new functionalities and features were added within Azure Data Factory. I would like to take you in some details in the blog below:Customer key With this new functionality you can add extra security to your...

Microsoft Build 2022 Book of News

Microsoft Build 2022MSBuild 2022 has just started. A lot of new Announcement will be released or demonstrated the upcoming days. Like every year or event most of these announcement are added to the Book of News.The book of news can be downloaded from the link below...

Restore a accidentally deleted Azure SQL Database

Help I deleted my Logical Server​OOPS Have you ever experienced that you accidentally deleted your Logical Server in Azure?  Because, for example, you made your Pipeline wrong. Surely. And of course you didn't have a backup in your storage either.Well I must confess...

Enable Pattern Rules in Azure Purview

How can I enable Pattern Rules?​Pattern Rules Last night I was preparing for a demo with Azure Purview. As always, I walk through all the activity hubs to see if there are any new options. This time I noticed that the Pattern Rules option was greyed out. Resource Set...

High Concurrency Notebook Activity in Microsoft Fabric

Microsoft Fabric is a cloud-based platform that provides integrated data engineering and data science solutions. One of the key features of Microsoft Fabric is the support for Apache Spark, a distributed computing framework that enables large-scale data processing and...

Calculate the Last Day of the Month using SQL

Calculate the Last DayToday I needed to calculate the last day of the previous month for a Customer.Ever heard from the function EOMONTH? Searching on the web I came across a function I never heard from before EOMONTH, this function can be used as of SQL Server 2012....

Proud to be an ADF Hackathon winner!

Azure Data Factory HackathonAround the clock During the event 'Around the clock Azure SQL and Azure Data Factory' event on Feb 3, 2021, they kicked-off the Azure Data Factory (ADF) Hackathon. Recording of this event can be found here.Winner I submitted a ADF Pipeline...