Azure Synapse Analytics Code Repository has arrived

by Nov 25, 2020

Azure Synapse Analytics Code repository

‎I just opened my Azure Synapse Analytics Workspace and got a great surprise, the option Git Configuration is available as of today‎.

 

 

After a long wait, today the Git Configuration option became available in Azure Synapse Analytics.

The setup isn’t much different from Azure Data Factory, which can be found in this link.

The difference is that we no longer use an adf_publish branch but a workspace_publish branch. Which makes sense if you want to use both Azure Services side by side. In this blog I do quick walkthrough with the Azure Dev Ops Configuration enabled.

Azure Synapse GIT Config

Once we have configured everything, we can walk through the Git Configuration options within Azure Synapse Analytics. I’m sure there will be a lot of them, but below is a list of the ones I noticed first.

Synapse live

After you published your code, it will be available in Synapse Live, like in Azure Data Factory you develop everything in Azure DevOps branches.

Synapse Live

Notebooks

After creating a Notebook, we have the option Commit, after you have committed it will be directly saved within your current working branch.

Azure Synapse Notebooks  Azure Synapse Notebooks

SQL Scripts

Like Notebooks,  After creating a SQL Script, we can Commit, after you have committed it will be directly saved within your current working branch.

Azure Synapse SQL Script

Azure Synapse SQL Scripts

Pipelines

Also here we have now a Commit option.

Azure Synapse Pipelines

Workspace_publish

Beside the Notebooks and the SQL Scripts we can also store the Credentials and Spark Job definitions in Azure Dev Ops

Azure Synapse Publish

Differences

So as we can see the main differences between Azure Data Factory and Azure Synapse Analytics are:

Workspace_publish branch instead of adf_publish branch.

Commit instead of Save.

Azure Data Factory Pipelines

By moving your code from Azure Data Factory to Azure Synapse Analytics in Azure Dev Ops your Azure Data Factory Configuration will be available in Azure Synapse Analytics.

I added my ADF code to Azure Synapse in Azure Dev Ops and it looks the same.

Azure Synapse Dev Ops Example

After Refreshing the Azure Synapse Analytics Workspace, in the Data Hub we see the Integration Datasets(ADF DataSets) and the Linked Storage accounts.

Azure Synapse Data Example

And in the Integrate Hub, we see all our Pipelines. And the same is working for our triggers

Azure Synapse PipeLine Example

It looks like that we can reuse our code quite easily. I haven’t tested everything yet but I wanted to share this with you as quick as possible. I’m sure a easier way to migrate from Azure Data Factory to Azure Synapse will be on his way, you can use above as a start.

Integration Runtimes

Does everything work as in Azure Data Factory, NO at this moment you can’t use the Azure SSIS Integration Runtime and the shared Self Hosted Integration Runtime? But hopefully this will take not that long before it will arrive.

Thank you for reading, this was a quick overview of the first changes I discovered. Please feel free to leave comment if you have discovered more.

Do you want to become more familiar with the various possibilities of Azure Synapse Analytics, please read the following articles which I published a while ago:

✅ Creating your Azure Synapse Analytics Workspace

✅ Exploring the new Azure Synapse Analytics Studio

Creating an Apache Spark Pool

Creating a SQL Pool

Integration with Power BI

Feel free to leave a comment

1 Comment

  1. Devops Training In Hyderabad

    thanks for sharing such valuable information with us. it’s hard to come by well-informed people in this particular topic, but you explained this really good..

    Reply

Submit a Comment

Your email address will not be published.

1 × 5 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Azure Data Factory: Save a pipeline as a Template

Saving a PipelineAnother great improvement in Azure Data Factory. Saving you template!How can you save your template? First of all you need to connect your Azure Data Factory to  a GIT integration. Both Azure DevOps GIT and GitHub are supported. Please follow this...

Goodbye 2021, Hello 2022

Goodbye 2021Recap First of all, I would like to wish everyone a very beautiful and healthy 2022. We are now 3 days on the road into the new year and it is always good to look back at what happened last year. It's certainly been an eventful year, topped off with my MVP...

Service Health Alerts in Azure

Maintenance and Service Healths in AzureI get often questions, is there no maintenance in Azure. Like every data center, the Azure data center also needs maintenance. You can even be notified when a specific service, in a specific data center, has scheduled...

10 Days of Azure Synapse Analytics

10 Days of Azure Synapse AnalyticsFor the next 10 days, every day a different topic is explained about Azure Synapse Analytics. The shortest and easiest way to see how Azure Synapse Analytics can help you, to make decisions within your data landscape.Day 1...

Azure Synapse Analytics

Azure Synapse Analytics  Insights for all Azure Synapse provides a breathtaking view of your data across data warehouses and big data analytics systems. Bringing these two worlds together into a single service is challenging as it requires unifying similar...

Rerun Pipeline activities in Azure Data Factory

Rerun Pipeline activities in ADF! As of today you can rerun or partially, yes you’re reading it correct partially, rerun you Azure Data Factory pipeline.Where you previously had to run the entire Pipeline again, you can now run a part of the Pipeline. This can save a...

A new year with a new job

Changing jobsAfter almost 11 years and 4 months I have decided to leave Axians and to start a new adventure in the new year. On January 2, 2020 I will start my day as Lead Data and AI at InsparkThe past years have flown by. I started at Eniac BI, which was...

My First Blog Post SQLSatHolland 790

Yeah, my first blog is LIVEAfter a good talk with Reza Rad from RADACAD during SQLSatHolland, I decided to start my first blog!Knowledge sharing is very important, it gives me a lot of energy. But it also gives others people in the community energies to pick up new...

My sessions at Pass Data Community Summit

A hybrid conference in Seattle and online This year's PASS Data Community Summit is more than a conference – it's a homecoming. Reconnect with old friends, build new relationships, gain new skills, and get the world-class training you need to take that next step in...

Book of News Microsoft Ignite 2019

Microsoft Ignite 2019MSIgnite 2019 has almost come to an end. A lot of new Annoucement have been released and presented.  Would you also like to know which announcements have been made?Microsoft has released during MSIgnite a "Book of News". This book contains all new...