Connect Azure Synapse Analytics with Azure Purview

by Jan 25, 2021

How do you integrate Azure Purview in Azure Synapse Analytics?

This article explains how to integrate Azure Purview into your Azure Synapse workspace for data discovery and exploration. Follow the steps below to connect your Azure Purview account in your Azure Synapse Workspace.

In the Management Hub you will see now a new option called Azure Purview.

Azure Purview Management HUb

Click on the option  “Connect to a Purview Account”. Please be aware that you need a Contributor role in your Azure Synapse workspace and access to your Azure Purview Account(Purview Reader or Purview Curator).

Find the Purview account you want to connect to from the drop down list or add it manually by adding the source ID.

Azure Puriew Connect Resource ID

If the connection is successful, you will see the following screen. If not, make sure you have the correct role to connect to your Azure Purview account.

Azure Purview Connected

 

Data discovery:

If you select your Data, Develop or Integrate HUB, you will see in the top center a Search bar.

Azure Purview Search

Using Azure Purview in Azure Synapse

To use Azure Purview in Azure Synapse it requires you to have access to the connected Azure Purview account. Azure Synapse will then passes-through the correct Azure Purview permissions.

Purview Reader role

  • Can read all content in Azure Purview

Purview Curator role

  • Can read all content in Azure Purview
  • Can edit assets, classification and glossary terms
  • Can apply classifications and glossary terms to assets.

Azure Purview actions

The following Azure Purview features are available in Azure Synapse Analytics(based on your role):

  • Overview of the metadata
  • View and edit schema of the metadata with classifications, glossary terms, data types, and descriptions
  • View lineage to understand dependencies and do impact analysis. For more information about, see lineage
  • View and edit Contacts to know who is an owner or expert over a dataset
  • Related to understand the hierarchical dependencies of a specific dataset. This experience is helpful to browse through data hierarchy.

Azure Purview Integration

Connect data to Azure Synapse

Add addition to above features, you can also connect directly to the assets you have searched.

Linked Service

  • Creating a new Linked Service will be required to copy data to Synapse or have them in your data hub (for supported data sources like ADLS Gen 2)

Integration Dataset

  • For objects like files, folders, or tables, you can directly create a new Integration Dataset and leverage an existing linked service if already created.

Develop in Azure Synapse

There are three actions that you can perform: New SQL ScriptNew Notebook, and New Data Flow.

SQL Script

  • View the top 100 rows in order to understand the shape of the data.
  • Create an external table from Synapse SQL database.
  • Load the data into a Synapse SQL database.

Notebook

  • Load data into a Spark DataFrame.
  • Create a Spark Table (if you do that over Parquet format, it also creates a serverless SQL pool table).

Data flow

  • Create an integration dataset that can be used as a source in a data flow pipeline.

Azure Purview Integration Develop

These new functionalities makes the integration between Azure Purview and Azure Synapse Analytics even more Powerful. More details can be found here.

Useful links

Create a Synapse workspace

Create an Azure Purview account

Thank you for reading, please feel free to ask questions and I’m more then happy to answer them.

Feel free to leave a comment

0 Comments

Submit a Comment

Your email address will not be published.

4 × 5 =

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Migrate Azure Storage to Azure Data Lake Gen2

Migrate Azure Storage to Storage Account with Azure Data Lake Gen2 capabilities Does it sometimes happen that you come across a Storage Account where the Hierarchical namespace is not enabled or that you still have a Storage Account V1? In the tutorial below I...

Change your Action Group in Azure Monitoring

Change a Action GroupPrevious Article In my previous artcile I wrote about how to create Service Helath Alerts. In this article you will learn how to change the Action Group to add, change or Remove members(Action Group Type Email/SMS/Push/Voice) Azure Portal In the...

Azure Data Factory updates June

Azure Data Factory updates There have been quite a few updates in Azure Data Factory and Azure Synapse Analytics in the last few days.Below is a summary of these updates:   Time-To-Live (TTL) on Integration Runtime with managed virtual network enabled The new TTL...

Create an Azure Synapse Analytics Apache Spark Pool

Adding a new Apache Spark Pool There are 2 options to create an Apache Spark Pool.Go to your Azure Synapse Analytics Workspace in de Azure Portal and add a new Apache Spark Pool. Or go to the Management Tab in your Azure Synapse Analytics Workspace and add a new...

Azure Data Factory: Generate Pipeline from the new Template Gallery

Last week I mentioned that we could save a Pipeline to GIT. But today I found out that you can also create a Pipeline from a predefined Solution Template.Template Gallery These template will make it easier to start with Azure Data Factory and it will reduce...

Exploring Azure Synapse Analytics Studio

Azure Synapse Workspace Settings In my previous article, I walked you through "how to create your Azure Synapse Analytics Workspace". It's now time to explore the brand new Synapse Studio. Most configuration and settings can be done through the Synapse Studio. In your...

Scale SQL Database dynamically with Metadata

Scale SQL Database Dynamically with Metadata Use this template to scale up and down an Azure SQL Database in Azure Synapse Analytics or in Azure Data Factory. This article describes a solution template how you can Scale up or down a SQL Database within Azure Synapse...

Azure SQL Data Warehouse: How to setup Reserved Capacity

Purchase your Azure SQL Datawarehouse Reservation   Since a few weeks you can buy Reserved Capacity for an Azure SQL Datawarehouse (SQLDW). This Reservation can save you up to 65% on the normal Pay as You go rates with a 3 year pre-commit. A pre-commit of 1 year...

Azure SQL Data Warehouse: Reserved Capacity versus Pay as You go

How do I use my Reserved Capacity correctly? Update 11-11-2020: This also applies to Azure Synapse SQL Pools. In my previous article you were introduced, how to create a Reserved Capacity for an Azure SQL Datawarehouse (SQLDW). Now it's time to take a look at how this...

Azure Synapse Analyics costs analyis for Integration Runtime

AutoResolveIntegrationRuntime! The last few days I've been following some discussions on Twitter on using a separate Integration Runtime in Azure Synapse Analytics running in the selected region instead of auto-resolve. The AutoResolveIntegrationRuntime is...