Data Saturday #1 Pordenone(Video)
Recording of my session during Data Saturday #1 Pordenone
Azure Purview Microsoft's answer to Data Governance and Data Lineage
Proud to be an ADF Hackathon winner!
Month: March 2021
Azure Data Factory Hackathon
Around the clock
During the event 'Around the clock Azure SQL and Azure Data Factory' event on Feb 3, 2021, they kicked-off the Azure Data Factory (ADF) Hackathon. Recording of this event can be found here.
Winner
I submitted a ADF Pipeline Template "Scale Dedicated SQL Pool Dynamically using Azure Data Factory control flow" and my submission was marked as WINNER. I am very proud that a simple template where you can easily save costs has won. See full post of the announcement Announcing the Azure Data Factory Hackathon winners! | LinkedIn
This template will help you can to scale up and down a Dedicated SQL Pool in Azure Synapse Analytics.
The pipelines is designed to Scale a SQL Pool within Azure Synapse Analytics. A SQL Pool(Former Azure SQL DW) linked to a SQL (Logical) Server has a slightly different approach(documentation can be found on Github).
Scaling a SQL Pool is actually a necessary functionality during your Data Movement Solutions, it will help to save and optimize your costs.
Documentation of this pipeline can be found on GitHub.
You can also use this template in Azure Synapse and the details can also be found on Github more details can also be found in this Article.
In case you have unanswered questions please do not hesitate to contact me.
Feel free to leave a comment
Scottisch Summit 2021(Video)
Month: March 2021
Recording of my session during Scottisch Summit 2021
Is there a way that we can build our Azure DataFactory all with parameters based on MetaData?
Azure Purview MSIgnite Spring 2021 Announcements
Month: March 2021
Azure Purview Ignite 2021 Annoucement
Pricing
This week the Azure Purview announced that they will extend the Azure Purview offer to provision 4 Capacity Units of the Data Map for free till May 31, 2021! Charging will start on June 1, 2021. Great news for customer who start exploring Azure Purview
More detail on Azure Purview pricing can be found here
New Sources
As you see above a lot of new Sources to scan have been announced and there will be more sources coming in the next months.
- AWS Simple Storage Service (S3) data
- SAP S/4 HANA
- SAP ECC
- Teradata
- Oracle
And last but least, Azure Multiple. With Azure Multiple you can now govern multiple Azure sources within Azure Purview by registering an entire Azure subscription or a resource group.
And you setup scan and triggers to the discover assets within your subscription.
More detail on how to setup Azure Multipe How to scan Azure multiple sources.
Also for Azure Multiple more resource types are coming in the upcoming weeks
Private Endpoint
You can now set up scans on your on-premises and Azure data sources with ingestion private endpoints and a self—hosted integration runtime to ensure network isolation, and metadata traversing only via private networks end-to-end and block all access to your Purview instance from public internet.
More details on pricing Pricing - Azure Purview
Azure Purview Documentation Documentation - Azure Purview
Azure Purview Q&A Q&A -Azure Purview
In case you have unanswered questions please do not hesitate to contact me.