Proud to be an ADF Hackathon winner!

Proud to be an ADF Hackathon winner!

by | Mar 15, 2021 | Azure, Azure Data Factory

Synapse

by Erwin | Mar 15, 2021

Azure Data Factory Hackathon

Around the clock

During the event 'Around the clock Azure SQL and Azure Data Factory' event on Feb 3, 2021, they kicked-off the Azure Data Factory (ADF) Hackathon. Recording of this event can be found here.

Winner

I submitted a ADF Pipeline Template "Scale Dedicated SQL Pool Dynamically using Azure Data Factory control flow" and my submission was marked as WINNER. I am very proud that a simple template where you can easily save costs has won. See full post of the announcement Announcing the Azure Data Factory Hackathon winners! | LinkedIn

This template will help you can to scale up and down a Dedicated SQL Pool in Azure Synapse Analytics.

The pipelines is designed to Scale a SQL Pool within Azure Synapse Analytics. A SQL Pool(Former Azure SQL DW) linked to a SQL (Logical) Server has a slightly different approach(documentation can be found on Github).

Scaling a SQL Pool is actually a necessary functionality during your Data Movement Solutions, it will help to save and optimize your costs.

ADF Synapse SQL Pool Scale

Documentation of this pipeline can be found on GitHub.

You can also use this template in Azure Synapse and the details can also be found on Github more details can also be found in this Article.

 

 

In case you have unanswered questions please do not hesitate to contact me.

Feel free to leave a comment

Scottisch Summit 2021(Video)

Scottisch Summit 2021(Video)

by | Mar 8, 2021 | Events

Synapse

Recording of my session during Scottisch Summit 2021

Is there a way that we can build our Azure DataFactory all with parameters based on MetaData?

Azure Purview MSIgnite Spring 2021 Announcements

Azure Purview MSIgnite Spring 2021 Announcements

by | Mar 3, 2021 | Azure, Microsoft Purview

Synapse

by Erwin | Mar 3, 2021

Azure Purview Ignite 2021 Annoucement

Pricing

This week the Azure Purview announced that they will extend the Azure Purview offer to provision 4 Capacity Units of the Data Map for free till May 31, 2021! Charging will start on June 1, 2021. Great news for customer who start exploring Azure Purview

More detail on Azure Purview pricing can be found here

New Sources

As you see above a lot of new Sources to scan have been announced and there will be more sources coming in the next months.

  • AWS Simple Storage Service (S3) data
  • SAP S/4 HANA
  • SAP ECC
  • Teradata
  • Oracle

And last but least, Azure Multiple. With Azure Multiple you can now govern multiple Azure sources within Azure Purview by registering an entire Azure subscription or a resource group.

Azure Multiple

And you setup scan and triggers to the discover assets within your subscription.

Azure Multiple scan

More detail on how to setup Azure Multipe How to scan Azure multiple sources. 

Also for Azure Multiple more resource types are coming in the upcoming weeks

Private Endpoint

You can now set up scans on your on-premises and Azure data sources with ingestion private endpoints and a self—hosted integration runtime to ensure network isolation, and metadata traversing only via private networks end-to-end and block all access to your Purview instance from public internet.

Azure Purview Endpoint

More details on pricing Pricing - Azure Purview

Azure Purview Documentation  Documentation - Azure Purview

Azure Purview Q&A Q&A -Azure Purview

 

In case you have unanswered questions please do not hesitate to contact me.

Feel free to leave a comment

Scottisch Summit 2021(Video)

My Virtual Session Scottish Summit 2021

by | Feb 27, 2021 | Events

Scottisch Summit 2021

This Saturday I've been speaking during Scottisch Summit 2021. It was my first Summit, but is was a great event, with more than  400 sessions covering the full Microsoft Stack in 7 different language English, Spanish, German, French, Italian, Portuguese and Polish. Proud that I was to able to join and to present.

Azure Data Factory

I presented a session on if there is a way that we can build our Azure Data Factory all with parameters based on MetaData?

In the beginning of my sessions the audio wasn't that well. I just double checked my uploaded recording and in there audio was fine.

You can watch the recording of my session Scottisch Summit 2021

Code from my demos can be found here.

You can find my slides below on Slideshare:

In case you have any questions left please feel free to ask them via the comment or Socials.

My Virtual Session DataSaturday #1 Pordenone

My Virtual Session DataSaturday #1 Pordenone

by | Feb 27, 2021 | Data Saturday, Events

Synapse

DATA SATURDAY #1 PORDENONE

This Saturday I've been speaking during DataSaturday #1 Pordenono. The first ever DataSaturday after Pass has retired. If you want to visit more Datasaturday events please visit the Data Saturdays event page.

Azure Purview

I presented a session about Azure Purview Microsoft's answer to Data Governance and Data Lineage

You can find my slides below on Slideshare:

 
Some useful links:
 
 
 
 
Must watch sessions during Microsoft Ignite 

In case you have any questions left please feel free to ask them via the comment or Socials