October=Speaking Month

October=Speaking Month

by | Aug 28, 2019 | Events

Erwin

DataSaturday Holland and DataMinds in Belgium

Like last year I have been selected again this year for both events to speak.

Cool to be able to speak at 3 different events within 7 days, but also to see many known people and to met new people.

Data DSaturdayData Saturday

Data Saturday Holland is the new name for SQL Saturday Holland, The Microsoft Data Platform themed community event which will be held at a new location Pathé Leidsche Rijn.

This year I will for the first time join forces with Ahmed Mansour, Cloud Solution Architect @Microsoft

 

Session Title:

Help, I need to migrate my On Premise Database to Azure, which Database Tier do I have to choose from.

Session Details:

During this session we will walk you through all the different Tiers in Azure, DTU, Vcore, Serverless and Managed Instance and will provide examples when to use which Tier.
We will also show you the Microsoft Data Migration Assesment (DMA). This tool will help you to decide which tier you should choose. So if you need help or just interested in the different Azure Database Tiers then visit our session

Saturday October 5, 2019 16:00pm - 16:45pm

Room 4

Tickets are still available, so reserve your spot quickly.

ConnectDataMinds Connect

This year DataMinds Connect will be held in Lamot, a unique convention center located along the water, in the historic center of Mechelen.

 

Session Title:

Azure Key Vault, Azure Dev Ops and Data Factory how do these Azure Services work perfectly together!

Session Details:

Can we store our Connectionstrings or BlobStorageKeys or other Secretvalues somewhere else then in Azure Data Factory(ADF)? Yes you can! You can store these valuable secrets in Azure Key Vault(AKV). But how can we achieve this in ADF? And finally how do we deploy our DataFactories in Azure Dev Ops to Test, Acceptance and Production environments with these Secrets ? Can this be setup dynamically? During this session I will give answers on all of these questions. You will learn how to setup your Azure Key Vault, connect these secrets in ADF and finally deploy these secrets dynamically in Azure Dev Ops. As you can see a lot to talk about during this session.

Tuesday October 8, 2019 16:45pm - 17:45pm

Room Alcazar

No tickets yet,  you can order them here!

Techorama NL

I will speaking during Techorama NL, I've written a post about this event a couple days ago, which you can find here

See you all in October on one of these events.

Speaking at SQL Server Konferenz 2020

SQL Server Konferenz  SQL Server Konferenz 2020 is a international Conference which is held each year in Darmstadt, Germany. Trilled to be speaking This year I will speak and attend SQL Server Konferenz 2020 from March 3th until March 5th. I was very happy when I...

Read More

My Virtual session at Data Toboggan

An inaugural event specializing on Azure Synapse Analytics Data Toboggan This Saturday I've been speaking during Data Toboggan an inaugural event specializing on Azure Synapse Analytics. 12 Hours of sessions with amazing speakers. Azure Purview I presented a session...

Read More

My Virtual Session at SQLBits 2020

SQL BITS 2020, the greatest data show  Last week was SQL BITS week. After the event was moved from April to September, it eventually became a Virtual event. Setting up a Virtual event requires a lot of adjustments in the Organization. Recording All regular sessions...

Read More

Data Toboggan January 2022 (Video)

Recording of my session during the Data Toboggan January 2022 conference. Dealing with different roles in Azure Synapse Analytics

Read More

My Virtual Session Cloud Lunch and Learn Marathon

Cloud Lunch and Learn Marathon 2021 This Thursday May 13th 2021 I've been speaking during Cloud Lunch and Learn Marathon 2021. It was the first Cloud Lunch and Learn Marathon conference, more then 1200 registered attendees, 24hours Live and pre recorded sessions....

Read More

My Virtual Session Scottish Summit 2021

Scottisch Summit 2021 This Saturday I've been speaking during Scottisch Summit 2021. It was my first Summit, but is was a great event, with more than  400 sessions covering the full Microsoft Stack in 7 different language English, Spanish, German, French, Italian,...

Read More

My first Virtual session in 2022 for Dataminds

DataMinds This Tuesday I've joined the DataMinds user Group to talk about Azure Purview.   Azure Purview It was the first session of the year for both me and DataMinds. Some great questions during the session, thank you for that. You can find my slides below on...

Read More

Scottisch Summit 2021(Video)

Scottisch Summit 2021(Video) Recording of my session during Scottisch Summit 2021 Is there a way that we can build our Azure DataFactory all with parameters based on MetaData?

Read More

SQL SERVER KONFERENZ 2020

SQL SERVER KONFERENZ 2020 Date: 4-5 March Location: KongressCenter Darmstadt   Speaking Dinner The speaker dinner on Tuesday evening was held in Restaurant Sitte. And yes what do you eat when you are in Germany a delicious Schnitzel with baked potatoes. It was a...

Read More

SQLBITS 2022

SQL BITS 2022 What a great time I had Wednesday 9th of March the time had finally come, at 1830 my plane would leave for London City. After all the preparations over the past few weeks, the time was there. After arriving in London City and checking in at the hotel, I...

Read More
Speaking at Techorama NL

Speaking at Techorama NL

by | Aug 19, 2019 | Events

Techorama = Deep Knowledge IT Conference

Techorama is a yearly international technology conference which takes place at Pathé Ede, Netherlands from September 30th until October 2nd. This will be the second year the event will be in held in the Netherlands. Last year more then 1100 people attend the Conference. This year the organisation expect around 1500 attendees.

Proud to be speaking

This year I will also speak at Techorama. I was very happy when I received a message at the end of April that my session was selected. Something to be very proud of.

Session Title:

Azure Key Vault, Azure Dev Ops and Data Factory how do these Azure Services work perfectly together!

Session Details

Can we store our Connectionstrings or BlobStorageKeys or other Secretvalues somewhere else then in Azure Data Factory(ADF)? Yes you can! You can store these valuable secrets in Azure Key Vault(AKV). But how can we achieve this in ADF? And finally how do we deploy our DataFactories in Azure Dev Ops to Test, Acceptance and Production environments with these Secrets ? Can this be setup dynamically? During this session I will give answers on all of these questions. You will learn how to setup your Azure Key Vault, connect these secrets in ADF and finally deploy these secrets dynamically in Azure Dev Ops. As you can see a lot to talk about during this session.

Wednesday October 2, 2019 10:00am - 11:00am

Room 5

 

Complete agenda

September 30th:

Workshops

October 1st and 2nd:

Sessions

 

No tickets yet, click here to reserve your spot. Hope to see you all at Techorama.

 

SQLBITS 2022

SQL BITS 2022 What a great time I had Wednesday 9th of March the time had finally come, at 1830 my plane would leave for London City. After all the preparations over the past few weeks, the time was there. After arriving in London City and checking in at the hotel, I...

Read More

DataMinds 2018

An event in the Footbal Arena from Gent! Two days after SQLSatholland it was time for DataMinds Connects, which was held in beautiful Belgian Ghent. The location for this event was in the stadium of KAA Gent. Because the organization of SQLSatHolland and DataMinds...

Read More

Data Saturday Stockholm

Data Saturday Stockholm Bra så alla tillbaka personligen This means great so everyone back in person. Last weekend I had the honor to speak at Data Saturday in Stockholm. About one of my favourite topics Azure Synapse Analytics. It was nice to see everyone in person...

Read More

DataGrillen 2022

DataGrillen 2022 Microsoft Purview When we say: Data, bratwurst and beer, we are of course talking about DataGrillen. After more than 2 years of absence, it was time again in recent days, with speakers from all over the world with almost 50 sessions, good weather and...

Read More

My Virtual session DataWeekender 4.2

DataWeekender 4.2 This Saturday I've joined the Van and Spoke at DataWeekender Azure Purview I presented a session on Azure Purview Microsoft's answer to Data Governance and Data Lineage You can find my slides below on Slideshare: Data weekender4.2 azure purview erwin...

Read More

The new and refreshing Azure Data Factory home page

The Data Factory home page has been redesigned for a modern and more accessible (WCAG 2.1) experience. The new design is more fluid with better contrast and reflow capabilities.  Thank you to the complete product team for this great experience The new and refreshing...

Read More

Techoroma NL 2019

Techoroma NL 2019 Azure Key Vault, Azure Dev Ops and Azure Data Factory how do these Azure Services work perfectly together! Some impressions during Techorama NL 2019.    

Read More

My virtual session at Data Toboggan

Data Toboggan This Saturday I've joined the Data Toboggan to talk about Azure Synapse Analytics.   Azure Synapse Analytics Today I've been talking on how to deal with all the different roles in Azure Synapse Analytics during Data Toboggan. An event 100% focussed...

Read More

Datatoboggan 2021(Video)

Recording of my session during Data Toboggan 2021 Azure Purview Microsoft's answer to Data Governance and Data Lineage

Read More

SQL BITS 2022 Session recordings

Recordings SQL Bits 2022 All sessions of SQLBits 2022 have been made available to everyone and can now be viewed via their Youtube channel. Microsoft asked me to present me this session during SQL Bits in the Cloud Scale Analytics solution area. Session Title: Lake...

Read More
ADF: Get Metadata Activity stopped working

ADF: Get Metadata Activity stopped working

by | Jul 1, 2019 | Azure, Azure Data Factory

Erwin

by Erwin | Jul 1, 2019

Meta Data Activity

Today my pipelines in Azure Data Factory (ADF) suddenly stopped working. The output structure was not found. Quit strange while these pipelines have been running for weeks.

 

 

Invalid Template

After debugging my Pipeline I found out the output.structure couldn't be evaluated.

ADF METADATA Error

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Field List

I don't know when this option was added, but if you open the Get Metadata Activity you will find under Dataset the option Field List

ADF METADATA Error field list

 

 

 

 

 

 

 

The only thing I needed to do get my previous output back was adding structure as an argument. The structure will take care of the Metadata of my table and will be used in the Copy Data Activity.

ADF METADATA Structure

 

 

 

 

 

 

 

 

After rerunning my Pipeline the error was gone.ADF METADATA working

 

 

 

 

 

 

 

 

I changed all my pipelines and the processes where running fine again. I took me bit to find out what the exact error was.

 

Hopefully it will save you some time after reading this blog.

More about the Get Metadata Activity can be found here .

Feel free to leave a comment

Datagrillen: Data, Bratwurst und Beer

Datagrillen: Data, Bratwurst und Beer

by | Jun 24, 2019 | Events

Data, Bratwurst and Beer

An event that started 5 years ago as a small event, that has grown into an event with 200 participants, 50+ sessions, 5 tracks, 2 days and a BBQ in a small place in Germany, called Lingen. William Durkin and Ben Weissman are the main organizers, but are supported by a bunch of volunteers and supported by a great group of sponsors.

Speaker/Volunteer diner

Wednesday the event started for me with a drink in Hotel Lobby and with a great dinner in the Alte Posthalterei. We had a great evening, met some old friends and made some new friends.

DataGrillen Dinner

Sessions

DataGrillen Keynote

The first day started early, we helped preparing the badges and did the check in. Around 9:00 am almost every attendee was in and ready for the Keynote. During the keynote there was a special welcome to all the newcomer Speakers and their mentors. This newcomer track is designed to accommodate people new to presenting. In my opinion this newcomer tracks should be introduced to all events.

During these 2 days there were 5 sessions divided over 5 tracks on both days. I have attended many nice sessions, but I have certainly gained a lot of new inspiration out of these sessions.

DataGrillen Axians

Day 1 ended with the traditional bbq. Personally, I love bbq a lot and know that it is hard work to get a bbq for 200 people, but the catering did very well. Compliments to the catering.

During the days I have spoken to many people, both well-known and also many new ones. From Axians we joined this event with 3 people.

We are 1 large #sqlfamily.

 

 

Rafle time

After 2 tiring days, the event was traditionally closed on Friday by William and Ben with the raffle. And that was for me also the end of 2 great days and I could start with my home trip.DataGrillen Raffle

 

Sponsors

Thank you Solisyon, Datamasterminds, SentryOne, PowerBi Sentinel, dbWatch, Rosen, Shop Apotheke Europe, Redgate, Hedda and IT Emsland without you the event wasn't free, there where no drinks and there was no food.

Next Year

Next year the event will take place on may 28 and may 29.

Call for Speakers is  open https://sessionize.com/datagrillen-2020/

See you next year. And once again thank you William and Ben for organizing

DataGrillen 2022

DataGrillen 2022 Microsoft Purview When we say: Data, bratwurst and beer, we are of course talking about DataGrillen. After more than 2 years of absence, it was time again in recent days, with speakers from all over the world with almost 50 sessions, good weather and...

Read More

Data Saturday #1 Pordenone(Video)

Recording of my session during Data Saturday #1 Pordenone Azure Purview Microsoft's answer to Data Governance and Data Lineage  

Read More

My First Blog Post SQLSatHolland 790

Yeah, my first blog is LIVE After a good talk with Reza Rad from RADACAD during SQLSatHolland, I decided to start my first blog!Knowledge sharing is very important, it gives me a lot of energy. But it also gives others people in the community energies to pick up new...

Read More

SQL SERVER KONFERENZ 2020

SQL SERVER KONFERENZ 2020 Date: 4-5 March Location: KongressCenter Darmstadt   Speaking Dinner The speaker dinner on Tuesday evening was held in Restaurant Sitte. And yes what do you eat when you are in Germany a delicious Schnitzel with baked potatoes. It was a...

Read More

Watch the MS Ignite sessions on-demand

MS Ignite Sessions MS Ignite 2020 was this year a virtual event. Most of the sessions were live in the evenings and the other sessions were available at different times in different time zones. Compliments to the MS Ignite team for organizing such a great event Most...

Read More

My Virtual Session DataSaturday #1 Pordenone

My Virtual Session DataSaturday #1 Pordenone DATA SATURDAY #1 PORDENONE This Saturday I've been speaking during DataSaturday #1 Pordenono. The first ever DataSaturday after Pass has retired. If you want to visit more Datasaturday events please visit the Data Saturdays...

Read More

Data Saturday Stockholm

Data Saturday Stockholm Bra så alla tillbaka personligen This means great so everyone back in person. Last weekend I had the honor to speak at Data Saturday in Stockholm. About one of my favourite topics Azure Synapse Analytics. It was nice to see everyone in person...

Read More

October=Speaking Month

October=Speaking Month DataSaturday Holland and DataMinds in Belgium Like last year I have been selected again this year for both events to speak. Cool to be able to speak at 3 different events within 7 days, but also to see many known people and to met new people....

Read More

DataSaturday NL 2019

Recording of my session on DataSaturdayNL 2019 Can we store our Connectionstrings or BlobStorageKeys or other Secretvalues somewhere else then in Azure Data Factory(ADF)? Yes you can! You can store these valuable secrets in Azure Key Vault(AKV). But how can we achieve...

Read More

My Virtual Session DataSaturday #14 Oslo

DATA SATURDAY #14 OSLO This Saturday I've been speaking during DataSaturday #4 Oslo. If you want to visit more Datasaturday events please visit the Data Saturdays event page. Azure Purview I presented a session on Azure Purview Microsoft's answer to Data Governance...

Read More

SSMS 18.xx: Creating your Azure Data Factory SSIS IR directly in SSMS

by | Jun 18, 2019 | Sql Server Management Studio, Azure Data Factory

Creating your Azure Data Factory(ADF) SSIS IR in SSMS

Since  version 18.0 we could see our Integration Catalog on Azure Instances directly.

Yesterday I wrote an article how to Schedule your SSIS Packages in ADF, during writing that article I found out that you can also created your SSIS IR(Integration Runtime) directly in SSMS.

I never saw the option before, it was always grayed out(because the SSISDB was already in Azure). But if you connect to an On Premise system it will not be grayed out.

An easy scenario to lift and shift your SSIS Packages to Azure, with a couple of steps you can create your SSIS IR.

Go to your Integration Services Catalog on your On Premise Server. Right Click and select the option "Try SSIS in Azure Data Factory" and click on Next.

Create ADF

 

 

 

 

 

 

 

 

Prerequisites

As Prerequisites you will need:

An Azure account.

An Azure SQL Database server or Managed Instance.

 

Configuration of your SSIS IR

Create SSIS IR in Azure wizard.

Creating ADF SSIS SSMS

On the right upper corner you can log in to your Azure Subscription.

Select the correction Subscription, in case your have more subscription assigned to this account.

Select the Catalog Server where you want to install your SSIS IR on. This server cannot have an existing SSISDB.

Provide the necessary log in details and click on connect.

 

 

 

 

 

 

Creating ADF SSIS SSMS Validation

 

The last step is to Create your SSIS IR.

After you have clicked on Create,  SSMS will do some validations and after these validations you will be asked to open the ADF Portal.

 

 

 

 

 

 

 

 

 

 

Azure Data Factory Portal

Creating ADF SSIS

 

 

 

After you logged into the portal you will see a summary of the detail how your SSIS IR will be setup.

If you click on next your SSIS IR will be started.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Change Settings

To change any of the Parameters above you need to use the previous button. You can change:

Number of Parallel Executions Per Node

Custom Setup Container SAS URI

VNET

Node Size

License

The name of your SSIS IR and Resource Group are create by the setup and cannot be changed.

The Resource Group for example is created in the Region US-East.

 

ADF Pipeline

 

 

Conclusion

It is a simple way to setup your SSIS IR in Azure, but I do prefer do this my self trough the Azure Portal or automated with a PowerShell script.