Configuring SKUs for Microsoft Purview Data Governance

Configuring SKUs for Microsoft Purview Data Governance

Microsoft Purview

by Erwin | Jan 19, 2025

In my previous blog I explained that Microsoft Purview for Data Governance has a new pricing model, in this blog we will dive a bit deeper on how you set the different SKU's

Microsoft Purview offers a comprehensive data governance solution that helps organizations manage, protect, and understand their data across various environments. One crucial aspect of optimizing Purview's performance is configuring the appropriate SKUs  for different tasks, especially for Data Quality (DQ) jobs within Data Health Management. This blog post will guide you through the process of selecting and configuring SKUs to enhance your data governance experience.

Understanding SKUs in Microsoft Purview

SKUs in Microsoft Purview determine the level of resources allocated to your data governance tasks. Higher SKUs provide more processing power, which can significantly speed up data quality jobs, especially those with high data volumes, complex rules, or frequent scans.

How can I set my SKU?

In Purview select the Settings tab followed by the Unified Catalog

Purview_SkU

Select the the Usage Settings to get to see the following option:

Now you can set the SKU Type

  • Data Health Management:
    • Basic: €14.40 per Data Governance Processing Unit
    • Standard: €57.57 per Data Governance Processing Unit
    • Advanced: €230.28 per Data Governance Processing Unit

Steps to Configure SKUs for Data Quality Jobs

  1. Access Data Quality Monitoring:
    • Use the Data Quality Monitoring functionality within Microsoft Purview to view and understand your data quality jobs. This tool provides insights into the performance and status of your DQ jobs, helping you identify areas that may benefit from higher SKUs.
  2. Select Higher SKUs for Faster Processing:
    • For jobs with high data volumes, complex rules, or very frequent DQ scans, consider selecting a higher SKU. Higher SKUs allocate more resources, resulting in faster processing times and more efficient data quality management.
  3. Understand the Impact on Your Bill:
    • Before upgrading to a higher SKU, it's essential to understand the impact on your bill. Navigate to your billing section in the Azure portal, ensuring you have the required permissions to view and manage billing information. Higher SKUs will incur additional costs, so it's crucial to balance performance needs with budget considerations.
  4. Correlate with Your Consumption Report:
    • Review your consumption report to understand how different SKUs affect your overall usage and costs. This report provides detailed insights into your resource consumption, helping you make informed decisions about SKU selection.

Additional Tips for Data Health Management

  • Upgrade SKU for Faster Processing: Regularly assess your data quality jobs and upgrade SKUs as needed to maintain optimal performance.
  • Data Quality Management: Start with the basic SKU for initial setups and gradually move to higher SKUs as your data governance needs grow.
  • Governance Health Controls: Implement governance health controls to monitor and resolve issues proactively.
  • Actions and Resolutions: Use self-serve analytics and reports to take informed actions and resolve data quality issues efficiently.

By carefully selecting and configuring SKUs, you can enhance the performance of your data quality jobs within Microsoft Purview, ensuring a robust and efficient data governance framework.

Conclusion

Configuring the appropriate SKUs for Microsoft Purview Data Governance is essential for optimizing the performance of your data quality jobs. By understanding the different SKUs available and how to set them, you can ensure that your data governance tasks are handled efficiently and effectively. Regularly monitoring your data quality jobs and adjusting SKUs as needed will help maintain optimal performance and manage costs effectively.

By leveraging the insights provided by Data Quality Monitoring and correlating your consumption reports, you can make informed decisions about SKU selection. This proactive approach will enable you to maintain a robust and efficient data governance framework, ensuring that your organization can manage, protect, and understand its data across various environments.

Let me know your thoughts

Useful Links:

Microsoft Purview Data Catalog

Microsoft Purview Data Catalog billing consent

Microsoft Purview data governance pricing concepts

Microsoft Purview data governance pricing announcement

 

Feel free to leave a comment

Configuring SKUs for Microsoft Purview Data Governance

Microsoft Purview’s new pricing model for Data Governance

Microsoft Purview

by Erwin | Jan 6, 2025

Microsoft Purview’s New Pay-As-You-Go Pricing Model for Data Governance

Starting January 6, 2025, Microsoft Purview is set to revolutionize its Data Governance capabilities with the introduction of a new pay-as-you-go pricing model. This change is designed to provide more flexibility and cost-efficiency for organizations managing their data governance needs.

What’s New?

The new pricing model is based on two key metrics:

  1. Number of Unique Governed Assets per Day: This metric counts the unique technical assets, such as tables, files, datasets, and reports, that are actively managed and curated within the Microsoft Purview Unified Catalog. Only assets associated with governance concepts, like data products or critical data elements, are considered governed assets.
  2. Data Governance Processing Units (DGPU) per Run: DGPUs are fully managed compute units used for running compute-heavy capabilities, such as data quality and data health management. Each DGPU represents 60 minutes of compute time, which can be run across varying sets of nodes based on the workload needs.

Why This Matters

This pay-as-you-go model allows organizations to scale their data governance efforts according to their specific needs and usage patterns. By only paying for the assets they actively govern and the compute resources they use, businesses can achieve greater cost efficiency and flexibility.

Understanding Data Governance Processing Units (DGPUs)

DGPUs are a crucial component of this new pricing model. They are designed to handle compute-intensive tasks within Microsoft Purview, such as:

  • Data Quality Management: Ensuring the accuracy, completeness, and reliability of data.
  • Data Health Management: Monitoring and maintaining the overall health of data assets.

Each DGPU provides 60 minutes of compute time, which can be distributed across different nodes depending on the specific requirements of the task. This flexibility allows organizations to efficiently manage their compute resources and optimize their data governance processes. DGPU is available in three different performance options: Basic, standard, and advanced. By default any data management rule or health control is run on the Basic SKU. A customer can switch SKU’s based on the speed of compute suitable for their organization.

Microsoft Purview Data Governance | Enterprise Data Catalog

Microsoft Purview Data Governance Enterprise Catalog is billed based on a single meter, the data catalog, which is initiated when customers govern unique data assets. Data assets such as tables, views, AI models, semantic models, and many others that are linked to governance concepts in the product, such as data products and critical data elements, are counted as governed assets. Assets collected in the Purview Data Map but not linked to governance concepts aren't counted as governed assets.

For example, if an organization has 500 tables, views, stored procedures, resource sets, and AI models in their data map, but only 200 unique governed assets in the data catalog, the monthly cost for 30 days would be €95.947 (without discounts). The 300 assets that aren't linked to data products or critical data elements aren't considered governed assets and therefore not counted. Data catalog managed assets are priced uniformly across regions. This pay-as-you-go model for a managed asset is prorated based on days governed within the monthly billing cycle.

Pricing Details 

  • Data Catalog:
    • Standard: €0.0159 per asset per day or ~€0.48 per month
  • Data Health Management:
    • Basic: €14.40 per Data Governance Processing Unit
    • Standard: €57.57 per Data Governance Processing Unit
    • Advanced: €230.28 per Data Governance Processing Unit

For example, if a customer runs 100 Data Management rules and controls in a single day, and each run produces 0.02 DGPU with the Basic SKU, then the total DGPU for that day would equal two DGPU, costing the customer €28.784. Pricing example is based on the US East pricing. Currently the Azure Price calculator is not updated yet.

In  the cost analysis in Azure you will see now 2 new meters:

Service Name:

Microsoft Purview

Meters:

Data Management Basic Data Governance Processing Unit
Data Catalog Standard Asset

Oh yeah finally Azure Purview has now been renamed to Microsoft Purview.

Microsoft Purview Billing Overview

Getting Started

To take advantage of this new pricing model, organizations need an Azure subscription and an Azure resource group within the same tenant as Microsoft Purview. If these resources are already in place for other purposes, they can be utilized for Microsoft Purview as well.

Consent

Make sure you have consent to the new Billing Model more details can be found here.

Conclusion

Microsoft Purview’s new pay-as-you-go pricing model is a significant step forward in making data governance more accessible and cost-effective. By aligning costs with actual usage, organizations can better manage their data governance expenses while ensuring robust data management practices.

Stay tuned for more updates and detailed pricing information as we approach the launch date! After the launch I will get back to with some more Pricing Examples.

Useful Links:

Microsoft Purview Data Catalog

Microsoft Purview Data Catalog billing consent

Microsoft Purview data governance pricing concepts

Microsoft Purview data governance pricing announcement

Feel free to leave a comment

Configuring SKUs for Microsoft Purview Data Governance

Microsoft Purview pricing is changing!

Microsoft Purview

by Erwin | Oct 17, 2024

Microsoft Purview’s New Pay-As-You-Go Pricing Model

UPDATE November 1, 2024

Pricing change will be postponed to January 6th, 2025.

Pricing Consent Purview

Starting November 1, 2024, Microsoft Purview is set to introduce a new pay-as-you-go pricing model for its Data Governance and Data Security capabilities. This update is designed to extend the benefits of Microsoft Purview beyond Microsoft 365, allowing organizations to manage costs more effectively by paying only for the resources they use.

Consent new Purview pricing

What’s New?

Switching to this new model brings several enhanced features and capabilities:

  • Enhanced Data Security Features: Now available for non-Microsoft 365 environments, these features include classification, labeling, and protection, ensuring robust security across various platforms.
  • Redesigned Data Governance Solution: This includes new capabilities such as:
    • Easy-to-Use, Business-Friendly Data Catalog: Simplifies data discovery and management for business users.
    • Top-Notch Data Quality and Health Management: Ensures high data quality and maintains the health of your data assets.
    • Built-In Governance Controls: Provides integrated controls to help manage and enforce data governance policies effectively.

Next Steps

Data Governance Customers

To take advantage of the new capabilities when they become available in your region, you need to consent to switch to the pay-as-you-go model by October 31, 2024. If you do not provide consent by this date, you will remain on the classic pricing model and lose access to the new Data Governance solution after November 2, 2024.

Data Security Customers

Starting November 1, 2024, the pay-as-you-go features for non-Microsoft 365 data in Insider Risk Management and Information Protection will transition from free to a paid preview. To continue using these features, you must consent to switch to the new model before February 28, 2025. If you do not consent by this date, you will lose access to these features, and any protection applied to non-Microsoft 365 data sources will be removed.

Pay-As-You-Go Billing Model

For organizations that operate in multi-cloud environments, the pay-as-you-go billing model offers greater flexibility. This model extends Microsoft Purview’s capabilities beyond Microsoft 365 to include environments such as Azure, AWS, GCP, Box, and Dropbox. The pay-as-you-go model charges based on actual usage, allowing organizations to scale their usage up or down as needed, providing cost efficiency and flexibility.

This model utilizes two types of meters:

  • Asset-Based Meter: This meter counts non-Microsoft 365 items, such as servers, tables, or files.
  • Processing Unit-Based Meter: This meter measures the compute units used for data security and governance tasks.

Microsoft Purview Data Catalog new pricing model with 2 meters that run based on:

  • Number of unique governed assets per day
  • Data Management processing units per run

More details on the what is a Governed Asset, can be found here and processing units can be found here.

Consent and Subscription

Existing Azure Purview customers need to provide consent to switch to the pay-as-you-go model. New customers can link their Azure subscription to start using these features immediately. This ensures a seamless transition and integration with existing Azure services.

Conclusion

Microsoft Purview’s billing models are designed to provide flexibility and scalability, catering to the unique needs of different organizations. Whether you are heavily invested in Microsoft 365 or operate across multiple cloud environments, Microsoft Purview offers a billing model that can help you manage your data governance and security efficiently.

By understanding these billing models, organizations can make informed decisions that align with their operational and financial goals, ensuring robust data governance and security in an ever-evolving digital landscape.

You have some guidelines to define the pricing. As soon as the new pricing model starts, I will try to make the a calculation example so that you will an example for your organization.

 

Links

Microsoft Purview Data Catalog

Microsoft Purview Data Catalog billing consent

Microsoft Purview data governance pricing concepts

Microsoft Purview data governance pricing announcement

 

 

Feel free to leave a comment

Configuring SKUs for Microsoft Purview Data Governance

Microsoft Purview Data Governance Public Preview Rolllout

Microsoft Purview

by Erwin | Apr 22, 2024

Get ready for the next enhancement in Microsoft Purview

Get ready for the next enhancement in Microsoft Purview, as it brings a range of exciting new features and capabilities. To ensure the best experience with Purview, it is recommended that you tag your existing Microsoft Purview accounts appropriately.

Introducing modern data governance for the era of AI

Check the schedule below when the new Experience will reach your region. Make sure you have enabled the new portal experience if not check out my blog post Microsoft Purview new Experience is coming

In the meantime you start watching some video's on the new experience on my Microsoft Purview Content Hub

Feel free to leave a comment

My 2023: A Year of Learning, Sharing and Growing

My 2023: A Year of Learning, Sharing and Growing

Microsoft Purview

by Erwin | Jan 2, 2024

As the year 2023 comes to an end, I want to take a moment to reflect on some of the amazing experiences I had in the past 12 months. It was a year full of learning, sharing and growing, both professionally and personally. I had the opportunity to attend, speak and help at various events, meet new people, visit new places, and become a part of the Microsoft Fabric community. Here are some of the highlights of my 2023:

My first MVP Summit in person in Redmond in April

One of the most memorable events of the year was attending the MVP Summit in person in Redmond, Washington. Visiting the Microsoft headquarters and meeting some of the brilliant minds behind the products and services I use every day. I learned a lot from the sessions, workshops and networking opportunities, and I also had a lot of fun exploring the campus and the city. It was an honor and a privilege to be part of this incredible community of MVPs.

Becoming a Microsoft Fabric Feature Partner in November

Another milestone of the year was becoming a Microsoft Fabric Feature Partner in November. Microsoft Fabric is a new SaaS data platform, I got early access to the platform and provided feedback and suggestions to the product team. I also got to showcase some of the features and benefits of Microsoft Fabric at various events.

Lauched the Microsoft Fabric Content Hub

With the Public review announcement of Microsoft Fabric there was so much content. With this Content hub I try to Stay up-to-date with the latest and most valuable content about Microsoft Fabric, all in one place! From insightful articles and tutorials to engaging videos and community blogs, you’ll find a treasure trove of resources to deepen your understanding. It is one of my best visisted pages this year on my blog

Microsoft Fabric Content Hub - Erwin & Data Analytics (erwindekreuk.com)

Attending and helping during Microsoft Ignite in Seattle in November

Microsoft Ignite is one of the biggest and most exciting events of the year for anyone who is passionate about technology. I had the chance to attend and help during the event in Seattle, Washington. I was amazed by the scale and the quality of the event, and I enjoyed learning from the experts, meeting new and old friends, and discovering the latest innovations and trends in the industry. One of the highlights of the event was the announcement of Microsoft Fabric GA (General Availability), which marked the official launch of the platform to the public.

Speaking and visiting Legoland in June during Data platform Next Step

Data platform Next Step is a unique event that combines data and fun. It was held in June in Billund, Denmark, the home of Lego. I had the opportunity to speak at the event and share my experience and insights on Microsoft Purview. I also had a blast visiting Legoland, the original and largest Lego theme park in the world. It was like a dream come true for a Lego fan.

Helper/volunteer/speaker during SQLBits in March

SQLBits is the leading conference for data professionals to network, develop and share data knowledge in Europe, and it was held in March in Newport. I was happy to be a helper/volunteer at the event, and contribute to its success. I helped with the sessions and anything else that was needed. Besides helping, I had also one session on Microsoft Purview. I also learned a lot from the speakers, the sponsors, and the attendees, and I had a great time in Newport.

Did 2 sessions for SQLDay in May, my first time in Wroclaw, I’ve seen a lot of dwarfs

SQLDay is the biggest data platform conference in Poland, and it was held in May in Wroclaw, the fourth-largest city in the country. It was my first time visiting Wroclaw, and I was impressed by its beauty and history. I did two sessions for SQLDay, one on Microsoft Purview and one on Azure Synapse Analytics. I received positive feedback and questions from the audience, and I enjoyed sharing my knowledge and experience. I also had fun exploring the city and seeing a lot of dwarfs. Wroclaw is famous for its dwarf statues, which are scattered all over the city. There are more than 400 of them, and each one has a different story and personality.

Speaking at events like DataSaturday Stockholm, with an amazing speaker dinner

DataSaturday is a series of events organized by the data community for the data community. I had the chance to speak at several DataSaturday events throughout the year, in different countries and cities. One of them was DataSaturday Stockholm, which was held in May in the capital of Sweden. It was a well-organized and well-attended event, with a lot of interesting sessions and speakers. I spoke about Meta Data driven frameworks and how it can help data professionals to simplify and optimize their data pipelines. I also had an amazing speaker dinner, where I met and mingled with other speakers and organizers. It was a wonderful evening of food, drinks and conversations.

Visited Portugal for the first time to speak during the Iberian Summit

The Iberian Summit is a event from the Portuguese data community, and it was held in April in Olhao, Portugal. It was my first time visiting Portugal, and I was amazed by its culture, cuisine and scenery. I spoke at the event and shared my insights and tips on Azure Synapse Analytics. I also learned from other speakers and attendees, and I had a lot of fun in Olhao.

Had an awesome Inspiration weekend in June with my employer InSpark

InSpark is the company I work for, and it is the leading Microsoft partner in the Netherlands. In June, we had an awesome Inspiration weekend, where we went to a beautiful location and spent two days of learning, brainstorming, and fun. We had sessions and we also had activities, such as climbing. In the evening we had a great White party with beer and wine tastings. It was a great way to get inspired, motivated, and connected with my colleagues.

DataScotland in the Murrayfield Stadium in September

DataScotland is a data platform conference in Scotland, and it was held in September in Edinburgh, the capital of the country. It was a unique event, as it took place in the Murrayfield Stadium, the home of the Scottish rugby team. I spoke at the event and presented on Azure Synapse Analytics togehter with Mathias. I also enjoyed the sessions, the networking, and the atmosphere of the event.

Finally this year DataSaturday Holland was back again, where I spoke about Microsoft Fabric

DataSaturday Holland is the biggest data platform event in the Netherlands, and it was back again this year after a hiatus due to the pandemic. It was held in October in Utrecht, the fourth-largest city in the country. I spoke at the event and gave an overview of Microsoft Fabric and its features and benefits. I also attended some of the sessions, met some of the sponsors, and chatted with some of the attendees. It was a fantastic event, and I was glad to be a part of it.

Techorama Belgium and Techorama the Netherlands

This is year I Spoke at the Belgium and the Dutch Edition. I Belgium I spoke togehter with Marc on Power BI and Microsoft Purview. In Utrecht, the Netherlands I had 2 sessions, one on Microsoft Purview and one on Azure Synapse Analytics

Organized a customer event for all of Data and AI customers of InSpark, which was a really successful event

In September, I organized a customer event for all of the Data and AI customers of InSpark. The event was held in the DDX in Zoetermeer, and it was aimed to showcase some of the latest and greatest technologies and solutions in the data and AI space. I invited some of the experts from Microsoft and other partners to speak at the event, and I also spoke myself about Microsoft Fabric and how it can help customers to achieve their data goals. The event was a really successful one, with a lot of positive feedback and engagement from the customers.

Spoke at several community events

Besides the events I mentioned above, I also spoke at several other community events throughout the year, such as Power BI Gebruikersdag, Power BI gebruikersgroep and more. I always enjoy speaking at community events, as they are a great way to share my passion and knowledge, learn from others, and meet new and old friends. I am very grateful to the organizers, the speakers, and the attendees of these events, for making them possible and valuable.

On a personal level

On a personal level, I also had some achievements and experiences that I am proud of and happy about. One of them was participating in a training called Taking the Stage at the The Speech Republic, led by Natascha Jacobsz and Jantien Streefkerk-van der Meer. In four days, we learned how to take the stage and tell our own story. I do often stand on a stage, but my goal was to create an inspiring story that was not technical. The audience, consisting of family, friends and colleagues, could be themselves during my story. It was a nice step for myself, which was a goal of my training, but which I did not think possible at the beginning and until one of the last days.

Another personal experience that I enjoyed was closing the year with an amazing Christmask dinner with all my colleagues in het Rijk van Keizer in Amsterdam. It was a lovely evening of delicious food, drinks, and conversations. It was a perfect way to celebrate the end of the year and the start of the new one.

Conclusion

2023 was truly an amazing year for me. I learned a lot, shared a lot, and grew a lot, both professionally and personally. I had the opportunity to attend, speak and help at various events, meet new people, visit new places, and become a part of the Microsoft Fabric community. I want to thank everyone who was a part of my 2023, for making it a wonderful and memorable year. ❤️

Let’s celebrate the memories we’ve created and look forward to the adventures that await in 2024! May the new year bring joy, peace, and prosperity to you and your loved ones. Happy New Year!

 

Feel free to leave a comment