The FMD Framework implements the Medallion Lakehouse architecture using Delta as the canonical storage format.

Landing Zone

• Raw data ingestion into Fabric Lakehouse
• File-based or structured ingestion
• Minimal transformation

Bronze Layer

• Data standardization and deduplication
• Structural normalization into Delta tables
• Preparation for downstream processing

Silver Layer

• Historical tracking and data validation
• Slowly Changing Dimension (SCD) patterns
• Data quality and transformation logic

Gold Layer (Business Domains)

• Business-ready datasets and semantic models
• Fact and dimension modeling
• Optimized for reporting and analytics

FMD Framework Workspaces & Separation of Concerns

The FMD Framework enforces a clear separation between configuration, data, and code across Microsoft Fabric environments.

Configuration Workspace

• Central Fabric SQL Database
• Metadata model and configuration tables
• Logging, monitoring, and audit data

Data Workspaces

• Landing, Bronze, and Silver Lakehouses
• Environment-specific data storage

Code Workspaces

• Data Pipelines
• Notebooks and Spark artifacts
• Orchestration logic

This separation allows the FMD Framework to scale across environments such as DEV, TEST, and PROD with minimal duplication

FMD Framework Metadata Model

At the core of the FMD Framework is a centralized metadata model stored in Fabric SQL Database.

Key Metadata Entities

• Connection – defines data connections and sources
• DataSource – logical grouping of systems and domains
• LandingzoneEntity – ingestion configuration
• BronzeLayerEntity – transformation rules and schema
• SilverLayerEntity – downstream processing metadata
• Execution tables – logging, load tracking, and orchestration

This metadata-driven approach allows the FMD Framework to dynamically execute pipelines without code changes.

More details can be found in WIKI

FMD Framework Orchestration & Processing Patterns

The FMD Framework provides a metadata-driven orchestration model that dynamically controls pipeline execution and data processing across Microsoft Fabric.

Orchestration Model

Within the FMD Framework, orchestrator pipelines determine what to execute based on metadata lookups. These pipelines dynamically trigger child pipelines and notebooks to process data through the standard Landing → Bronze → Silver progression.

This approach ensures consistent, scalable, and automated execution without hardcoding logic.

Supported Processing Activities

The FMD Framework standardizes execution patterns by using a consistent set of supported Fabric activities, including:

• Lookup – retrieve metadata and configuration
• Copy – perform data ingestion and movement
• Get Metadata – inspect source and file information
• Delete – manage lifecycle of intermediate data
• Script / Stored Procedure – execute database logic and transformations

These activities enable the FMD Framework to support diverse data sources while maintaining a unified orchestration model.

Bulk Onboarding Pattern

The FMD Framework includes automation for rapid onboarding of new data sources:

• PL_TOOLING_POST_ASQL_TO_FMD extracts table metadata from SQL-based sources
• Populates the FMD metadata model automatically
• Eliminates manual configuration effort
• Accelerates onboarding of large numbers of tables

This bulk onboarding capability allows the FMD Framework to scale efficiently across enterprise environments, reducing time-to-value for new integrations.

FMD Framework Logging & Observability

The FMD Framework includes built-in logging and observability capabilities, providing full visibility into pipeline execution, data movement, and notebook processing across Microsoft Fabric.

Out-of-the-box audit tables capture execution details across all core components of the FMD Framework:

• audit.PipelineExecution – tracks start, end, and failure status for each pipeline run
• audit.CopyActivityExecution – captures execution details for individual copy activities
• audit.NotebookExecution – logs execution status for notebooks and Spark workloads

These audit tables provide a centralized foundation for monitoring, troubleshooting, and operational insight.

Monitoring & Integration

With the FMD Framework, you can easily integrate these audit logs into your broader monitoring strategy:

• Connect audit tables to your enterprise monitoring solutions
• Build custom dashboards and reporting in Fabric
• Create semantic models for operational insights and analytics
• Enable proactive alerting and performance tracking

This approach ensures the FMD Framework delivers not just data integration, but also enterprise-grade observability and operational control.

When to Use the FMD Framework

The FMD Framework is ideal when:

• You want to standardize data integration across teams
• You need scalable and reusable pipeline patterns
• You are building a Microsoft Fabric-native data platform
• Governance, lineage, and auditability are critical
• You want to reduce manual pipeline development

FMD Framework GitHub & Resources

To explore the FMD Framework in detail, including deployment and configuration:

To deploy the framework:

1. Review the Deployment Guide – Walks through prerequisites and setup steps, can be found in the GitHub repository.

Conclusion: Why the FMD Framework Matters

The FMD Framework represents a shift toward metadata-driven data engineering on Microsoft Fabric. By separating configuration from execution and standardizing patterns across the platform, the FMD Framework enables organizations to build scalable, governed, and future-proof data solutions.

Whether you are modernizing an existing data platform or building a new one on Microsoft Fabric, the FMD Framework provides the foundation for automation, consistency, and enterprise-grade data integration.

 Video's