---
id: intro_data_azure
title: Ensono Stacks Azure Data Platform
linkText: Introduction
weight: 1
keywords:
  - data
  - python
  - etl
  - databricks
  - azure
  - adf
  - template
---

The https://github.com/ensono/stacks-azure-data[Ensono Stacks Azure Data Platform] solution provides
a framework for accelerating the deployment of a production-ready modern data platform in Azure.

image::../images/stacks-data-overview.png[Ensono Stacks Data Overview]

. Use the link:../../../stackscli/about.adoc[Ensono Stacks CLI] to generate a new data platform project.
. Build and deploy the data platform infrastructure into your Azure environment.
. Accelerate development of data workloads and ETL pipelines with link:./data_engineering/datastacks.adoc[Datastacks].

The Ensono Stacks Data Platform delivers a modern *Lakehouse* solution, based upon the link:./data_engineering/data_engineering_intro_azure.adoc#medallion-architecture[Medallion Architecture], with Bronze, Silver and Gold layers for various stages of data preparation. The platform utilises tools including **Azure Data Factory** for data ingestion and orchestration, **Databricks** for data processing and **Azure Data Lake Storage Gen2** for data lake storage. It provides a foundation for data analytics and reporting through *Microsoft Fabric* and *Power BI*.

Key elements of the solution include:

* Infrastructure as code (IaC) for all infrastructure components (Terraform).
* Deployment pipelines to enable CI/CD and DataOps for the platform and all data workloads.
* link:./data_engineering/datastacks.adoc[Datastacks] - a library and CLI built to accelerate the development of data engineering workloads in the data platform.
* A framework for link:./data_engineering/data_quality_azure.adoc[data quality validations].
* link:./data_engineering/testing_data_azure.adoc[Automated tests] to ensure quality assurance and operational efficiency.
* Sample link:./data_engineering/ingest_data_azure.adoc[data ingest pipelines] that transfer data from a source into the landing (Bronze) data lake zone.
* Sample link:./data_engineering/data_processing.adoc[data processing pipelines] performing data transformations from Bronze to Silver and Silver to Gold layers.

== High-level architecture

image::../images/Stacks_Azure_Data_Platform-HLD.png[High-level architecture]

== Infrastructure deployed

* Resource Group
* Key Vault
* Azure Data Lake Storage Gen2
* Azure Blob Storage
* Azure Data Factory
* Log Analytics Workspace
* Databricks Workspace
* Azure SQL Database (optional, for testing the platform)

The deployed platform can link:./data_engineering/data_engineering_intro_azure.adoc#fabric-lakehouse[integrate with Microsoft Fabric] to provide a suite of analytics tools and capabilities.

The solution may be deployed within a secure private network. For details please see
link:./architecture/infrastructure_data_azure.adoc[Infrastructure & Networking] 

== Data Engineering workloads

Example data engineering workloads are provided, based upon the Datastacks templates:

- link:./data_engineering/ingest_data_azure.adoc[Ingest] 
- link:./data_engineering/data_processing.adoc[Bronze to Silver] 
- link:./data_engineering/data_processing.adoc[Silver to Gold] 

Each of the ingest and data processing workloads may optionally include - link:./data_engineering/data_quality_azure.adoc[Data Quality checks]
