Customer systems

CRM and ERP platforms expose data through REST APIs, bulk APIs, webhooks, database replication, or scheduled file exports.

Private/on-prem sources usually require a self-hosted integration runtime on customer-side VMs.

API + ingestion layer

Azure API Management is the front door for FlockIQ APIs and partner/source APIs. It supports hybrid scenarios, and Azure recommends private connectivity using Private Link/private endpoints for secure inbound access.

Logic Apps is best for packaged SaaS connectors and event-driven workflows.

Azure Data Factory handles scheduled extraction, normalization, and orchestration for bulk loads and historical backfills. Azure’s architecture guidance distinguishes ETL/ELT orchestration patterns clearly here.

Azure Event Hubs is the streaming backbone for change events, snapshots, and app telemetry. In Standard, Azure documents roughly 1 MB/s ingress and 2 MB/s egress per throughput unit, with partitions used to scale parallelism.

Storage + governance

ADLS Gen2 stores raw, conformed, and curated history.

Purview tracks lineage, schema, and governance.

Azure documents general-purpose v2 storage accounts as the current standard account type, and notes default egress/ingress scalability targets for GPv2 storage.

FlockIQ processing layer

AKS runs the FlockIQ services:

Capture: ingest snapshots and events

Merge: align records across time/system

Compare: detect deltas, drift, anomalies

Signal: generate alerts, recommendations, actions

Databricks or Synapse Spark performs heavier joins, historical compare, feature generation, and large-scale transformations.

Azure SQL Database stores metadata, jobs, pipeline status, config, mapping rules, and user/workflow state.

Redis improves low-latency retrieval for current signals and orchestration.

Serving

Curated datasets flow to Power BI / Fabric.

Operational read models can be served from Cosmos DB or Azure SQL depending on query shape.

Optional Azure OpenAI / ML endpoints add summarization, forecasting, and recommended actions.