matt-taylor.tech
← Back to projects

Work · Affinity Group

Office 365 → Snowflake user sync

Snowflake Python (Snowpark) Microsoft Graph Entra ID Snowflake Tasks

Daily sync from Microsoft Graph to Snowflake, implemented as a Snowflake-native Python stored procedure running in Snowflake's Python runtime. Replaces an external server + scheduled Python script pattern with no infrastructure to manage and no credentials to rotate outside of Snowflake.

What it does

  • Pulls filtered users from Microsoft Graph (transitive group membership) on a daily schedule.
  • Captures 15 Entra ID extension attributes plus manager ID and manager display name.
  • MERGE for incremental updates: only users with actual changes get written.
  • Soft-delete tracking for users removed from Entra ID; rows are flagged, not silently dropped.

Why Snowflake-native

  • No external server to maintain, patch, or monitor.
  • No separate Python environment, no requirements.txt drift, no venv to worry about.
  • No external secrets manager: Graph credentials live in Snowflake.
  • Outbound Graph calls wired through Snowflake Network Rules and EXTERNAL ACCESS INTEGRATION, so network egress is explicit and auditable.
  • Runs on Snowflake Tasks: scheduling, monitoring, and failure surfacing use the same tooling everything else in Snowflake uses.

Setup

  • Entra ID app registration with the Graph application permissions for user and group read.
  • Snowflake Network Rule allowing outbound to graph.microsoft.com.
  • EXTERNAL ACCESS INTEGRATION binding the Network Rule to the stored procedure.
  • Snowflake Task scheduling the procedure daily.