Azure CSP Billing Export

Modified on Sun, 17 Aug at 6:56 PM

Azure Service Principal Onboarding with Billing Analytics

A Complete Guide to Automated Azure Billing Export and Analysis

Executive Summary

This solution provides automated Azure billing data export and analysis using Azure Synapse Analytics with Managed Identity authentication. It creates a service principal (wiv_account) that handles all authentication without tokens or manual maintenance.

Key Benefits:

✅ Zero maintenance - No tokens to renew
✅ Automated setup - Single script deployment
✅ Cost-effective - Pay only when querying data
✅ Cross-subscription support - Centralized billing view

Cost Summary

What Will This Cost Me?

Company Size	Monthly Azure Resources	Monthly Cost
Startup	< 100 resources	$0.06
Small Business	100-500 resources	$0.50
Medium Company	500-1,000 resources	$1.60
Large Enterprise	5,000-10,000 resources	$15.50
Very Large	100,000+ resources	$25-75 (with optimization)

What Am I Paying For?

Storage: ~$0.10/month (storing billing data)
Queries: $5 per TB of data analyzed (you'll use < 0.3 TB = $1.50/month)
No hidden costs: No VMs, no licenses, no idle charges

Cost Comparison

This Solution: $0.06 - $15/month
Manual Reports: 4 hours × $50/hour = $200/month
Your Savings: ~$185-195/month

Quick Start Guide

Prerequisites

Azure subscription with billing access
Azure CLI installed
Bash shell (Linux/Mac/WSL)
Python 3.x (optional, for remote queries)

One-Command Setup

# Clone and run
git clone -b feature/billing-export-synapse https://github.com/wiv-ai/AzureOnBoarding.git
cd AzureOnBoarding
./startup_with_billing_synapse.sh

That's it! The script handles everything automatically.

What Gets Created

1. Service Principal (`wiv_account`)

A secure identity for accessing Azure resources without passwords.

Assigned Roles:

Cost Management Reader (view billing data)
Storage Blob Data Reader (access exports)
Synapse Administrator (manage analytics)
Monitoring Reader (track usage)

2. Resource Group (`wiv-rg`)

Container for all billing analytics resources.

3. Storage Accounts

Billing Export Storage: Stores daily cost data
Synapse Data Lake: Required for analytics workspace

4. Synapse Workspace (`wiv-synapse-billing`)

Analytics engine for querying billing data.

5. Billing Export Configuration

Automated daily export of Azure costs to storage.

Architecture Overview


Your Azure Account
    │
    ├── Service Principal (wiv_account)
    │   └── Manages all authentication
    │
    ├── Daily Billing Export
    │   └── Automatically saves to Storage
    │
    └── Synapse Analytics
        └── Query billing data on-demand

Setup Options

Fresh Setup (Recommended)


./startup_with_billing_synapse.sh
# When prompted: Use existing billing export? (y/n): n

Use Existing Billing Export


./startup_with_billing_synapse.sh
# When prompted: Use existing billing export? (y/n): y
# Then provide:
# - Storage Account Name
# - Resource Group
# - Container Name
# - Export folder path

⚠️ Important Notes

First-Time Setup

Initial export: Takes 5-30 minutes to appear
Daily updates: Run at midnight UTC
Historical data: Available after first export

Data Deduplication

The system automatically handles Azure's cumulative exports:

Day 1: Contains Day 1 data
Day 2: Contains Day 1 + Day 2 data
Solution: Views automatically query only the latest file

Cross-Subscription Access

✅ Can read billing from other subscriptions
✅ Requires proper permissions
✅ Centralized cost analysis

Troubleshooting

Common Issues & Solutions

Issue	Solution
"Login failed"	Wait 2-3 minutes for permissions to propagate
"No data found"	Wait for first export (5-30 minutes)
"Access denied"	Re-run script, it will fix permissions
"Lock conflict"	Script auto-retries 10 times

Manual Verification

# Check service principal az ad sp show --id wiv_account 
# Check Synapse workspace az synapse workspace show --name wiv-synapse-billing --resource-group wiv-rg 
# Check billing export az costmanagement export list --scope "/subscriptions/YOUR_SUBSCRIPTION_ID"