Network Automation Toolkit

Production automation scripts for the Essential Coach VMware Aria Automation environment (vra.example.com). This toolkit manages NSX network profiles, fabric network tagging, BlueCat IPAM range mapping, cloud zone capability tags, VM flavor/image/storage profiles, and duplicate cleanup across the TX and VA datacenters.

All scripts are idempotent — safe to re-run. Every write operation supports --dry-run before --execute.

---

Directory Layout

Network_Automation/
├── scripts/
│   ├── aria_mapping.py          Infrastructure mapping tool (v1.6.0)
│   ├── mapper.py                NSX network profile manager (v5.1)
│   ├── cleanup_profiles.py      NSX Federation duplicate cleanup (v2.0)
│   ├── example_config.yaml       Main config for aria_mapping.py
│   └── vlan_location_map.json   286-segment map for --servicenow-tags
└── configs/
    ├── config_tx.yaml           TX datacenter profiles (mapper.py)
    ├── config_va.yaml           VA datacenter profiles (mapper.py)
    └── config_global.yaml       NSX Global Stretched profile (mapper.py)

Two config systems exist because the two tools solve different problems:

• aria_mapping.py uses example_config.yaml — single file, covers the whole Aria infrastructure (flavors, images, storage, cloud zone tags, segment tags, DNS).
• mapper.py uses per-datacenter configs in configs/ — each file targets a specific NSX profile pair (TX, VA, Global) so you can populate one datacenter without touching the others.

---

Which Script Do I Use?

The three scripts have overlapping-sounding names, so here's the practical decision tree. Pick the task you're trying to accomplish, and the table tells you which script to reach for.

What I want to do	Script	Command
Add a new VM size (T-shirt flavor)	aria_mapping.py	--flavors --execute
Onboard a new vCenter template (Windows 2025, RHEL 9, etc.)	aria_mapping.py	--images --execute
Create a storage profile for a dedicated cluster (Oracle, SQL)	aria_mapping.py	--storage --execute
Set capability tags on cloud zones, network profiles, or compute clusters	aria_mapping.py	--tags --execute
Set DNS servers / domain on fabric networks for guest customization	aria_mapping.py	--dns --execute
Add segment:<n> tags to fabric networks (for blueprint tag-based selection)	aria_mapping.py	--segment-tags --execute
Add a network to the ServiceNow catalog dropdown	aria_mapping.py	--servicenow-tags --execute (after editing vlan_location_map.json)
Create the five NSX network profiles (first-time setup)	mapper.py	--create-profiles --execute
Add matching fabric networks to an NSX profile (TX, VA, or Global)	mapper.py	--populate --execute
Remove overlay segments that leaked into VLAN-only (SDDC) profiles	mapper.py	--clean-sddc --execute
Link BlueCat IPAM ranges to fabric networks by CIDR	mapper.py	--ipam-map --execute
Find and remove Federation duplicates (same segment appearing 6–7 times)	cleanup_profiles.py	--analyze then --execute
Resolve a blueprint warning that "multiple networks match"	cleanup_profiles.py	--analyze --segment "<name>" then --execute

---

Technical Breakdown — What Each Script Actually Does

The three scripts sound similar but communicate with different systems and have distinct API footprints. Here's exactly what each one touches and how.

aria_mapping.py — Aria Automation API Only

Purpose: Manage Aria Automation infrastructure configuration. Read and write exclusively against the Aria IaaS REST API (/iaas/api/*). Does not talk to NSX. Does not talk to BlueCat.

Authentication:

• POST /csp/gateway/am/api/login with admin + password → CSP refresh token
• POST /iaas/api/login with refresh token → Bearer access token
• All subsequent calls use Authorization: Bearer <token>

API endpoints consumed:

Endpoint	Method	Purpose
/iaas/api/regions	GET	Enumerate cloud regions (TX-DC01, VA-DC01)
/iaas/api/flavor-profiles	GET / POST / PATCH	Manage T-shirt VM sizes bundled per region
/iaas/api/image-profiles	GET / POST / PATCH	Manage vCenter template-to-image mappings
/iaas/api/storage-profiles	GET / POST / PATCH	Manage storage profiles with computeHostId binding
/iaas/api/zones	GET / PATCH	Apply capability tags to cloud zones (uses tags, not tagsToMatch)
/iaas/api/network-profiles	GET / PATCH	Apply capability tags to network profiles
/iaas/api/fabric-compute	GET / PATCH	Apply capability tags to compute clusters
/iaas/api/fabric-networks-vsphere/{id}	GET / PATCH	Apply segment:<n> tags, servicenow:visible tags, DNS/domain settings
/iaas/api/fabric-network-cidr-ranges/{id}	GET / PATCH	Read DNS profile details per fabric network

Functions (top-level):

• cmd_list_regions() / cmd_list_flavors() / cmd_list_images() / cmd_list_storage() / cmd_list_tags() / cmd_list_networks() — read-only state dumps
• cmd_process_flavors() — create/update flavor profiles, 11 T-shirt sizes bundled into one flavorMapping per region
• cmd_process_images() — create/update the single essential-coach-image-profile with template mappings
• cmd_process_storage() — create storage profiles with compute binding for dedicated clusters
• cmd_process_tags() — apply capability tags to cloud zones, network profiles, and compute clusters
• cmd_process_segment_tags() — apply segment:<n> tags to fabric networks
• cmd_process_dns() — apply domain and DNS server settings to fabric networks
• cmd_process_servicenow_tags() — apply servicenow:visible tags using vlan_location_map.json as the source list

Config file: single example_config.yaml — one file covers both datacenters because the Aria API is unified across regions.

What it does NOT do: does not read from NSX Managers, does not read from BlueCat, does not manage fabric network membership in NSX profiles, does not link IPAM ranges.

---

mapper.py — Aria Automation API + NSX Manager + BlueCat IPAM

Purpose: Manage NSX network profile membership in Aria and link BlueCat IPAM address ranges to fabric networks by CIDR. This is the only script in the toolkit that talks to BlueCat.

Authentication:

• Aria: same flow as aria_mapping.py (CSP login → IaaS Bearer token)
• NSX: HTTP Basic against each NSX Manager's /policy/api/v1/* or /api/v1/* endpoint
• BlueCat: accessed indirectly via Aria's External IP Ranges endpoint (not directly)

Aria API endpoints consumed:

Endpoint	Method	Purpose
/iaas/api/cloud-accounts	GET	Discover cloud accounts and their internal sub-account IDs
/iaas/api/regions	GET	Resolve region IDs for new profile creation
/iaas/api/fabric-networks	GET	Enumerate all discovered fabric networks across vCenter + NSX Local + NSX Global
/iaas/api/external-network-ip-ranges	GET / PATCH	Read BlueCat ranges and link them to fabric networks
/iaas/api/network-profiles	GET / POST / PATCH	Create the five NSX network profiles and manage their fabric network membership
/iaas/api/fabric-networks-vsphere/{id}	PATCH	Apply segment tags and DNS settings (legacy path — aria_mapping.py now authoritative)

NSX API endpoints consumed:

Endpoint	Method	Purpose
/policy/api/v1/infra/segments	GET	Build local segment cache for CIDR resolution
/global-manager/api/v1/global-infra/segments	GET	Build global-manager segment cache for stretched networks

Used only during --ipam-map to resolve segment CIDRs when Aria's fabric network record does not carry the CIDR directly.

Functions (top-level):

• op_list() — dump cloud accounts, profiles, and fabric network counts categorized by source
• op_create_profiles() — create the five NSX network profiles (ESXP TX W01, ESXP VA W01, NSX Overlay TX, NSX Overlay VA, NSX Global Stretched) with their capability tags
• op_clean_sddc() — remove overlay segments that leaked into VLAN-only (SDDC) profiles
• op_populate() — add matching fabric networks to each profile based on segment_prefix, exclude_account_ids, and network_types filters
• op_tags() — apply segment:<n> tags (legacy — prefer aria_mapping.py --segment-tags)
• op_dns() — apply DNS/domain settings (legacy — prefer aria_mapping.py --dns)
• op_ipam_map() — the BlueCat linker. Fetches BlueCat's external IP ranges from Aria, matches them to fabric networks by CIDR (exact / subnet / supernet), and PATCHes the range record to link it to the fabric network. Writes a timestamped CSV report.
• matches_profile() — the core filter logic. Evaluates segment_prefix, cloud_account_filter, exclude_account_ids, network_types, include_patterns, exclude_patterns against each candidate network.

Config files: three per-datacenter YAMLs (config_tx.yaml, config_va.yaml, config_global.yaml) so each datacenter's NSX profile pair can be managed independently. Necessary because NSX Federation causes cross-datacenter contamination if you try to run one pipeline across everything.

What it does NOT do: does not manage flavors, images, storage profiles, or cloud zone capability tags. Those are aria_mapping.py territory.

---

cleanup_profiles.py — Aria Automation API Only (Federation Dedup)

Purpose: Detect and remove duplicate fabric network entries from Aria network profiles. Specifically solves the NSX Federation problem where the same segment is discovered 6–7 times because Local Manager and Global Manager both see it.

Authentication: same Aria flow as aria_mapping.py.

API endpoints consumed:

Endpoint	Method	Purpose
/iaas/api/fabric-networks	GET	Enumerate all fabric networks, group by name to find duplicates
/iaas/api/network-profiles	GET / PATCH	Remove duplicate fabric network IDs from each profile's fabricNetworkIds list
/iaas/api/fabric-networks-vsphere/{id}	DELETE	(Optional, --delete-networks) Delete orphaned fabric network records entirely

Functions (top-level):

• analyze_duplicates() — group fabric networks by name, flag any with 2+ members as duplicates
• select_keeper() — for each duplicate group, pick the keeper using the priority order: (1) has segment:<n> tag AND in preferred cloud account → (2) has tag, any account → (3) in preferred account, no tag → (4) first found
• plan_cleanup() — enumerate every profile and determine which fabric network IDs to remove
• execute_cleanup() — PATCH each affected profile to strip the non-keeper IDs
• delete_orphan_networks() — (opt-in) DELETE the fabric network records that are no longer referenced by any profile

Config: accepts CLI flags only (--analyze, --segment, --keeper-accounts, --execute, --delete-networks). Connection details are hardcoded in the ESSENTIAL COACH_CONFIG dict at the top of the script.

What it does NOT do: does not touch NSX, does not touch BlueCat, does not create or modify non-duplicate networks.

---

Quick reference — which systems each script talks to

Script	Aria IaaS API	NSX Manager API	BlueCat (via Aria)
aria_mapping.py	✅	❌	❌
mapper.py	✅	✅	✅
cleanup_profiles.py	✅	❌	❌

Where the scripts overlap

Both aria_mapping.py and mapper.py expose --tags and --dns commands. Use aria_mapping.py — it is the current tool and was the consolidation target in v1.5.0. The equivalents in mapper.py are retained for backward compatibility with existing runbooks but are not the path forward.

BlueCat IPAM linking is exclusive to mapper.py --ipam-map. aria_mapping.py does not manage the BlueCat-to-fabric-network relationship.

---

Prerequisites

• Python 3.6+
• pip install requests pyyaml urllib3
• Network access to:
  • Aria Automation: vra.example.com
  • NSX Managers (TX, VA, Global): referenced in configs
  • BlueCat IPAM: ipam.example.com
• Service account: admin (stored in configs — rotate via Aria Secrets as needed)

Production host for execution: esxq-vra-ansible at /root/network6/.

---

Script 1 — aria_mapping.py (Infrastructure Mapping Tool)

The Swiss Army knife for Aria Automation infrastructure configuration. Handles everything except NSX profile membership (that's mapper.py's job).

What it manages

Component	Purpose
Flavor profiles	11 T-shirt VM sizes per region (xs → 4xl) bundled into one profile per region
Image profiles	vCenter template-to-image mappings (Windows 2019/2022/2025, RHEL, etc.)
Storage profiles	Creates storage profiles with computeHostId binding for dedicated clusters (Oracle, SQL)
Capability tags	Tags on cloud zones, network profiles, and compute clusters for blueprint placement
Segment tags	segment:<name> tags on fabric networks for blueprint tag-based selection
DNS settings	Domain and DNS server configuration per fabric network (for guest customization)
ServiceNow tags	servicenow:visible tags for dynamic dropdown filtering (reads vlan_location_map.json)

Read-only commands (safe, no changes)

python3 aria_mapping.py --config example_config.yaml --list-regions
python3 aria_mapping.py --config example_config.yaml --list-flavors
python3 aria_mapping.py --config example_config.yaml --list-images
python3 aria_mapping.py --config example_config.yaml --list-storage
python3 aria_mapping.py --config example_config.yaml --list-tags
python3 aria_mapping.py --config example_config.yaml --list-networks

Processing commands (require --dry-run or --execute)

# Preview a single component
python3 aria_mapping.py --config example_config.yaml --flavors --dry-run
python3 aria_mapping.py --config example_config.yaml --segment-tags --dry-run
python3 aria_mapping.py --config example_config.yaml --dns --dry-run

# Process a component for real
python3 aria_mapping.py --config example_config.yaml --tags --execute
python3 aria_mapping.py --config example_config.yaml --servicenow-tags --execute

# Do everything (use with care — hit --dry-run first)
python3 aria_mapping.py --config example_config.yaml --all --dry-run
python3 aria_mapping.py --config example_config.yaml --all --execute

Common workflows

Adding a new VM size — Edit the flavors: block in example_config.yaml, then:

python3 aria_mapping.py --config example_config.yaml --flavors --execute

Onboarding a new Windows OS template — Add the template to the images: block, then:

python3 aria_mapping.py --config example_config.yaml --images --execute

Adding a network to the ServiceNow dropdown — Add the segment name to vlan_location_map.json (key = segment name, value = TX-SDDC or VA-SDDC), then:

python3 aria_mapping.py --config example_config.yaml --servicenow-tags --execute

The vRO action getNetworkSegmentsAll will immediately see the new network on the next catalog request.

---

Script 2 — mapper.py (NSX Network Profile Manager)

Manages fabric network membership across five network profiles that mirror the physical NSX architecture:

Profile	Capability Tag	Contents	Source
ESXP TX W01	network:tx-sddc	NZ-* VLAN segments	TX vCenter
ESXP VA W01	network:va-sddc	NZ-* VLAN segments	VA vCenter
NSX Overlay TX	network:nsx-tx	US-CI-*, G-CI-* overlays	TX NSX Local Manager
NSX Overlay VA	network:nsx-va	US-CI-*, G-CI-* overlays	VA NSX Local Manager
NSX Global Stretched	network:nsx-global	G-* stretched segments	NSX Global Manager

Why three configs

NSX Federation causes the same segment to be discovered multiple times (Local Manager + Global Manager). The three config files isolate each datacenter's profile membership so you don't cross-pollinate:

• config_tx.yaml — manages ESXP TX W01 and NSX Overlay TX. Uses segment_prefix and exclude_account_ids to filter out VA Global and ghost federation entries.
• config_va.yaml — manages ESXP VA W01 and NSX Overlay VA. Mirror of TX.
• config_global.yaml — manages NSX Global Stretched. Includes only ^G- patterns from the Global Manager.

Operations

# Read-only: see current state
python3 mapper.py --config configs/config_tx.yaml --list

# Create missing profiles (first-time setup)
python3 mapper.py --config configs/config_tx.yaml --create-profiles --execute

# Strip overlay segments out of SDDC (VLAN-only) profiles
python3 mapper.py --config configs/config_tx.yaml --clean-sddc --execute

# Add matching networks to their target profiles
python3 mapper.py --config configs/config_tx.yaml --populate --dry-run
python3 mapper.py --config configs/config_tx.yaml --populate --execute

# Apply segment:<n> tags to fabric networks
python3 mapper.py --config configs/config_tx.yaml --tags --execute

# Apply DNS/domain settings
python3 mapper.py --config configs/config_tx.yaml --dns --execute

# Map BlueCat IPAM ranges to fabric networks by CIDR (original v4.0 flow)
python3 mapper.py --config configs/config_tx.yaml --ipam-map --dry-run

# Do everything in sequence: create → clean → populate → tags → dns → ipam-map
python3 mapper.py --config configs/config_tx.yaml --all --execute

Standard full-pipeline order (per datacenter)

# TX first, then VA, then Global
python3 mapper.py --config configs/config_tx.yaml --all --execute
python3 mapper.py --config configs/config_va.yaml --all --execute
python3 mapper.py --config configs/config_global.yaml --all --execute

Filtering concepts (important)

• segment_prefix — authoritative filter. Only networks whose names start with one of the listed prefixes match the profile. This is what cleanly separates US-CI-* (overlay) from NZ-* (VLAN) from G-* (stretched).
• exclude_account_ids — hard blocklist. If a network is associated with any of these cloud account IDs, it's skipped. Used to reject ghost federation entries (e.g., 5b003030 duplicates).
• overlay_only / vlan_only — externalId prefix filter. overlay_only rejects DistributedVirtualPortgroup:* externalIds (VLANs), vlan_only accepts only them.

All three can be combined. segment_prefix is the primary tool; the other two are used when Federation pollution or externalId quirks need to be cleaned up.

CSV reports

Every --ipam-map run writes a timestamped CSV to the configured output directory:

network_mapping_YYYYMMDD_HHMMSS.csv
    Network, CIDR, BlueCat Range, Status

Status values: Linked, Already Linked, No BlueCat Range.

---

Script 3 — cleanup_profiles.py (Federation Duplicate Cleanup)

Solves the NSX Federation double-counting problem. When both Local and Global Managers are integrated into Aria, the same segment gets discovered multiple times — each with a different internal cloud account ID. The result is fabric network lists with 6–7 entries for one real segment, which causes placement warnings and ambiguous IPAM lookups.

Keeper selection priority

When multiple duplicates exist, the script picks the "keeper" in this order:

1. Has a proper segment:<n> tag and is in a preferred cloud account
2. Has a proper segment:<n> tag (any cloud account)
3. In a preferred cloud account (even without tag)
4. First network found (fallback)

Usage

# Analyze — show what's duplicated, no changes
python3 cleanup_profiles.py --analyze

# Analyze a single segment
python3 cleanup_profiles.py --analyze --segment "US-CI-IDM-PROD-SEG01"

# Plan — show exactly what would be removed
python3 cleanup_profiles.py --plan

# Execute — actually remove duplicates from profiles
python3 cleanup_profiles.py --execute

# Execute AND delete the orphaned fabric networks (aggressive)
python3 cleanup_profiles.py --execute --delete-networks

# Specify which cloud accounts to prefer as keepers
python3 cleanup_profiles.py --keeper-accounts "acct-id-1,acct-id-2" --execute

When to run

After any of the following:

• New NSX Manager integrated into Aria
• Cloud account refresh in Aria
• mapper.py --populate run that added networks from overlapping accounts
• Blueprint deployment warnings about "multiple networks match"

---

Config File Reference

example_config.yaml (for aria_mapping.py)

Top-level sections:

• aria: — connection (host, username, password, domain, verify_ssl)
• regions: — target cloud regions (e.g., TX-DC01, VA-DC01)
• flavors: — VM sizing definitions (the 11 T-shirt sizes)
• image_profile_name: — name for the consolidated image profile
• images: — template-to-image mappings per region
• storage_profiles: — storage profile definitions with compute binding
• tags: — capability tag assignments (cloud zones, network profiles, computes)
• segment_tags: — segment tag rules for fabric networks
• network_dns: — domain and DNS server settings per network pattern

config_tx.yaml / config_va.yaml / config_global.yaml (for mapper.py)

Top-level sections:

• vra: — Aria connection
• nsx: — NSX Manager connections (used by --ipam-map)
• sddc_profiles: — VLAN-only profile definitions (ESXP TX/VA W01)
• nsx_overlay_profiles: — overlay profile definitions (NSX Overlay TX/VA)
• nsx_global_profiles: — stretched profile definitions (NSX Global Stretched)
• Each profile supports: name, capability_tag, cloud_account_filter, segment_prefix, exclude_account_ids, network_types, include_patterns, exclude_patterns
• ipam: — BlueCat IPAM settings (provider name, address space, matching method)
• options: — dry_run, skip_existing, verify_ssl

vlan_location_map.json

Flat JSON object mapping segment name → datacenter location:

{
  "G-CI-CC-PROD-SEG02": "TX-SDDC",
  "NZ-77-1-DEV-DATA":   "TX-SDDC",
  ...
}

286 segments. Used by aria_mapping.py --servicenow-tags to tag those networks with servicenow:visible. To add a network to the ServiceNow catalog dropdown, add the key/value here and run the command.

---

Credentials

Credentials for Aria, NSX, and BlueCat are stored inline in the config YAML files. The service account admin has the necessary permissions across all three systems. Rotate through Aria Secrets when needed, and update the configs in this repo afterward.

For hardening, the passwords can be swapped to environment variable reads (os.getenv('VRA_PASSWORD')) in the scripts — not currently wired up, but straightforward to add if the operator prefers not to store them in source control.

---

Common Gotchas

• mapper.py --populate returning 2000+ networks when you expect ~150 — you're missing segment_prefix or exclude_account_ids in the config. This was the TX Federation pollution issue fixed in v5.1.
• Aria Cloud Zone capability tags not sticking — aria_mapping.py v1.5.3 fixed this. The API requires "name" in the PATCH body (400 without it) and uses "tags" not "tagsToMatch". Setting tagsToMatch with location/network tags filters out all hosts. Do not downgrade.
• Flavor profiles only showing the last flavor — you're on aria_mapping.py < v1.5.1. Upgrade — the fix bundles all flavors into a single flavorMapping per region instead of creating one profile per flavor.
• Blueprint deployment warns "multiple networks match" — run cleanup_profiles.py --analyze to find Federation duplicates, then --execute to remove them.

---

Version Summary

Script	Version	Last Major Change
aria_mapping.py	1.6.0	Added --servicenow-tags for dynamic dropdown filtering
mapper.py	5.1	Added exclude_account_ids, overlay_only/vlan_only, authoritative segment_prefix
cleanup_profiles.py	2.0	Federation-aware keeper selection with tag + account preference

---

Contact

Original author: Noah Farshad (noah@essential.coach) Engagement: VMware / Aria Automation reference implementation