Mapping vendor fields to the canonical tower schema

Every tower vendor exports the same underlying facts under a different set of column names, units, and casing conventions, and none of them agree with each other or with the Canonical Field Dictionary this page belongs to. One inspection contractor ships "HGT (m)", another ships "Tower Height", a legacy structural database ships "structure_elev_ft" — and all three describe the same field, tower_height_ft, at two different units. If a pipeline maps these by direct key lookup, any vendor whose column header doesn’t match exactly is silently dropped or, worse, silently miscoerced when a metric height slides straight into a feet-typed column. This page gives you the concrete, runnable recipe for closing that gap: build an alias table that maps every known vendor spelling onto its canonical field, resolve incoming keys against it case-insensitively, convert units before anything is compared to a bound, coerce types, and reject — never guess at — anything left unresolved. The record this recipe produces is the same shape the Telecom Compliance Data Reference expects every downstream consumer to receive, and it draws its raw input from vendor exports and from structured data recovered by Automated Structural Report Parsing & Document Ingestion, which turns scanned inspection PDFs into exactly this kind of messy, inconsistently-keyed dictionary.

Prerequisites & Context #

This recipe runs on Python 3.10 or newer and uses nothing outside the standard library — no pandas, no jsonschema, just re, json, hashlib, and logging. That matters operationally: a vendor-field mapper sits at the very front of the pipeline, often inside a lightweight ingestion worker, and pulling in a heavy dependency there just to alias a dozen column names is a cost the rest of the system shouldn’t pay.

Before you write a single alias, three things need to be settled:

• The canonical field list. tower_height_ft, fcc_asr_number, structure_type, design_capacity_kg, and site_id are defined once, with their types and units, in the Canonical Field Dictionary. This page does not invent field semantics; it only routes vendor spellings onto that existing vocabulary.
• A known set of source vendors. You cannot alias a column you have never seen. In practice the alias table grows every time a new inspection contractor or leasing vendor is onboarded, and it is reviewed, not auto-generated — a wrong alias silently corrupts every record that vendor ever sends.
• A downstream identifier contract. Once a record is mapped, its fcc_asr_number field is expected to already be a plausible seven-digit FCC ASR string; the deeper question of whether that string is actually a valid, registered ASR number belongs to Validating FCC ASR numbers in Python, which runs immediately after this mapping step and treats a syntactically well-formed but unverified number as its own input.

With the canonical vocabulary and the vendor set both fixed, mapping a single incoming record becomes a five-step, deterministic operation — no fuzzy matching, no heuristics, no silent defaults.

Figure: a vendor record resolves through aliasing and unit conversion, then either raises or seals.

Step-by-Step Implementation #

Step 1 — Build an alias table. For every canonical field, list every spelling a real vendor has ever sent, paired with the unit that spelling implies. "HGT (m)" and "height_m" both resolve to tower_height_ft but carry a metres tag; "Tower Height" and "structure_elev_ft" resolve to the same field already in feet.

ALIAS_TABLE = {
    "hgtm": ("tower_height_ft", "m"),
    "heightm": ("tower_height_ft", "m"),
    "towerheight": ("tower_height_ft", "ft"),
    "structureelevft": ("tower_height_ft", "ft"),
}

Step 2 — Resolve each incoming key case-insensitively. Normalize both the alias table’s keys and the incoming record’s keys to the same comparison form — lowercased, punctuation and whitespace stripped — before looking anything up. "HGT (m)", "hgt(m)", and " Hgt_M " must all resolve to the identical alias.

def normalize_key(raw_key: str) -> str:
    return re.sub(r"[^a-z0-9]", "", raw_key.lower())

Step 3 — Convert units before anything else touches the value. A resolved field carrying a "m" tag gets multiplied by the metres-to-feet constant; a field already tagged "ft" passes through unchanged. This has to happen before type coercion or bound checks, or a 42.7-metre tower reads as a 42.7-foot one.

METERS_TO_FEET = 3.28084

def convert_unit(value: float, unit: str) -> float:
    return round(value * METERS_TO_FEET, 2) if unit == "m" else value

Step 4 — Coerce types and validate required fields. Every canonical field has a declared Python type; cast to it explicitly rather than trusting the vendor’s serialization, and confirm every field the canonical schema marks required is present once resolution finishes.

FIELD_TYPES = {"site_id": str, "tower_height_ft": float,
               "fcc_asr_number": str, "structure_type": str}
REQUIRED_FIELDS = ("site_id", "tower_height_ft", "fcc_asr_number", "structure_type")

Step 5 — Seal the canonical record with hashlib.sha256. Once a record clears aliasing, unit conversion, and type coercion, hash its sorted-key serialization so any later mutation — accidental or adversarial — is detectable against the sealed digest.

def seal(record: dict) -> str:
    canonical = json.dumps(record, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(canonical.encode("utf-8")).hexdigest()

Any incoming key that survives normalization without matching an alias, and is not already spelled as a canonical field name, is a field the pipeline has never been told how to interpret — the mapper rejects the whole record rather than guessing, exactly the way an unconvertible unit or an uncoercible type rejects it.

Complete Runnable Example #

The block below is self-contained, runs on the standard library alone, and mirrors the five steps against one realistic mixed-vendor record for site TWR-8842: a Site ID field, a height reported in metres under "HGT (m)", an FCC ASR number under the bare alias "ASR", and a structure type under "Structure Type".

import json
import hashlib
import logging
import re

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
log = logging.getLogger("vendor_mapper")

METERS_TO_FEET = 3.28084

ALIAS_TABLE = {
    "siteid": ("site_id", None),
    "towerid": ("site_id", None),
    "hgtm": ("tower_height_ft", "m"),
    "towerheight": ("tower_height_ft", "ft"),
    "structureelevft": ("tower_height_ft", "ft"),
    "asr": ("fcc_asr_number", None),
    "fccregistration": ("fcc_asr_number", None),
    "structuretype": ("structure_type", None),
}
FIELD_TYPES = {"site_id": str, "tower_height_ft": float,
               "fcc_asr_number": str, "structure_type": str}
REQUIRED_FIELDS = ("site_id", "tower_height_ft", "fcc_asr_number", "structure_type")

class FieldMappingError(Exception):
    """Raised when a vendor key, unit, or type cannot be resolved."""

def normalize_key(raw_key: str) -> str:
    return re.sub(r"[^a-z0-9]", "", raw_key.lower())

def map_vendor_record(raw: dict) -> tuple[dict, str]:
    canonical, unresolved = {}, []
    for raw_key, raw_value in raw.items():
        match = ALIAS_TABLE.get(normalize_key(raw_key))
        if match is None:
            unresolved.append(raw_key)
            continue
        field, unit = match
        value = raw_value
        if unit == "m":
            value = round(float(value) * METERS_TO_FEET, 2)
        canonical[field] = value
    if unresolved:
        raise FieldMappingError(f"unresolved vendor field(s): {unresolved}")
    for field, expected_type in FIELD_TYPES.items():
        if field in canonical:
            try:
                canonical[field] = expected_type(canonical[field])
            except (TypeError, ValueError) as exc:
                raise FieldMappingError(f"{field} cannot coerce to {expected_type}: {exc}")
    missing = [f for f in REQUIRED_FIELDS if f not in canonical]
    if missing:
        raise FieldMappingError(f"missing required field(s): {missing}")
    digest = hashlib.sha256(
        json.dumps(canonical, sort_keys=True, separators=(",", ":")).encode("utf-8")
    ).hexdigest()
    log.info("mapped %s audit=%s", canonical["site_id"], digest[:16])
    return canonical, digest

if __name__ == "__main__":
    vendor_record = {
        "Site ID": "TWR-8842",
        "HGT (m)": "42.7",
        "ASR": "1290123",
        "Structure Type": "Monopole",
    }
    try:
        record, audit_id = map_vendor_record(vendor_record)
        print(json.dumps({"record": record, "audit_id": audit_id[:16]}, indent=2))
    except FieldMappingError as err:
        log.error("mapping rejected: %s", err)

Verification & Expected Output #

Save the block as vendor_map.py and run python3 vendor_map.py. With the sample above, "HGT (m)": "42.7" resolves through hgtm to tower_height_ft, converts 42.7 * 3.28084 to 140.09, and the full output is:

INFO mapped TWR-8842 audit=<16-char hex prefix>
{
  "record": {
    "site_id": "TWR-8842",
    "tower_height_ft": 140.09,
    "fcc_asr_number": "1290123",
    "structure_type": "Monopole"
  },
  "audit_id": "<16-char hex prefix>"
}

The digest is deterministic for these exact field values because it hashes the sorted-key JSON serialization, not the arrival order of the vendor’s original columns — re-run it and the same four fields produce the same sixteen-character prefix every time. To see the rejection path, rename "Structure Type" to "Struct Class" — a spelling not in ALIAS_TABLE — and the run instead logs an error and prints nothing, because map_vendor_record raises FieldMappingError: unresolved vendor field(s): ['Struct Class'] before any hash is computed. Feed in "HGT (m)": "not-a-number" and the unit-conversion line raises a plain ValueError inside the float() call, which is intentional — it surfaces immediately rather than being caught and coerced into a default.

Gotchas & Edge Cases #

• A vendor reusing an alias for a different field. If two vendors both ship a column literally named "Height" but one means feet and the other means the mounting bracket’s height above the platform, a single shared alias silently merges two different quantities. Keep the alias table keyed per-vendor-source where that ambiguity exists, rather than assuming one alias is safe to reuse globally, even when the normalized spelling matches exactly.
• Numeric strings with embedded units or thousands separators. Vendor exports routinely ship "140 ft" or "1,290,123" where the canonical schema expects a bare float or a plain digit string. float("140 ft") raises rather than truncating, which is the correct failure — but it means the alias table’s unit tag alone isn’t enough; incoming values need a strip pass for units and separators before conversion, or every such record rejects on a formatting artifact instead of a real data problem.
• Silent precision loss compounding across resellers. A height reported to one decimal in metres and converted to feet, then re-exported by a reseller and converted back to metres, accumulates rounding error with every hop. Store tower_height_ft from the first conversion as the canonical value and never re-derive it from a re-converted downstream copy, so the audit hash reflects one authoritative conversion rather than a chain of them.

FAQ #

Related #

• Parent topic: Canonical Field Dictionary
• Sibling guide: Validating FCC ASR numbers in Python
• Section overview: Telecom Compliance Data Reference
• Data source: Automated Structural Report Parsing & Document Ingestion