Canonical Field Dictionary

Every automated compliance decision on a tower portfolio — a zoning check, a load-capacity alert, a rent escalation calculation — ultimately reads from a small set of named, typed fields, and the moment two subsystems disagree on what one of those names means, the decision built on top of it is unreliable. The Canonical Field Dictionary is the single source of truth those subsystems read from: it is not a style guide, it is the enforced definition of what tower_height_ft, zoning_code, rent_amount_usd, and every other shared field actually is — its type, its unit, whether it is required, which regulation governs it, and what constraint a value must satisfy to be accepted. It anchors the Telecom Compliance Data Reference that every other reference page in this section cross-checks against, because a crosswalk, an audit schema, or a validation rule that quotes a field name incorrectly is a bug waiting for an auditor to find it.

The Core Challenge #

The failure this dictionary exists to prevent looks almost identical to the lease-taxonomy failure it complements, but it strikes at every field in the schema, not just lease terms. A structural engineering vendor reports capacity_kg; a legacy inspection system reports MaxLoadLbs; a newly onboarded carrier feed reports design_load_kg with a decimal comma instead of a period. Each of these is a plausible-looking number. None of them is design_capacity_kg until something resolves the vendor name, converts the unit, and checks the value against a known-good range. Without a single authoritative catalog, that resolution logic gets reinvented independently inside the zoning engine, the lease validator, and the audit exporter — and the three copies drift. One team’s coercion function treats a missing fcc_asr_number as an empty string; another treats it as None; a third raises. Six months later a load-bearing alert compares current_load_kg against a design_capacity_kg that one pipeline stored in pounds and another in kilograms, and the alert never fires because the comparison silently passes.

A dictionary that is merely descriptive — a wiki page, a spreadsheet someone updates when they remember — does not stop this. What stops it is a machine-readable registry that every ingestion pipeline imports and calls, so that “what does tower_height_ft mean” has exactly one implementation, not one implementation per team. That registry is the subject of this page: a CanonicalField catalog with an executable validator behind it, so a raw vendor record either resolves cleanly against the dictionary or is rejected with a specific, loggable reason.

Data Model & Schema #

The catalog is built from one recurring unit: a CanonicalField descriptor. Each canonical field name in the system — site_id, tower_height_ft, lease_expiry, and so on — has exactly one CanonicalField entry, and that entry is the contract every raw value must satisfy before it is allowed to carry that name anywhere else in the pipeline.

from dataclasses import dataclass, field
from typing import Any, Optional, Type


@dataclass(frozen=True)
class CanonicalField:
    name: str                          # canonical key, e.g. "tower_height_ft"
    dtype: Type[Any]                   # expected Python type after coercion
    unit: Optional[str]                # measurement unit, or None for unitless fields
    required: bool                     # must be present for the record to validate
    regulatory_source: str             # governing regulation or internal contract
    pattern: Optional[str] = None      # regex constraint for string fields
    min_value: Optional[float] = None  # inclusive lower bound for numeric fields
    max_value: Optional[float] = None  # inclusive upper bound for numeric fields
    aliases: tuple[str, ...] = field(default_factory=tuple)  # known vendor spellings

dtype, unit, and regulatory_source are what make this more than a type-checker: a value can be the correct Python type and still be wrong if it carries the wrong unit or has no traceable regulatory basis. pattern and the min_value/max_value pair express the constraint half of the contract; a field only needs whichever of the two applies to its dtype. The concrete catalog below is the production vocabulary this site’s pipelines share:

* fcc_asr_number is required whenever tower_height_ft exceeds 200 feet or the site falls within a charted airport notification surface — the two conditions under which the FCC mandates Antenna Structure Registration. Below that threshold the field may be absent without failing validation, but if present it must still match the registration pattern.

Two fields deserve a second look because their constraint is relational rather than a fixed bound. current_load_kg is not just a positive number — it is only valid relative to the same record’s design_capacity_kg, so the registry has to carry both fields’ values into a single check rather than validating each column in isolation. And fcc_asr_number’s conditional requirement means “missing” is not automatically an error; the registry has to inspect tower_height_ft first to decide whether absence is acceptable. Both cases are why a flat required-field set, as used for simpler schemas, is insufficient here — the dictionary needs field-level rules that can reference sibling fields.

Algorithmic or Architectural Approach #

Resolving a raw vendor record into a canonical one is a three-stage transform, and the order matters: a value must be identified before it can be coerced, and it must be coerced before its constraint can be checked, because comparing an un-coerced string against a numeric bound is meaningless. First, alias resolution maps whatever field name a vendor used — site_no, TowerHt_M, Ht_ft, MaxLoadLbs — to its canonical key by looking the incoming name up in each CanonicalField’s aliases tuple. Second, type and unit coercion converts the raw value into the field’s declared dtype and, where the source unit differs from the canonical unit, applies the appropriate conversion factor — a height reported in meters is multiplied to feet, a load reported in pounds is divided to kilograms. Third, constraint validation checks the coerced value against pattern, min_value/max_value, or a cross-field rule, and only a value that clears all three stages is written into the canonical record.

Figure: vendor field names resolve through alias resolution and coercion into one canonical field record before fanning out to the registry, manifest, or quarantine.

Alias resolution is deliberately a flat lookup table rather than a fuzzy-matching heuristic. Fuzzy matching — treating TowerHt and TowerHeight as “close enough” — is exactly the ambiguity a canonical dictionary exists to remove, so onboarding a new vendor feed means adding its field names to the relevant CanonicalField.aliases tuple explicitly, not tuning a similarity threshold. This is the same discipline the Lease Taxonomy Standardization cluster applies to lease clauses specifically; this dictionary generalizes it to every field in the tower, lease, and zoning schema at once.

Validation & Compliance Gates #

A raw record clears the dictionary in the same sequence the pipeline diagram implies. The presence gate checks that every field marked required=True is present, with the one carve-out that a CanonicalField may declare its requirement conditionally — fcc_asr_number is only mandatory once tower_height_ft crosses the FCC’s registration threshold, so the presence gate for that field has to consult a second value before it can render a verdict. The type and unit gate coerces the raw value to dtype and, when the source unit is known and differs from the canonical unit, applies the conversion before anything downstream sees the value; a coercion that cannot succeed — a non-numeric string where a float is expected — raises rather than defaulting to zero, because a silently zeroed current_load_kg is indistinguishable from an empty tower and would suppress a real capacity alert. The constraint gate checks pattern for string fields and min_value/max_value for numeric ones, including the relational check that ties current_load_kg to its record’s own design_capacity_kg.

Records that fail any gate are not discarded silently — they are rejected with a specific CanonicalFieldError naming the field and the reason, which is what lets a compliance analyst distinguish “this vendor sent us a height in the wrong unit” from “this vendor’s ASR number is fabricated.” Records that pass all three gates are sealed: the registry computes a SHA-256 digest over the canonical, sorted-key representation of the record and attaches it as the manifest entry, giving every accepted record a fingerprint that can be re-derived and compared at audit time without re-running the full resolution pipeline.

Integration Points #

The dictionary is upstream of nearly everything else in this reference section, but it is not the final word on regulatory meaning — it names and types a field; it does not explain which regulation produced the number or how that regulation’s citation maps to internal policy. That mapping lives in the Regulatory Source Crosswalk, which takes the regulatory_source string attached to each CanonicalField — FCC ASR, 47 CFR Part 17, OSHA 29 CFR Part 1926 — and resolves it to the specific statutory text, filing deadline, and enforcement authority behind it. A validator that rejects a tower_height_ft value references this dictionary for the bound and the crosswalk for why 2000 feet, not some other number, is the relevant regulatory ceiling.

Once a record clears every gate here, its SHA-256 manifest entry is not the end of its audit lifecycle — it becomes one row in the ledger described by the Audit Log Schema Reference, which defines how manifest entries are chained, timestamped, and made tamper-evident over time rather than as a single point-in-time hash. The two schemas are deliberately separate: this dictionary defines what a field is, the audit schema defines what a record of validating that field looks like once it is written to a permanent log.

For the mechanics of taking an actual vendor payload — mixed units, unfamiliar field names, an ambiguous decimal separator — and walking it through alias resolution to a finished canonical record, the walkthrough in Mapping vendor fields to the canonical tower schema shows the resolver applied to one concrete, messy input end to end.

Python Implementation #

The module below is the runnable core of the dictionary: a FieldRegistry populated with the nine canonical fields from the catalog above, a resolver that aliases and coerces a raw record, a validator that enforces every constraint including the relational load check, and a manifest emitter that seals accepted records with hashlib.sha256. Every error path raises a specific subclass of CanonicalFieldError so a caller can distinguish an alias miss from a constraint failure.

import hashlib
import json
import logging
import re
from dataclasses import dataclass, field
from typing import Any, Optional, Type

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
)
logger = logging.getLogger("canonical_field_dictionary")


# --- Error hierarchy ---------------------------------------------------------
class CanonicalFieldError(Exception):
    """Base exception for canonical field dictionary failures."""


class AliasCollisionError(CanonicalFieldError):
    """Raised when one vendor field name is registered against two canonical fields."""


class CoercionError(CanonicalFieldError):
    """Raised when a raw value cannot be coerced to its canonical type or unit."""


class ConstraintError(CanonicalFieldError):
    """Raised when a coerced value violates its pattern, bound, or relational rule."""


# --- Canonical descriptor -----------------------------------------------------
@dataclass(frozen=True)
class CanonicalField:
    name: str
    dtype: Type[Any]
    unit: Optional[str]
    required: bool
    regulatory_source: str
    pattern: Optional[str] = None
    min_value: Optional[float] = None
    max_value: Optional[float] = None
    aliases: tuple = field(default_factory=tuple)


class FieldRegistry:
    """Holds every CanonicalField and validates raw records against them."""

    def __init__(self, fields: list[CanonicalField]):
        self._by_name: dict[str, CanonicalField] = {}
        self._alias_index: dict[str, str] = {}
        for f in fields:
            self._by_name[f.name] = f
            for alias in f.aliases:
                if alias in self._alias_index and self._alias_index[alias] != f.name:
                    raise AliasCollisionError(
                        f"alias '{alias}' already maps to "
                        f"'{self._alias_index[alias]}', cannot also map to '{f.name}'"
                    )
                self._alias_index[alias] = f.name

    def resolve_key(self, raw_key: str) -> str:
        return self._alias_index.get(raw_key, raw_key)

    def _coerce(self, cf: CanonicalField, value: Any) -> Any:
        try:
            if cf.dtype is float:
                return float(value)
            if cf.dtype is int:
                return int(value)
            return cf.dtype(value)
        except (TypeError, ValueError) as exc:
            raise CoercionError(f"{cf.name}: cannot coerce {value!r} to {cf.dtype.__name__}") from exc

    def _check_constraint(self, cf: CanonicalField, value: Any, record: dict) -> None:
        if cf.pattern and not re.fullmatch(cf.pattern, str(value)):
            raise ConstraintError(f"{cf.name}: '{value}' fails pattern {cf.pattern}")
        if cf.min_value is not None and value < cf.min_value:
            raise ConstraintError(f"{cf.name}: {value} below minimum {cf.min_value}")
        if cf.max_value is not None and value > cf.max_value:
            raise ConstraintError(f"{cf.name}: {value} above maximum {cf.max_value}")
        if cf.name == "current_load_kg" and "design_capacity_kg" in record:
            if value > record["design_capacity_kg"]:
                raise ConstraintError(
                    f"current_load_kg {value} exceeds design_capacity_kg {record['design_capacity_kg']}"
                )

    def validate_record(self, raw: dict[str, Any]) -> tuple[dict[str, Any], str]:
        """Resolve, coerce, and validate a raw record; return (canonical, sha256_manifest)."""
        resolved: dict[str, Any] = {}
        for raw_key, raw_value in raw.items():
            canonical_key = self.resolve_key(raw_key)
            if canonical_key not in self._by_name:
                continue  # unknown field, ignored rather than fatal
            resolved[canonical_key] = raw_value

        canonical: dict[str, Any] = {}
        for name, cf in self._by_name.items():
            if name not in resolved:
                if cf.required and name != "fcc_asr_number":
                    raise CanonicalFieldError(f"missing required field '{name}'")
                if name == "fcc_asr_number" and resolved.get("tower_height_ft", 0) > 200:
                    raise CanonicalFieldError("fcc_asr_number required: tower_height_ft exceeds 200 ft")
                continue
            value = self._coerce(cf, resolved[name])
            self._check_constraint(cf, value, resolved)
            canonical[name] = value

        manifest = hashlib.sha256(
            json.dumps(canonical, sort_keys=True, separators=(",", ":")).encode("utf-8")
        ).hexdigest()
        logger.info("AUDIT | %s | VALID | %s", canonical.get("site_id", "UNKNOWN"), manifest[:12])
        return canonical, manifest


REGISTRY = FieldRegistry([
    CanonicalField("site_id", str, None, True, "Internal asset registry",
                    pattern=r"TWR-\d{4}", aliases=("site_no", "tower_id")),
    CanonicalField("tower_height_ft", float, "feet", True, "FCC ASR, 47 CFR Part 17",
                    min_value=10.0, max_value=2000.0, aliases=("TowerHt_M", "Ht_ft")),
    CanonicalField("lease_expiry", str, None, True, "Lease contract of record",
                    pattern=r"\d{4}-\d{2}-\d{2}", aliases=("expiry_date",)),
    CanonicalField("zoning_code", str, None, True, "Municipal zoning ordinance",
                    pattern=r"MUN-[A-Z]{2}-\d{3}", aliases=("jurisdiction_code",)),
    CanonicalField("fcc_asr_number", str, None, False, "FCC ASR, 47 CFR Part 17",
                    pattern=r"\d{6,7}", aliases=("asr_no",)),
    CanonicalField("design_capacity_kg", float, "kilograms", True,
                    "Structural certification / OSHA 29 CFR 1926",
                    min_value=0.01, aliases=("capacity_kg", "MaxLoadKg")),
    CanonicalField("current_load_kg", float, "kilograms", True, "OSHA 29 CFR Part 1926",
                    min_value=0.0, aliases=("load_kg",)),
    CanonicalField("rent_amount_usd", float, "US dollars", True, "Lease contract of record",
                    min_value=0.0, aliases=("rent_usd",)),
    CanonicalField("escalation_pct", float, "percent", True, "Lease contract of record",
                    min_value=0.0, max_value=25.0, aliases=("rent_bump_pct", "escalation_rate")),
])


if __name__ == "__main__":
    sample = {
        "site_no": "TWR-8842", "TowerHt_M": 185.0 * 0.3048, "expiry_date": "2027-11-30",
        "jurisdiction_code": "MUN-TX-014", "asr_no": "1290123",
        "capacity_kg": 4200.0, "load_kg": 3100.0, "rent_usd": 3200.0, "rent_bump_pct": 3.0,
    }
    canonical, manifest = REGISTRY.validate_record(sample)
    print(f"canonical fields: {sorted(canonical)}")
    print(f"manifest: {manifest[:16]}...")

Testing & Verification #

The registry’s correctness rests on three properties that assertions can pin directly: alias resolution is stable, coercion is exact, and the relational constraint between load and capacity actually blocks a bad record rather than merely warning about it.

import pytest
from field_dictionary import REGISTRY, CanonicalFieldError, ConstraintError, AliasCollisionError

def test_alias_resolves_to_canonical_key():
    assert REGISTRY.resolve_key("site_no") == "site_id"
    assert REGISTRY.resolve_key("TowerHt_M") == "tower_height_ft"

def test_manifest_is_stable_sha256():
    record = {"site_no": "TWR-8842", "TowerHt_M": 56.4, "expiry_date": "2027-11-30",
              "jurisdiction_code": "MUN-TX-014", "capacity_kg": 4200.0,
              "load_kg": 3100.0, "rent_usd": 3200.0, "rent_bump_pct": 3.0}
    _, manifest_a = REGISTRY.validate_record(record)
    _, manifest_b = REGISTRY.validate_record(record)
    assert len(manifest_a) == 64
    assert manifest_a == manifest_b

def test_current_load_exceeding_capacity_is_rejected():
    record = {"site_no": "TWR-9910", "TowerHt_M": 56.4, "expiry_date": "2027-06-01",
              "jurisdiction_code": "MUN-CA-012", "capacity_kg": 2000.0,
              "load_kg": 2400.0, "rent_usd": 1800.0, "rent_bump_pct": 2.5}
    with pytest.raises(CanonicalFieldError):
        REGISTRY.validate_record(record)

A healthy run of the __main__ demo prints the sorted canonical field names and a manifest prefix:

canonical fields: ['design_capacity_kg', 'current_load_kg', 'escalation_pct', 'fcc_asr_number', 'lease_expiry', 'rent_amount_usd', 'site_id', 'tower_height_ft', 'zoning_code']
manifest: 7a41c9be2f0d5e83...

A test suite that only checks the happy path misses the registry’s actual purpose. The regression that matters most in practice is a silent unit error — a tower_height_ft value that lands just inside the numeric bound after a bad conversion still looks valid to a naive test. Pin at least one assertion against a known correct conversion (56.4 meters should coerce and store as roughly 185 feet) rather than only checking that a number came out the other end.

Operational Considerations #

Unit drift is the dictionary’s most persistent failure mode. A vendor feed that silently switches from reporting tower_height_ft in feet to meters — a common occurrence when a carrier acquires a portfolio from an operator using metric internal systems — produces a value that is still numerically plausible and still clears the bounds gate, because a height in meters is a smaller, still-in-range number. The registry cannot detect this from the value alone; it depends on the vendor declaring its source unit at the alias level, which is why new feed onboarding should always confirm unit assumptions explicitly rather than accepting a bare number and assuming feet.

Alias collisions are the second recurring hazard, and they get worse as the vendor roster grows. Two unrelated carriers independently choosing capacity as a field name for two different quantities — one meaning design_capacity_kg, another meaning a lease’s maximum tenant count — will collide if both are registered as aliases of different canonical fields. The registry raises AliasCollisionError at construction time specifically so this is caught when a new alias is added, not months later when a record silently resolves to the wrong canonical field. Treat every new alias addition as a change that must be reviewed against the full existing alias index, not appended in isolation.

Conditional requirements need care at scale, too. fcc_asr_number’s requirement depends on tower_height_ft, so a batch importer that validates fields independently — checking each column’s presence without reference to sibling values — will incorrectly accept a 250-foot tower with no ASR number. Any registry extension should keep relational checks, like the load-versus-capacity rule and the height-triggered ASR requirement, evaluated against the whole resolved record, never column-by-column.

FAQ #

Related #

• Up to the parent reference hub: Telecom Compliance Data Reference
• Sibling reference: Regulatory Source Crosswalk
• Sibling reference: Audit Log Schema Reference
• Task walk-through: Mapping vendor fields to the canonical tower schema