Clinical Data

Handling US Core Must-Support Elements in Clinical ETL

In production clinical ETL, the single most misread word in the US Core Implementation Guide is mustSupport. Engineers routinely conflate it with required cardinality and then either fabricate placeholder values to “pass validation” or drop the element entirely — both of which corrupt longitudinal records and create compliance exposure. This page is the runnable companion to the US Core Implementation Guide deep dive: it pins down what mustSupport actually obligates a producer and consumer to do, and shows how a mapper populates the element from source, records genuine absence with a Data Absent Reason extension, and never invents data.

What mustSupport actually means (quick reference)

In FHIR R4 (4.0.1) and US Core 7.0.0, mustSupport is a flag on an element definition — the little S in the IG’s differential tables. It is a two-sided obligation and, critically, it is not the same as a minimum cardinality of 1. The distinction is the entire subject of this page:

  • Producer obligation. If the source system holds the data for a mustSupport element, the producer must populate it in the resource it emits. Silently dropping data you actually have is a conformance failure.
  • Consumer obligation. A consumer must not error out or discard the resource when a mustSupport element is present; it must be able to process, store, and (where relevant) display it.
  • Cardinality is orthogonal. An element can be mustSupport and still 0..1 or 0..*. mustSupport says “handle this when it exists,” while min = 1 says “this element must be present for the resource to be valid at all.” A required (1..1) element that is also mustSupport must always appear; a mustSupport element with min = 0 appears only when the source has it.

The consequence for ETL is the hard case: an element that is required (min ≥ 1) but whose value genuinely does not exist in the source. US Core does not let you omit a required element, and it forbids fabricating a value. The sanctioned escape is the Data Absent Reason extension (http://hl7.org/fhir/StructureDefinition/data-absent-reason) — the FHIR R4 equivalent of an HL7 v2 nullFlavor — attached in place of the missing primitive.

Element status in the profile Source data present? Correct ETL action
Required 1..1 (with or without S) Yes Populate the value directly
Required 1..1 mustSupport No Attach data-absent-reason (unknown/masked/asked-unknown) — never omit, never fabricate
mustSupport 0..1 / 0..* Yes Populate from source (producer obligation)
mustSupport 0..1 / 0..* No Omit the element entirely — absence is valid
Optional (no S), 0..* Either Populate if convenient; omission carries no obligation

The two rows that trip up mappers are the second and fourth. Both describe missing source data, but one demands an explicit data-absent-reason (because the element is required) and the other demands plain omission (because it is optional). Getting this wrong in either direction — omitting a required element or padding an optional one with a fake unknown — is the defect this pattern eliminates.

Decision flow for populating a US Core must-support element in ETL A decision diagram for one must-support element. First the mapper asks whether the source has a value; if yes, the element is populated from source and marked satisfied. If no, it asks whether the element is required by profile cardinality; if required, a data-absent-reason extension is attached with a reason code of unknown, masked, or asked-unknown; if not required, the element is omitted. No branch ever fabricates a value. Per must-support element · populate, mark absent, or omit — never invent Source has a value? Populate from source producer obligation met Required min ≥ 1? data-absent-reason unknown / masked / asked-unknown Omit element (valid) yes no yes no
Every must-support element resolves to exactly one of three outcomes; fabrication is never one of them.

Implementation pattern

The worker below takes a mapped resource (already shaped into a FHIR R4 resource, pre-conformance) and a table of mustSupport paths with their profile cardinality. For each path it reads the source-derived value; if present it populates the element, and if absent it either omits the element (optional) or injects a data-absent-reason extension (required). It returns a conformance report the pipeline can log and route on. Nothing in the function invents a clinical value.

from dataclasses import dataclass, field
from typing import Any, Callable, Optional

DAR_URL = "http://hl7.org/fhir/StructureDefinition/data-absent-reason"

# Reason codes from the FHIR R4 data-absent-reason value set.
# 'unknown'       -> value exists somewhere but is not known here
# 'asked-unknown' -> source was asked and could not provide it
# 'masked'        -> value withheld for privacy/security (PHI redaction)
VALID_DAR_CODES = {"unknown", "asked-unknown", "masked"}


@dataclass
class MustSupportSpec:
    """One mustSupport element: dotted path, whether the profile requires it,
    and the reason code to record if a required value is genuinely absent."""
    path: str
    required: bool                 # True when profile min >= 1
    absent_reason: str = "unknown" # used only when required and source is empty


@dataclass
class ConformanceReport:
    populated: list = field(default_factory=list)
    marked_absent: list = field(default_factory=list)
    omitted: list = field(default_factory=list)
    violations: list = field(default_factory=list)

    @property
    def conformant(self) -> bool:
        return not self.violations


def _set_path(resource: dict, path: str, value: Any) -> None:
    """Assign a value at a simple dotted path, creating intermediate dicts."""
    parts = path.split(".")
    node = resource
    for key in parts[:-1]:
        node = node.setdefault(key, {})
    node[parts[-1]] = value


def _mark_data_absent(resource: dict, path: str, code: str) -> None:
    """Attach a data-absent-reason extension on the element's sibling
    '_<name>' primitive-extension slot, per FHIR R4 JSON representation."""
    if code not in VALID_DAR_CODES:
        raise ValueError(f"invalid data-absent-reason code: {code!r}")
    parts = path.split(".")
    node = resource
    for key in parts[:-1]:
        node = node.setdefault(key, {})
    # Primitive extensions live on the '_'-prefixed sibling property.
    node[f"_{parts[-1]}"] = {
        "extension": [{"url": DAR_URL, "valueCode": code}]
    }


def apply_must_support(
    resource: dict,
    specs: list[MustSupportSpec],
    source_lookup: Callable[[str], Optional[Any]],
) -> ConformanceReport:
    """Reconcile each mustSupport element against the source.

    source_lookup(path) returns the mapped source value or None if the
    source genuinely has no data for that element. This function NEVER
    substitutes a default clinical value for a missing one.
    """
    report = ConformanceReport()

    for spec in specs:
        value = source_lookup(spec.path)

        if value is not None:
            # Producer obligation: we have the data, so we populate it.
            _set_path(resource, spec.path, value)
            report.populated.append(spec.path)
            continue

        # Source is genuinely empty from here on.
        if spec.required:
            # Required element with no value -> data-absent-reason, not a guess.
            _mark_data_absent(resource, spec.path, spec.absent_reason)
            report.marked_absent.append((spec.path, spec.absent_reason))
        else:
            # Optional mustSupport with no data -> omission is conformant.
            report.omitted.append(spec.path)

    return report

The design turns on one rule enforced structurally: source_lookup may return a real value or None, and only those two cases exist. There is no code path that yields a fabricated string, a 0, or a placeholder date. When the value is None and the element is required, the code attaches a data-absent-reason on the FHIR R4 primitive-extension slot (the _-prefixed sibling of the element, e.g. _birthDate), which is how absence is expressed for a primitive while keeping the resource valid. When the element is optional, absence is simply omission — the fourth row of the reference table.

A minimal driver shows the three outcomes on a single Patient, where birthDate is required-but-unknown, telecom is a genuinely masked contact, and an absent optional element is dropped:

patient = {"resourceType": "Patient", "id": "example"}

specs = [
    MustSupportSpec("gender", required=True),
    MustSupportSpec("birthDate", required=True, absent_reason="unknown"),
    MustSupportSpec("telecom", required=False, absent_reason="masked"),
]

source = {"gender": "female"}  # birthDate + telecom absent in source

report = apply_must_support(
    patient, specs, source_lookup=lambda p: source.get(p)
)

# gender populated; birthDate -> data-absent-reason 'unknown'; telecom omitted
assert patient["gender"] == "female"
assert patient["_birthDate"]["extension"][0]["valueCode"] == "unknown"
assert "telecom" not in patient and "_telecom" not in patient

For masking Protected Health Information deliberately — for example suppressing a contact number for a shielded patient — the same mechanism carries the masked code, so the resource stays valid while signalling why the value is missing rather than that it never existed.

Validation and testing

The property worth locking down in CI is the one the whole page exists to protect: an absent required mustSupport element must yield a data-absent-reason, never a fabricated value. The parametrized test below asserts each of the three outcomes and, crucially, asserts the negative — that no invented value leaked into the resource.

import pytest


def test_absent_required_gets_data_absent_reason():
    resource = {"resourceType": "Patient"}
    specs = [MustSupportSpec("birthDate", required=True, absent_reason="unknown")]
    report = apply_must_support(resource, specs, lambda p: None)

    # birthDate must NOT be fabricated with any placeholder value.
    assert "birthDate" not in resource
    # It must carry a data-absent-reason extension instead.
    dar = resource["_birthDate"]["extension"][0]
    assert dar["url"] == DAR_URL
    assert dar["valueCode"] == "unknown"
    assert report.marked_absent == [("birthDate", "unknown")]


def test_present_value_is_populated_not_masked():
    resource = {"resourceType": "Patient"}
    specs = [MustSupportSpec("gender", required=True)]
    report = apply_must_support(resource, specs, lambda p: "male")
    assert resource["gender"] == "male"
    assert "_gender" not in resource          # no spurious extension
    assert report.populated == ["gender"]


def test_absent_optional_is_omitted():
    resource = {"resourceType": "Patient"}
    specs = [MustSupportSpec("telecom", required=False)]
    report = apply_must_support(resource, specs, lambda p: None)
    assert "telecom" not in resource and "_telecom" not in resource
    assert report.omitted == ["telecom"]


@pytest.mark.parametrize("bad_code", ["", "n/a", "redacted", "null"])
def test_invalid_reason_code_rejected(bad_code):
    resource = {"resourceType": "Patient"}
    specs = [MustSupportSpec("birthDate", required=True, absent_reason=bad_code)]
    with pytest.raises(ValueError):
        apply_must_support(resource, specs, lambda p: None)

Run these fixtures against every US Core version you support so a future IG that promotes an element to mustSupport, or tightens a cardinality, fails CI rather than the live pipeline. After this in-process check, run the resource through the official validator gate described in validating FHIR resources against US Core profiles — the two layers are complementary, not redundant: this one guarantees you never fabricated, and that one confirms profile conformance end to end.

Gotchas and compliance constraints

  • mustSupport is not required — and required is not mustSupport. Treating every S as min = 1 produces resources stuffed with data-absent-reason extensions on optional elements, which is itself non-conformant noise. Read cardinality and the support flag as two independent facts; only the required-and-absent intersection warrants a data-absent-reason, and only present source data satisfies the producer obligation.

  • Masking PHI is a first-class reason code, not an omission. When you deliberately withhold a value for privacy — a sensitive telecom, an address for a protected patient — use data-absent-reason code masked rather than dropping the element. Silent omission is indistinguishable from “the source never had it,” which destroys the audit signal a downstream consumer or compliance reviewer needs. This is the same nullFlavor-to-DAR discipline covered in handling nullFlavor in FHIR resource extraction.

  • Fabricating data is a data-integrity and compliance violation, not a shortcut. Injecting a default 1900-01-01 birth date or a fake unknown@example.com to clear a validator is falsification of a clinical record. It corrupts patient matching, skews analytics, and can breach the HIPAA requirement that records be accurate; a data-absent-reason is the only sanctioned way to represent “we do not have this.”