Clinical Data

Date-Shifting Clinical Timestamps for De-identification

HIPAA Safe Harbor strips every date element more specific than a year, but almost every clinical research question depends on relative timing: days since admission, the interval between a diagnosis and its treatment, the gap between two lab draws. Collapsing dates to a bare year destroys that signal, which is why limited datasets and Expert-Determination releases keep full timestamps and instead apply a consistent per-patient date shift — a single offset that slides every event for one patient by the same random amount, preserving intervals while obscuring absolute dates. This page is the runnable companion to Safe Harbor vs. Expert Determination: where that page compares the two de-identification standards, here we implement the date-shifting pattern that lets an Expert-Determination dataset retain analytic timing without exposing calendar dates.

The core invariant is simple to state and easy to break: if patient P’s admission moves by +217 days, then every timestamp belonging to P — labs, meds, notes, discharge — must move by exactly +217 days. Shift each record independently and you have scrambled the intervals that made the dataset worth keeping. The offset must therefore be a deterministic function of the patient, not of the row.

Date element treatment: Safe Harbor vs. shifted

The lookup below is the decision table to keep next to your de-identification job. Safe Harbor (§164.514(b)(2)) removes all elements of dates directly related to an individual except the year, and requires ages 90 and over to be aggregated. A shifting strategy under Expert Determination keeps the full datetime but slides it by the patient offset, so intervals survive and the absolute calendar is broken.

Date element Safe Harbor treatment Consistent-shift treatment
Admission / discharge datetime Drop to year only (2024) Full datetime shifted by patient offset
Service / encounter date Drop to year only Full date shifted by patient offset
Lab collection / result timestamp Drop to year only Full timestamp shifted by patient offset
Date of birth Drop to year; derive age Shift, or retain only year plus banded age
Date of death Drop to year only Full date shifted by same patient offset
Age in years (< 90) Permitted as-is Permitted as-is
Age ≥ 90 Aggregate into a single 90+ band Aggregate into a single 90+ band
Inter-event interval (days) Not directly emitted; recompute if needed Preserved exactly (shift cancels)

The final row is the whole point: because both endpoints of an interval move by the same offset, the difference between them is unchanged. t2_shifted - t1_shifted == t2 - t1. The shift is invisible to any calculation that only cares about elapsed time.

Consistent per-patient date shifting preserves inter-event intervals Two parallel timelines for one patient. The upper timeline shows original events — admission, a lab draw eight days later, and discharge twelve days after admission. The lower timeline shows the same three events after adding a constant patient offset of plus 217 days: all events slide right by the same amount, so the eight-day and twelve-day gaps between them are identical on both timelines. One patient, offset = +217 days · gaps unchanged Original admit lab d/c 2024-03-01 2024-03-09 2024-03-13 ← 8 days → ← 4 d → + 217 days (constant) Shifted admit lab d/c 2024-10-04 2024-10-12 2024-10-16 ← 8 days → ← 4 d →
Adding a constant per-patient offset slides every event equally, so all inter-event gaps are preserved exactly.

Implementation pattern

The offset must be reproducible across pipeline runs — a nightly incremental load has to shift a patient’s new labs by the same amount it shifted last month’s admission, or the intervals break across load boundaries. Drawing a fresh random number per run fails this. The robust pattern derives the offset deterministically from the patient’s enterprise identifier via a keyed HMAC: same patient plus same secret key always yields the same offset, while the key stays in a vault so the mapping cannot be reproduced by an attacker who only has the released data.

import hmac
import hashlib
from datetime import datetime, timedelta, timezone

# The HMAC key is the de-identification secret. It lives in a KMS / vault,
# NEVER in the released dataset and never in source control. Rotating it
# re-shuffles every offset, so treat it as a stable per-release secret.
SHIFT_KEY = b"__from_vault__"           # e.g. boto3 KMS / HashiCorp Vault fetch
SHIFT_RANGE_DAYS = 365                    # offsets drawn from [-365, +365] days


def patient_offset(enterprise_id: str, key: bytes = SHIFT_KEY) -> timedelta:
    """Deterministic per-patient day offset in [-SHIFT_RANGE_DAYS, +SHIFT_RANGE_DAYS].

    Same enterprise_id + same key => identical offset on every run, so all of
    a patient's timestamps shift together and intervals are preserved. Different
    patients get independent offsets because the HMAC decorrelates the inputs.
    """
    digest = hmac.new(key, enterprise_id.encode("utf-8"), hashlib.sha256).digest()
    # Map the first 8 bytes to a signed day offset spanning the full range.
    raw = int.from_bytes(digest[:8], "big")
    span = 2 * SHIFT_RANGE_DAYS + 1                 # inclusive of both endpoints
    days = (raw % span) - SHIFT_RANGE_DAYS
    return timedelta(days=days)

With the offset function in place, applying the shift is a matter of normalizing each timestamp to a single timezone (UTC), then adding the patient’s offset to every datetime field on the record. Ages are handled separately: they are not dates, so shifting does not apply, but the §164.514(b)(2)(i) rule that ages 90 and over be banded still holds.

DATETIME_FIELDS = ("admit_dt", "discharge_dt", "collected_dt", "birth_dt", "death_dt")


def shift_record(record: dict, enterprise_id: str) -> dict:
    """Return a de-identified copy: every datetime shifted by the patient offset,
    age >= 90 collapsed to the '90+' band. The offset itself is never emitted."""
    offset = patient_offset(enterprise_id)
    out = dict(record)

    for field in DATETIME_FIELDS:
        value = record.get(field)
        if value is None:
            continue
        # Normalize to UTC FIRST so a DST boundary never adds/removes an hour
        # that would corrupt a same-day interval after shifting.
        if value.tzinfo is None:
            raise ValueError(f"{field} is timezone-naive; normalize before shifting")
        value = value.astimezone(timezone.utc)
        out[field] = value + offset

    # HIPAA: ages >= 90 must be aggregated into a single band.
    age = record.get("age_years")
    if age is not None:
        out["age_years"] = "90+" if age >= 90 else age

    # The offset is re-identifying. It goes to the crosswalk vault, not the row.
    _write_crosswalk(enterprise_id, offset)
    return out


def _write_crosswalk(enterprise_id: str, offset: timedelta) -> None:
    """Persist enterprise_id -> offset in the access-controlled crosswalk vault,
    stored separately from the released dataset (see the anonymization guide)."""
    ...  # append to the encrypted, restricted-access re-identification table

Two design choices carry the compliance weight. First, shift_record returns a copy and never writes the offset onto the output row — the offset is the secret that reverses the de-identification, so it belongs only in the crosswalk vault. Second, timezone normalization happens before the arithmetic; adding a timedelta(days=n) to a timezone-aware UTC datetime is DST-safe, whereas shifting a naive local timestamp across a spring-forward boundary can silently move a same-day interval by an hour.

Validation and testing

The pattern has exactly two properties worth asserting, and both are cheap to test: intervals must survive the shift, and two distinct patients must get distinct offsets. A regression test that pins these will catch the classic mistake of re-drawing the offset per record.

from datetime import datetime, timezone


def test_intervals_are_preserved():
    """Shifting must not change any inter-event interval for one patient."""
    rec = {
        "admit_dt": datetime(2024, 3, 1, 8, 0, tzinfo=timezone.utc),
        "collected_dt": datetime(2024, 3, 9, 8, 0, tzinfo=timezone.utc),
        "discharge_dt": datetime(2024, 3, 13, 8, 0, tzinfo=timezone.utc),
        "age_years": 54,
    }
    shifted = shift_record(rec, enterprise_id="EID-000123")

    original_gap = rec["collected_dt"] - rec["admit_dt"]
    shifted_gap = shifted["collected_dt"] - shifted["admit_dt"]
    assert shifted_gap == original_gap            # 8 days, unchanged
    los_before = rec["discharge_dt"] - rec["admit_dt"]
    los_after = shifted["discharge_dt"] - shifted["admit_dt"]
    assert los_after == los_before                # length of stay unchanged


def test_two_patients_get_different_offsets():
    """Distinct enterprise IDs must decorrelate; a shared offset would leak
    that two patients were admitted on the same real calendar day."""
    o1 = patient_offset("EID-000123", key=b"unit-test-key")
    o2 = patient_offset("EID-000999", key=b"unit-test-key")
    assert o1 != o2


def test_offset_is_deterministic_across_runs():
    """Same patient + same key must reproduce the offset (incremental loads)."""
    key = b"unit-test-key"
    assert patient_offset("EID-000123", key) == patient_offset("EID-000123", key)


def test_age_ceiling_is_banded():
    rec = {"admit_dt": datetime(2024, 3, 1, tzinfo=timezone.utc), "age_years": 93}
    assert shift_record(rec, "EID-000042")["age_years"] == "90+"

The determinism test is the one teams forget, and it is the one that protects incremental pipelines: without it, a second load draws a new offset and every interval that spans the two loads is silently corrupted. Run all four in CI against a fixed test key, and validate a sample of released records against the crosswalk in a controlled environment to confirm the shift round-trips.

Gotchas & compliance constraints

  • Per-record shifting destroys the very thing you are preserving. If the offset is a function of the row instead of the patient, admission and discharge move by different amounts and the length of stay becomes noise. The offset must key on the patient’s enterprise identifier — verify with test_intervals_are_preserved above, and treat any per-row randomness as a bug.
  • Normalize timezones before you shift, not after. A dataset that mixes America/New_York and America/Chicago local stamps will produce inconsistent intervals if shifted in place; a same-day pair straddling a DST change can drift by an hour. Convert everything to UTC first — the same class of bug covered in debugging timezone mismatches in clinical timestamps. A ±365-day shift also does not hide day-of-week or season: a myocardial-infarction cluster on winter mornings or a holiday-weekend admission pattern can survive shifting and re-identify a cohort, so widen the range or add jitter when seasonality is a disclosure risk.
  • The offset is Protected Health Information and re-identifies the patient. Anyone holding the enterprise-ID-to-offset crosswalk can reverse every shifted date back to the real calendar, which is exactly why a limited dataset with dates is not de-identified under Safe Harbor and requires a data use agreement or Expert Determination. Store the crosswalk in an access-controlled, encrypted vault separate from the released data, and keep the HMAC key in a KMS — the broader controls are laid out in clinical data anonymization techniques.
  • Ages 90 and over must still be banded even when dates are shifted. Shifting a birth date does not satisfy the §164.514(b)(2)(i) age-aggregation rule; a shifted birth date combined with a shifted encounter date still yields the true age. Collapse age >= 90 to a single 90+ value, and prefer emitting banded age over a shifted date of birth wherever the analysis allows.

Date shifting is a targeted tool: it keeps the temporal structure that clinical research needs while removing the absolute calendar that Safe Harbor forbids. It is only defensible as part of an Expert-Determination or limited-dataset release with a vault-protected crosswalk — never as a substitute for the full identifier removal that Safe Harbor requires.