Feed Monitoring & Alerts

Why this matters

Every external data feed (labs, devices, electronic patient‑reported and clinical outcome assessments (ePRO/eCOA), imaging, pharmacovigilance) introduces timing and quality risk. If a file is late, structurally invalid, or quietly changes shape, downstream processes break and reconciliation debt grows. The goal of feed monitoring is simple:

1. See deviations fast (minutes to hours, not days).
2. Tell the right owner exactly what failed and what to do next.
3. Prove control during audits and at database lock.

Key terms (spelled out on first use):

• EDC (Electronic Data Capture): your clinical database used by sites.
• KPI (Key Performance Indicator): a measurable signal tracked over time.
• SLA (Service Level Agreement): the time/quality commitment in the contract.
• Schema drift: a change to file layout (columns, data types, formats) that no approved spec version covers, usually unannounced.
• Conformance: whether a file matches the agreed transfer specification (structure and basic rules).
• Presence gap: data expected from both sources (EDC and vendor) but missing in one.

What to monitor (KPIs & thresholds)

Instrument each feed with the KPIs below. Start with default thresholds and tighten near interim analysis, a data safety monitoring board (DSMB) review, or database lock.

• On‑time delivery rate (by vendor) — target ≥ 98%

  • What it tells you: Are files arriving by the agreed cut time (e.g., 09:00 UTC daily)?
  • How to measure: For each expected delivery, mark on‑time vs late; report rolling 30‑day rate.
  • Owner: Vendor operations.
  • Notes: Track planned blackouts separately (e.g., vendor maintenance) so they don’t mask issues. With a daily feed, one late file in 30 days already gives 96.7%, so report the count of late files beside the rate.

• Structural conformance pass rate — target 100%

  • What it tells you: Does the file match the specification (file naming pattern, columns, order, types, required fields, delimiters, control file/manifest)?
  • How to measure: Run automated checks on ingest; count pass/fail. Fail‑closed (do not load) on structural errors.
  • Owner: Vendor (to fix and re‑deliver); Data Engineering reviews evidence.
  • Notes: Store the spec version with each file (e.g., spec_version=1.2).

• Schema drift events — target 0 (layout changes come only through an approved new spec version)

  • What it tells you: A column was added/removed/renamed or a format changed without notice.
  • How to measure: Compare file header and data types to the spec; flag any deviation.
  • Owner: Vendor to roll back or issue a change notice (a request for a new spec version, which applies only once approved, from its effective date); data management (DM) and Data Engineering (DE) to impact‑assess and plan the upgrade.

• Presence gaps (EDC vs vendor) — target trend to 0

  • What it tells you: Which source is missing expected records?

  • How to measure: By subject/visit/timepoint, compare the set of records present in EDC vs vendor. Classify:

    • EDC present; vendor missing → vendor action
    • Vendor present; EDC missing → site/CRA action (source not transcribed)
    • Both present; values mismatch → site (transcription) or vendor (derivation error), based on the rule in the DMP or reconciliation plan that names each field’s source of truth; a mismatch caused by the sponsor’s own mapping, unit conversion or derivation goes to the internal owner of that program (Data Engineering, or DM and biostatistics for a sponsor derivation), not to the site or vendor. (A mismatch is not a presence gap but shares its backlog.)

  • Owner: Varies by classification; see routing table below.

• Volume anomalies — target: investigate all “high” anomalies

  • What it tells you: Sudden dips/spikes vs normal volume.
  • How to measure: Compare the latest load count to a rolling baseline (e.g., last 14 deliveries). Flag large percentage changes or changes beyond typical variability (no equations required).
  • Owner: Vendor (missing/duplicate deliveries) or Site/CRA (batched sample shipments or missed visits); DE supports diagnosis.

• Age since last file — target ≤ 1 × delivery interval plus the grace period (the agreed time after the cut before a file counts as missing, e.g., 2 hours)

  • What it tells you: How long it has been since the last valid file (e.g., >26h for a daily feed with a 2‑hour grace period).
  • Owner: Vendor (for scheduled feeds); DM/DE for ad‑hoc or event‑driven feeds.

• Backlog to reconcile — target within SLA

  • What it tells you: Count of open presence gaps and mismatches, grouped by owner and aging buckets.
  • Owner: DM Lead to drive closure; CRA for site items; Vendor PM for vendor items.

• Error budget (for repeated minor issues) — target not exceeded

  • What it tells you: Tolerance for small, non‑impacting defects before escalation (e.g., up to 3 non‑blocking warnings per month, such as a duplicate re‑send caught by its checksum; late files count against the on‑time rate instead).
  • Owner: DM Lead; ties to vendor performance management.

Implementation tips

• Store a delivery log for each feed (expected/received timestamps, SLA, spec version, checksum).
• Keep evidence artifacts: validation report, conformance results, and the email/ticket requesting re‑delivery.
• Use time‑zone aware scheduling; report SLA status in a single reference zone (e.g., UTC).

Alerting & runbooks

Design alerts to be actionable: clear severity, owner, and next step. Avoid noisy or duplicate alerts.

Severity levels (default)

• P1 – File missing/blocked Trigger: expected delivery missed (not received by the end of the grace period) or a structural conformance failure, including schema drift, that blocks the load. Response target: 2 hours. Example actions: request re‑delivery, rollback vendor release, open incident ticket.

• P2 – Data quality at risk Trigger: repeated volume anomalies or high presence gaps (the data load, but their quality is at risk). Response target: 1 business day. Example actions: open a CAPA (corrective and preventive action) with the vendor, inform study team, adjust thresholds near interim/lock.

• P3 – Informational/low Trigger: small, explained variance; non‑blocking warning. Response target: next weekly review; monitor trend.

Who to notify (typical)

• Vendor operations + vendor project manager (PM) (for delivery/conformance/drift).
• CRA (Clinical Research Associate) / Site monitors (for site‑owned presence gaps).
• DM Lead + Data Engineering (DE) (for all P1s and P2s).
• Study PM / CRO lead when SLAs are breached or milestones are at risk.

What to include in every alert

• Feed name and vendor, severity, and why triggered.
• Spec version, last successful delivery timestamp, last pass result.
• Counts affected (subjects/visits/records), and proposed next step.
• Link to the runbook section and the original transfer specification.

Runbook blueprint (one page per alert type)

• P1 – Missing or failed file

  1. Verify schedule (no planned blackout).
  2. Check delivery logs and the SFTP (secure file transfer) inbox; confirm no quarantined file.
  3. Open vendor ticket with required message (see template below).
  4. On re‑delivery: re‑run conformance; if pass, proceed to load; if fail, keep blocked and escalate to vendor leadership.

• P1 – Schema drift (file blocked)

  1. Capture drift details (column name/type/format).
  2. Confirm spec version; request rollback or change notice.
  3. Impact‑assess downstream mappings and listings.
  4. Approve planned upgrade and coordinate effective date; retest.

• P2 – Presence gaps (site‑owned)

  1. Produce subject/visit list.
  2. DM opens targeted site queries with instructions (no general “please enter data” messages); the CRA follows up with the site.
  3. Track closure and aging weekly.

Vendor ticket template (copy/paste)

Subject: P1 – [FEED] [DATE] delivery missing / conformance failed

Context: Spec v[1.2], last pass [2025‑06‑14 09:02 UTC].

Issue: [Missing file / New column ‘HRV’ detected / Column order mismatch].

Impact: [Blocked load / n=312 records delayed].

Request: Re‑deliver conformant file by [time]; confirm root cause and CAPA.

Pre‑milestone windows In the week before interim analysis or lock, tighten thresholds (e.g., on‑time target to 99.5%, lower anomaly tolerance) and increase alert cadence (twice daily checks).

Routing & escalation

Route by cause, not by who noticed the problem.

Escalation ladder

1. Owner team (Vendor Ops or CRA)
2. Vendor PM / CRA Lead
3. CRO/Vendor leadership with CAPA
4. Contract remedies if CAPA ineffective (measured by effectiveness checks)

Contract line: escalate through the vendor’s formal chain named in the contract or statement of work (SOW), from the vendor PM to its functional lead and then its account/executive sponsor, citing the SLAs, KPIs and remedies in that contract and bringing the KPI trend and ticket history; a vendor issue is never pushed to the sites instead. In a CRO‑run study, start with the CRO’s lead data manager and PM. The sponsor keeps oversight of every vendor even when a CRO manages it.

Worked examples (CV trial)

Context for all examples: A cardiovascular (CV) outcomes trial with daily central lab feeds, a central ECG vendor feed, and hourly wearable device feeds. EDC is the system used by sites for data entry. Time zone standard is UTC.

1) Late central lab file (P1 – missing delivery)

• Expected: Daily file by 09:00 UTC; last success yesterday at 08:57 UTC.
• Monitor fires at 11:00 UTC: “No lab file since 2025‑09‑06 08:57 UTC (age 26h).” The 2‑hour grace period after the 09:00 cut has ended.
• Automated checks: SFTP inbox empty; no maintenance outage logged.
• Action: Vendor ticket using P1 template; vendor confirms scheduler failure; re‑delivers at 12:10 UTC.
• Conformance: Passes structure; loaded successfully.
• Follow‑up: Vendor provides root cause and CAPA; this one late file takes the month’s on‑time rate to 96.7% (29 of 30 daily files), below the 98% target; the miss goes to the monthly vendor review.

2) Device schema drift (P1 – drift, blocked until rollback)

• Expected: Spec v1.0 with columns [SUBJID, TS, HR, STEPS].
• Observed: New column HRV appears (unannounced). Conformance fails by design (fail‑closed).
• Action: Vendor rolls back to the v1.0 layout within 4 hours and re‑delivers the blocked files; opens a change request for spec v2.0 with HRV (files and programs built to v1.0 do not meet a layout with a new column, so it is a breaking change and a major version).
• Impact assessment: DM and DE review listings, the impact on SDTM (Study Data Tabulation Model) submission datasets, and downstream dashboards.
• Outcome: Controlled upgrade to v2.0, effective next week, with regression test evidence attached to the change record.

3) Presence gaps hint at site transcription lag (P2 – site‑owned)

• Signal: 18 ECG results in vendor file for Site 104 lack matching EDC entries for the same subjects/visits.
• Diagnosis: CRA confirms ECGs were performed; coordinator is behind on data entry due to staff turnover.
• Action: DM opens targeted site queries listing subject/visit/time; the CRA agrees a 7‑day catch‑up plan with the site (new staff are trained and on the delegation log before they get EDC access).
• Follow‑up: Weekly backlog trend shows closure; gap count returns to baseline.

4) Volume anomaly after holiday (P3 → watch, then close)

• Signal: Daily lab results drop ~35% vs typical weekday volume after a public holiday.
• Context: More missed visits on the holiday; no missed deliveries.
• Action: Mark as explained variance; document in the weekly DM note; no escalation.

Each example stands alone: it states the expectation, the exact alert, the diagnosis, the owner, and the close‑out evidence.

Pitfalls & misconceptions

• Loading failed files to “keep moving.” This hides problems and creates rework during reconciliation and mapping. Keep structural failures blocked until fixed.
• Escalating to sites for vendor omissions. If EDC has the record but the vendor feed does not, start with the vendor.
• One‑size‑fits‑all alerts. Tune thresholds by feed and by phase (e.g., tighten before interim/lock).
• Ignoring time zones. Define a single reference zone (e.g., UTC) for SLAs and monitoring.
• No spec versioning. Always capture and report the spec version used for each delivery.

Practice checklist

• Track on‑time, conformance, schema drift, presence gaps, and volume anomalies for every feed.
• Keep delivery logs, validation reports, and spec version with each file.
• Maintain short runbooks per alert type with owners and SLAs.
• Use fail‑closed on structural conformance; never load non‑conformant files.
• Escalate through the contract line; require CAPA and effectiveness checks.
• Tighten thresholds and increase cadence ahead of DSMB/interim/lock.

Key points to remember

• Instrument every feed for on‑time, structural conformance, schema drift, presence gaps, and volume anomalies—all reported in a single reference time zone (e.g., UTC).

• Fail‑closed on structure. Any file that violates the transfer specification (naming, columns, order, types, required fields, manifest) does not load; vendor re‑delivers a conformant file.

• Route by cause, not discoverer.

  • EDC present; vendor missing → vendor action (cut/omission).
  • Vendor present; EDC missing → site/CRA action (transcription lag).
  • Both present; values mismatch → site (transcription) or vendor (vendor derivation) based on the rule; a mismatch from the sponsor’s own mapping, conversion or derivation goes to its internal owner.

• Control schema changes. Column or format changes outside an approved spec version = schema drift → file blocked (P1), then rollback or formal change notice, impact assessment, controlled upgrade with test evidence.

• Honor SLAs. Define delivery windows in UTC; track on‑time rate (e.g., ≥ 98%) and trigger P1 when the grace period is exceeded; recurring breaches escalate via the contract line.

• Actionable alerts only. Include severity, owner, trigger reason, counts affected, spec version, last pass timestamp, and the next step with a link to the runbook and spec.

• Keep runbooks tight. One page per alert type (missing/failed file, drift, presence gaps) with verification steps, owners, and response targets (e.g., P1: 2 hours).

• Maintain audit evidence. Delivery log (expected/received timestamps, SLA outcome, checksums), conformance results, spec version, tickets, and re‑delivery confirmations.

• Use an error budget for low‑severity, non‑impacting defects; escalate when the budget is exceeded.

• Tighten before milestones. In the week before DSMB/interim/lock, raise targets (e.g., on‑time ≥ 99.5%), lower anomaly tolerances, and increase alert cadence.

• Escalate with CAPA. Persistent or repeated issues move to vendor leadership/CRO with CAPA and an effectiveness check; close only with objective improvement in KPIs.

• Feed monitoring drives lock readiness. These signals roll up directly into the lock‑readiness dashboard and support inspection narratives.