KPI Definitions for Interim & Lock

Why this matters

If key performance indicators (KPIs) aren’t precise, teams chase the wrong work. These definitions standardize how we judge current readiness for interim analyses and final database lock. The focus is on where we are today, not how hard we worked last month.

Glossary & formulas

Plain‑language definitions first, then how to compute them step‑by‑step. Abbreviations expanded on first use: EDC = Electronic Data Capture; SoA = Schedule of Assessments/Activities; CRF = Case Report Form; PV = Pharmacovigilance (safety case management).

1) Expectedness — “Do we have what we should have by now?”

What it is: The percent of expected CRF pages/forms that are actually present in the EDC, as of today, based on the SoA and visit windows. (Not the safety meaning of “expectedness”: whether an adverse reaction’s nature or severity is consistent with the reference safety information.)

How to compute (no math notation needed):

1. List what should exist by today (by visit window) for the enrolled/visited participants.
2. Count how many of those pages/forms are present in the EDC.
3. Express as a percent: present out of expected.
4. Report overall and for critical domains/variables (data the study defines as critical, such as primary endpoint, interim analysis, safety, eligibility and dosing data), plus the count of missing forms (expected minus present) by site and visit so sites know what to enter.

Tips:

• Use the same counting rules study‑wide (e.g., partial pages count as “present” only if all required fields are entered).
• Apply visit windows so a not‑yet‑due form does not hurt expectedness.

2) Currency — “How quickly did the site enter data?”

What it is: The percent of forms entered within the study’s timeliness target after the visit date (for example, within 5 or 7 calendar days). Also track the median lag from visit to entry. Note: the status page in the previous chapter reports the share of expected pages entered as study‑level currency; this chapter calls that share expectedness and keeps currency for entry speed. State in the DMP which definition the study uses.

How to compute:

1. Set the timeliness target per protocol or Data Management Plan (DMP).
2. For each form, subtract the visit date from the first entry date (from the audit trail, not the last edit) to get the lag in days.
3. Count how many forms met the target; divide by total forms entered; express as a percent. Forms not yet entered are outside this count, so read currency next to expectedness.
4. Also provide the median lag as a single number for context.

Tips:

• Segment by site and by form type (e.g., Adverse Events vs. concomitant medications (Con Meds)).
• Report both the timeliness percent and the median lag.

3) Cleanliness — “How much of our data is query‑free right now?”

What it is: The percent of pages/records with zero open queries (system or manual) at this moment.

How to compute:

1. For the reporting date, count pages with no open queries. A query stays open until it is closed, by whoever raised it (DM, the medical coder, or the clinical research associate (CRA) for its own monitoring queries) or by the system once a corrected value passes the edit check; an answered query awaiting review is still open.
2. Divide by the total pages entered in EDC (missing pages count against expectedness, not here); express as a percent.
3. Provide a separate view for critical variables/domains.

Tips:

• Cleanliness is not “queries closed last month.” It is the current open‑query footprint.
• Include edit checks, manual data review queries, and medical coding queries if they block analysis.

4) Query aging — “How old are the problems that remain?”

What it is: Two views:

• Median age of open queries (in days).
• Percent of open queries older than the study’s aging target (e.g., more than 14 days).

How to compute:

1. For each open query, calculate days since it was opened. Do not restart the clock at each response; that hides queries that bounce between site and DM. Time since the last response, if tracked, is a separate view.
2. Report the median age across all open queries.
3. Calculate the percent that are older than the target threshold.

Tips:

• Segment by owner (site vs. sponsor/contract research organization (CRO) vs. vendor) to route action.
• For trending, plot the percent older‑than‑target over time.

5) Reconciliations — “Does EDC match external sources?”

Reconciliation happens in two layers; always solve presence gaps before value mismatches.

Presence gaps (by owner):

• EDC present; vendor missing (e.g., a serious adverse event (SAE) recorded in EDC but missing from the PV safety database; adverse events (AEs) that are not serious are usually not held there, so they are not gaps).
• Vendor present; EDC missing (e.g., lab result in lab feed but no corresponding EDC page/record).

Value mismatches:

• Same record exists in both places, but values differ after unit, time zone, and rounding rules are applied (e.g., date/time shifts, unit conversions).

How to compute:

1. Run two‑way joins by subject/visit/specimen or other keys.
2. Count presence gaps separately by direction and owner (site vs. vendor).
3. For matched records, count value mismatches after normalization rules.
4. Track trends and burndown to zero before interim/lock (see thresholds below).

6) Coding & adjudication completeness — “Are special processes finished?”

What it is:

• Medical coding: percent of coded terms out of those requiring coding (e.g., MedDRA for AEs; WHO Drug for Con Meds).
• Endpoint adjudication: percent of events with a final committee decision out of all events that require adjudication (decided plus awaiting).

How to compute:

1. Count records requiring coding/adjudication.
2. Count those completed (approved/final).
3. Express as a percent and show dictionary/algorithm version.

7) Derivation freeze coverage — “Are calculated fields locked to a version?”

What it is: Percent of derived variables produced with a frozen (versioned) algorithm and input dictionary.

How to compute:

1. Inventory derived variables used in tables/listings/figures (TLFs) or endpoints.
2. Mark which ones are produced with the frozen version of algorithms/dictionaries.
3. Express as a percent and list what remains unfrozen.

Tip: Freeze 2–4 weeks before lock to avoid last‑minute shifts.

Thresholds & gates

Use these as defaults; tailor per protocol and risk assessment.

Interim (decision‑making cut):

• Expectedness: at least 95% for critical domains.
• Cleanliness (critical): no open critical queries; overall cleanliness trending up.
• Query aging: median ≤ 14 days; older‑than‑target trending down.
• Reconciliations: presence gaps = 0 for any domains feeding interim analyses (e.g., labs used in the interim endpoint, PV events that affect analyses); value mismatches in those domains resolved before interim outputs.
• Coding/adjudication: completed for data feeding interim outputs.
• Derivations/dictionaries: versions defined; high coverage if derivations feed interim.

Final lock:

• Expectedness: at least 99% for critical domains (and ≥ 98% overall unless justified); every expected page still missing is confirmed by the site as not done or not available, with the reason recorded.
• Cleanliness: zero open queries (study‑wide), including vendor‑owned.
• Query aging: not applicable at lock, because no queries remain open; on the way to lock, keep every open query under the threshold with a plan to zero them.
• Reconciliations: all presence gaps and value mismatches resolved.
• Coding/adjudication: 100% complete with final dictionary versions recorded.
• Derivation freeze coverage: 100% for all derived fields used in final outputs.

Interpreting trends

• Look for converging curves: expectedness rising; percent of older‑than‑target queries falling.
• Segment by owner to route work: site vs. vendor vs. sponsor/CRO.
• Use a burndown: if 30 critical queries remain and 6 working days are left, target 5 closures per day.
• Watch version stability: if derivation or dictionary versions change near lock, expect re‑work and re‑generation of outputs.
• Compare median lag for currency across sites to find training or staffing gaps.

Worked examples (CV trial)

Self‑contained scenario for a cardiovascular (CV) Phase 3 study planning an interim analysis in Week −2. Today is Week −5.

Study setup

• Critical domains for the interim: Primary endpoint visit pages, central lab results contributing to the endpoint, electrocardiogram (ECG) vendor feed, Adverse Events (AEs) affecting eligibility or censoring.
• Timeliness target: entry within 5 days of visit.
• Query aging target: ≤ 14 days.
• Coding dictionaries: MedDRA for AEs (version 26.x), WHO Drug for Con Meds (B3 format).
• Derivations: baseline, change‑from‑baseline, and an endpoint flag; algorithms are versioned and scheduled to freeze at Week −3.

Today’s numbers (Week −5)

Burndown plan to hit Week −2

• Critical queries: 42 remain; with 15 working days (3 weeks) left before Week −2, target about three closures per day (42 divided by 15 is 2.8).

• Owner routing:

  • Site B: site staff answer 31 queries, endpoint fields first (the CRA schedules 2 query resolution sessions with them); DM reviews each answer and closes the query.
  • ECG vendor: resolves 11 queries on its feed, correcting and re‑sending its data where needed (weekly joint review; escalate long‑pending tickets); DM closes each query once the vendor’s answer or new transfer confirms the fix.

• Reconciliation: DM queries the sites to record the 3 SAEs held in PV on the EDC AE pages (each SAE went from the site to PV directly); PV confirms the match, and DM checks that the new terms are coded.

• Derivations: lock algorithm/dictionary versions at Week −3; re‑run critical derived fields immediately after freeze to confirm 100% coverage.

Decision

• Interim track: Go, contingent on zeroing critical queries and PV presence gaps, resolving the 6 lab unit mismatches and coding the remaining 2% of AEs by Week −2.
• Lock outlook: To reach final lock on time, raise expectedness to ≥ 99%, complete coding to 100%, and bring derivation freeze coverage to 100%.

Pitfalls & misconceptions

• “Queries closed last month” proves cleanliness. It does not. Cleanliness is about open queries today.
• Mixing presence and value issues. Always solve presence first; value mismatches come next.
• Changing derivations near lock. Late changes force re‑runs, re‑QC, and can delay lock.
• Ignoring windows. Counting a not‑yet‑due form as “missing” will deflate expectedness unfairly.
• One big number hides problems. Without owner segmentation, you can’t route fixes efficiently.

Practice checklist

• Publish KPI definitions and counting rules in the DMP appendix.
• Tag dashboard views by owner (site/vendor/sponsor).
• Add a burndown with daily closure targets and show projection to interim/lock dates.
• Freeze derivations and dictionaries 2–4 weeks before lock; display versions on the dashboard.
• Separate presence gaps from value mismatches in reconciliation reports.
• Highlight critical variables/domains in every KPI view.

Key points to remember

• Define KPIs once and apply consistently across studies for comparability.
• Judge the “now” state—KPIs reflect current readiness, not historical effort.
• Pick the right KPI for the scenario: Expectedness (what should be present), Currency (timeliness of entry), Cleanliness (percent query‑free now), Aging (how old open issues are).
• Expectedness: percent of expected forms/pages present as of today. Targets: Interim ≥ 95% (critical domains); Lock ≥ 99% (critical) and strong overall, with every page still missing confirmed by the site as not done or not available.
• Currency: track percent meeting the timeliness target and the median lag from visit to entry to coach sites and forms.
• Cleanliness = percent query‑free now, not “queries closed last month.” Targets: zero critical queries for interim; zero open queries study‑wide for lock.
• Query aging: monitor median age and % older than threshold (e.g., >14 days) and drive the older‑than‑target line down as you approach milestones.
• Reconciliation: handle presence gaps first (EDC↔vendor); presence must be zero, and value mismatches resolved, for any domain feeding an interim cut; all presence gaps and value mismatches resolved for lock.
• Coding & adjudication completeness: done for interim‑relevant data; 100% complete with final dictionary versions recorded for lock.
• Derivation freeze coverage: freeze algorithm and dictionary versions (2–4 weeks pre‑lock); achieve 100% coverage for all derived fields used in final outputs.
• Owner segmentation & burndown: segment every KPI by owner (site/vendor/sponsor) and convert remaining critical items into daily closure targets to hit interim/lock dates.