Site Performance & Screening — Playbook

Why this matters

Performance almost never looks the same across sites. Some sites screen many but enroll few, others enter data late, and a few accumulate large query backlogs. If you find and fix these differences early, you protect interim reads, DSMB reviews, and your lock timeline. If you don’t, late‑stage surges force weekend “all‑hands,” create quality risk, and strain relationships with sites and vendors.

Goal of this playbook: give the study team a simple, repeatable way to (1) measure site performance clearly, (2) act on the right root causes, and (3) staff capacity for predictable “burndown” rather than last‑minute heroics.

Acronyms used

EDC = Electronic Data Capture; KPI = Key Performance Indicator; CRF = Case Report Form; CRA = Clinical Research Associate (monitor); CRO = Contract Research Organization; DSMB = Data and Safety Monitoring Board; DM = Data Management; FTE = Full‑Time Equivalent; IA = Interim Analysis; SAE = Serious Adverse Event; SDV = Source Data Verification; PI = Principal Investigator; AE = Adverse Event.

KPIs & thresholds

Design your per‑site dashboard so anyone can read it in under a minute. Start with these five, define them once, and publish the thresholds in the Data Management Plan (DMP) or the study’s monitoring/oversight plan, so escalation follows pre‑agreed rules. In risk‑based monitoring these per‑site thresholds work as key risk indicators (KRIs).

1) Conversion (screen → enroll)

• What it tells you: Are screened participants becoming enrolled participants at a healthy rate for this protocol? Low conversion can signal tough criteria, inconsistent eligibility interpretation, or site pre‑screening gaps.

• How to calculate: enrolled divided by screened for the same site and period.

• Suggested starting thresholds (tune to protocol):

  • Green: less than 10 percentage points below the study median, or above it (before a median exists, compare with the conversion rate assumed in study planning).
  • Amber: 10–20 percentage points below the study median.
  • Red: 20+ points below the study median or sudden drop persisting 2+ weeks.

• Investigate when: consent withdrawals rise, protocol deviations cluster around eligibility, or screen failure reasons concentrate on one criterion.

2) Timeliness (expectedness and entry lag)

• Expectedness (% expected present): Of the CRFs due by the reporting cutoff, what percent are in the EDC?

• Entry lag: Days from visit date to data entry date (median is easier to reason about than mean). Together, expectedness and entry lag measure data currency (how up to date entry is). They are leading indicators (they move before problems show downstream); query aging is a lagging one (it confirms a problem, or its fix, afterwards).

• Starting thresholds:

  • Expectedness: Green ≥95%, Amber 90–94%, Red <90%.
  • Entry lag (by visit type): Green ≤3 days routine visits, ≤1 day for safety‑critical; Amber 4–7 days routine or 2 days safety‑critical; Red >7 days routine or >2 days safety‑critical.

• Why it matters: Late data hides safety signals and inflates query aging later. Entry lag never delays SAE reporting: the site reports each SAE directly to the sponsor’s PV (pharmacovigilance) immediately (usually within 24 hours of learning of it; the protocol sets the timeline) and also records it in EDC.

3) Query aging

• What to track: median days open and percent of open queries older than 14 days.

• Starting thresholds:

  • Median age: Green ≤7 days, Amber 8–14, Red >14.
  • % older than 14 days: Green <20%, Amber 20–40%, Red >40%.

• Tip: Split by origin (SDV/CRA, data management, auto‑checks) to target the right action.

4) Attrition (discontinuation rate)

• What it tells you: Are participants staying on study? High discontinuations reduce power and often relate to consent process, visit burden, or AE management.
• Measure: discontinuations divided by enrolled for the site. Trend monthly and compare to the study median. Starting thresholds: Green up to 5 percentage points above the study median, Amber more than 5 and up to 10 points above, Red more than 10 points above or a cluster of early discontinuations. Read small sites with care: at 10 enrolled, one discontinuation moves the rate 10 points.
• Watch for: uneven early discontinuations (first 2 visits) at a single site.

5) Backlog (work you owe the database)

• Definition: open queries plus expected‑but‑missing CRFs. Both consume capacity, on two sides: only site staff enter missing CRFs and answer queries in EDC; DM (or the CRA, for its own SDV queries) reviews the answers and closes or re‑queries.
• Use: drives your FTE math (see examples) and “time‑to‑green” plans.

Display tip

A compact RAG (red‑amber‑green) table per site keeps reviews fast: Conversion, %Expected, Entry Lag, Query Median Age, %>14d, Discontinuations, Backlog.

Operational flow

1. Weekly refresh (same day/time). Pull a data snapshot (reporting extract) at a defined cutoff. This is not an EDC freeze, which blocks edits to clean data until it is unfrozen; sites keep entering and correcting data. Publish site‑level and study‑level views to the team (and to sites when appropriate).

2. Rapid triage. In a 15–30 minute huddle, flag sites in red/amber for any KPI. Tag likely cause: People (training/coverage), Process (visit flow, pre‑screening), Technology (EDC/device issues), Protocol (eligibility/complexity).

3. Root‑cause review (48–72 hours).

   • Sampling approach: pull 3–5 recent cases per signal (e.g., most recent screen failures).
   • Ask: Is the issue consistent? Is it within the site’s control? Is there a known fix?

4. Intervention plan with “time‑to‑green.”

   • Define the target (e.g., “%>14d queries under 20% within 2 weeks”).
   • Convert backlog to FTE weeks, separately for site‑owned work and DM work (see examples).
   • Assign owners by RACI (Responsible, Accountable, Consulted, Informed) and set a check‑in cadence.

5. Re‑baseline and monitor. Don’t celebrate a one‑week improvement—hold the green for 2–3 cycles before closing the issue.

Interventions & escalation

Coaching (usually via CRA)

• What good looks like: short, focused sessions tied to the exact outlier KPI. Examples: clarifying one eligibility criterion, a micro‑training on EDC query filters, or a quick walk‑through of visit scheduling to reduce late entries.
• Artifacts to leave behind: a 1‑pager “How we’ll keep this green” with screenshots, do/don’t, and contacts.

Capacity and surge

• When to add hands: large backlog, new sites coming online, or a fixed deadline (DSMB/IA/lock).
• What to add: temporary DM‑side data review, reconciliation, or coding FTEs, each trained first and then given a personal, least‑privilege EDC account that is removed when the surge ends; batch tasks (e.g., lab recon) first to create momentum. Extra hands for site‑owned work (data entry, query answers) are added by the site: staff on its delegation log, trained before access. Sponsor and CRO data managers never enter or correct site data.
• How to size: use task minutes × volume × frequency and divide by productive minutes per FTE per week (see Worked examples). For recurring work, keep volume and capacity in the same time unit and use the capacity figure your team sets (e.g., 1,800 productive minutes per FTE‑week, or 160 paid hours per FTE‑month; a paid‑hours figure gives a smaller FTE than a productive‑time one). Example: 100 vendor files a week × 12 minutes = 20 hours a week; × 52 ÷ 12 (about 4.33 weeks a month) ≈ 87 hours a month; ÷ 160 ≈ 0.54 FTE.

Governance‑aligned escalation

Use the same ladder every time so escalation feels procedural, not personal.

1. Site‑level: CRA → Site Coordinator/PI with concrete metrics and time‑to‑green.
2. Regional/CRO line management: when aging or missing data threatens DSMB/lock or persists 2+ weeks.
3. Sponsor study leadership: for systemic issues (e.g., protocol ambiguity) or vendor performance gaps.
4. Formal CAPA (Corrective and Preventive Action): when issues recur after closure, or at once when an issue significantly affects participant safety or data reliability (root‑cause analysis first). If serious or persistent noncompliance continues, ICH E6 expects the sponsor to end the site’s participation in the trial and promptly notify the regulatory authorities.

Escalation triggers to pre‑define

• %Expected Present <90% for 2 consecutive weeks.
• Median query age >14 days or >40% of queries older than 14 days.
• Conversion drops 20+ points below the study median and persists 2+ weeks.
• Any risk to DSMB/IA timelines based on the current burn rate.

Pitfalls & misconceptions

• “We closed a lot of queries, so we’re clean.” Total closures can spike when a site finally logs in; aging and expectedness tell the true story.
• Ignoring surge math. When late sites go active, expectedness dips and queries surge—plan capacity before the dip shows up.
• Blaming sites for vendor gaps. If monitors, data managers, or coding staff aren’t staffed to the plan, site performance won’t fix the backlog.
• Using study averages to judge all sites. Case‑mix, language, and visit complexity differ; compare sites to the study median and to their own baseline.
• One‑and‑done coaching. Training without follow‑up rarely sticks; schedule a quick re‑check the following week.

Worked examples

This section is self‑contained. It defines every term it uses and shows each step in words, not equations.

Example A — Conversion differences between two sites

• Definitions:

  Screened = people consented and assessed for eligibility.

  Enrolled = people confirmed eligible and enrolled as the protocol defines it (e.g., randomized), not people who only started screening procedures.

  Conversion = enrolled out of screened at the same site.

• Data (last 8 weeks):

  Site A: 40 screened, 18 enrolled → conversion is “18 out of 40.”

  Site B: 20 screened, 15 enrolled → conversion is “15 out of 20.”

• Interpretation: Site A converts fewer than half; Site B converts three‑quarters. Site A’s screen failures cluster around one criterion: the protocol requires an out‑of‑range lab value to be confirmed by one repeat test, but staff often screen‑fail the participant after the first result.

• Action: Using the sponsor’s written clarification of the criterion, the CRA delivers a 20‑minute micro‑training with a one‑page checklist for the two‑step confirmation (first result, then the repeat test), plus a note template the site uses to record the repeat result. Two weeks later, Site A shows “12 out of 20” (60%, up from 45%; small numbers, so confirm it holds over the next cycles).

Example B — Timeliness using expectedness and entry lag

• Definitions:

  Expectedness = among CRFs that were due by Friday 5 pm, what percent are present in the EDC.

  Entry lag = days from visit date to when the CRF was entered; we look at the middle value (median).

• Data (this week): 50 CRFs due; 42 are present (expectedness is “42 out of 50”). The median entry lag is 6 days.

• Thresholds: we aim for at least 95% expectedness and a 3‑day median lag for routine visits.

• Interpretation: expectedness is 84%, below the 90% red line, and the median lag is double the target (amber; red starts above 7 days). The site explains that the coordinator covers both clinic and data; data entry happens only on Fridays.

• Action: site tests a “10 minutes after each visit” habit and a Tuesday/Friday EDC session. Next week: 55 CRFs due; 53 present (96%, green); median lag 3 days (on target).

Example C — Sizing surge capacity with FTE math

• Definitions:

  Backlog = open queries plus expected‑but‑missing CRFs.

  FTE (full‑time equivalent) capacity per week = productive minutes available for a person after meetings and admin. Many teams use 1,800 minutes (about 30 productive hours).

• Scenario: backlog of 3,000 site‑answered queries waiting for DM review. Each takes a data manager about 4 minutes on average to review and close (or re‑query). You want the backlog cleared in 3 weeks.

• Steps:

  1. Multiply volume by minutes per item: 3,000 × 4 → 12,000 minutes of work.
  2. One FTE can contribute about 1,800 minutes per week.
  3. Over 3 weeks, one FTE contributes 5,400 minutes (1,800 × 3).
  4. Divide total work by what one FTE can do in the time window: 12,000 divided by 5,400 is about 2.2, so you need a little over 2 FTEs.
  5. Round up: 3 FTEs give 16,200 minutes (5,400 × 3), about 4,200 minutes of headroom for re‑queries, new queries arriving during the 3 weeks, and holidays → plan 3 FTEs for 3 weeks.

• Tip: If the backlog mixes tasks (e.g., queries and missing CRFs), do the same steps for each bucket, then add the FTEs only within the team that owns the work. Missing CRFs and unanswered queries are site hours (agreed with the site via the CRA); reviewing answers, closing or re‑querying, and reconciliation are DM FTEs.

Example D — Escalation trigger and message template

• Trigger: median query age hits 18 days for two consecutive weeks and 45% of queries are older than 14 days.

• Action ladder:

  1. CRA aligns with Site Coordinator: share metrics, agree on a 2‑week “time‑to‑green” plan.
  2. If no improvement by next week, escalate to CRO line manager with specifics and the agreed plan.

• Message template (to CRO line manager):

  Subject: Study ABC – Site 104 query aging above threshold (plan attached)

  Body: “As of Friday’s cutoff, Site 104 median query age is 18 days with 45% >14 days. Site agreed to twice‑weekly EDC sessions and a dedicated hour post‑clinic (plan attached). If the next refresh remains above threshold, request 1 week of temporary DM support (approx. 900 minutes, half of one FTE‑week) to review the site’s answers and close or re‑query them promptly, speeding burndown and protecting the DSMB timeline.”

Practice in the wild (checklist)

• Publish definitions and thresholds for Conversion, Expectedness, Entry Lag, Query Aging, Attrition, Backlog.
• Set a refresh day/time and stick to it; share a one‑page site view with RAG colors.
• Add a comment column per site capturing the suspected cause and the “time‑to‑green” target.
• Size capacity with an FTE calculator (task minutes × volume × frequency; divide by productive minutes per FTE in the same time unit).
• Pre‑agree escalation triggers and the ladder (Site → CRO line → Sponsor lead → CAPA).
• Track payment‑relevant items (visit completions and data currency, i.e., how up to date entry is) to improve attention and trust.
• For new sites or protocol amendments, schedule a post‑go‑live check at week 2 focusing on expectedness and entry lag.

Key points to remember

• Watch conversion and timeliness, not just totals. Use screen→enroll conversion (compare to the study median) to diagnose screening variability early; investigate if a site drops ≥20 points below the median for 2+ weeks.
• Timeliness = leading signal. Define Expectedness as “% of due CRFs present” (target ≥95%), and Entry lag as median days from visit to entry (target ≤3 days for routine visits; ≤1 day for safety‑critical).
• Track both leading and lagging indicators. Expectedness and Entry lag are leading; Query aging is lagging. You need both to see problems forming and to confirm they were fixed.
• Query aging thresholds guide action. Aim for median ≤7 days and <20% >14 days. Escalate when median >14 days or >40% >14 days persists, or when risk to DSMB/IA timelines is evident.
• Backlog = work you owe the database. It is open queries + expected‑but‑missing CRFs; size site work (entry, query answers) and DM work (review, closure) separately to drive capacity and “time‑to‑green” plans.
• FTE sizing (surge math) is simple and explicit. FTEs needed ≈ (items × minutes per item) ÷ (1,800 productive minutes per FTE per week, or your team’s figure, × weeks to target) → round up, and check the result leaves headroom for re‑queries, new work and holidays.
• Plan for peaks, not averages. Add temporary DM‑side hands early (trained first, with personal least‑privilege accounts) for known surges (DSMB/IA/lock) and start with batchable tasks (e.g., lab recon) to create momentum.
• Monitor attrition (discontinuations ÷ enrolled) by site and month; early drop‑offs often point to consent clarity, visit burden, or AE management.
• Escalate through governance, not personalities. Follow a consistent ladder (CRA → Site Coordinator/PI → CRO line → Sponsor lead → CAPA) tied to pre‑defined triggers and a concrete time‑to‑green target; go straight to CAPA when an issue significantly affects participant safety or data reliability.
• Operate on a steady cadence. Weekly fixed cutoff, one‑page RAG per site, and a comment column with suspected cause and target; hold green for 2–3 cycles before closing.
• Show payment‑relevant metrics (e.g., visit completions) in dashboards to focus attention and build trust.
• Make the dashboard readable in under a minute. Keep to the core KPIs: Conversion, Expectedness, Entry lag, Query median age, %>14d, Discontinuations, Backlog.