Boulder Creek, Colorado — Regulation 93 / 303(d) E. coli Compliance Monitoring
Limited-Use Alternate Test Procedure Application under 40 CFR 136.5
The EPA Alternate Test Procedure (ATP) program, codified at 40 CFR 136.4 (nationwide) and 40 CFR 136.5 (limited-use, facility-specific), provides a pathway for developers to submit new analytical methods for regulatory approval as alternatives to methods listed in 40 CFR Part 136. These are the required methods for Clean Water Act compliance reporting. This document describes the study design and statistical analysis plan for a limited-use ATP application under 40 CFR 136.5, submitted by the City of Boulder Utilities to EPA Region 8.
Boulder Creek is listed as impaired under Colorado Regulation 93 (the state implementation of the federal 303(d) program for impaired waters) for E. coli exceedances. The applicable water quality standard is a geometric mean of 126 CFU/100 mL over a 30-day period, with a single-sample maximum of 235 CFU/100 mL. The City of Boulder Utilities currently monitors compliance at 6 designated monitoring locations on Boulder Creek using IDEXX Colilert as the reference method. This study will determine whether the Virridy Lume tryptophan-like fluorescence (TLF) sensor produces results statistically equivalent to Colilert at these compliance monitoring sites.
Current Reg 93 compliance monitoring on Boulder Creek relies on discrete grab samples processed by a laboratory with an 18–24 hour delay before results are available. This creates a structural monitoring gap: contamination events driven by storm runoff, upstream discharges, or wildlife activity may occur and resolve entirely between sampling events, leaving the public exposed without warning. The Lume sensor provides continuous, in-situ E. coli estimates at 15-minute intervals, enabling real-time detection of exceedance events and substantially increasing the density of compliance-relevant data.
Virridy has previously collected 856 paired Lume–Colilert observations across multiple sites and waterbodies (global dataset, 2021–2025) (R² = 0.67, Cohen’s kappa = 0.82–0.84, MAPE = 7.1% in log-transformed space). This existing dataset is used to support the pre-submission consultation with EPA Region 8 and demonstrates technical feasibility, but does not constitute the formal regulatory validation dataset. The prospective study described in this document will generate the regulatory-grade paired dataset required for the 136.5 application, under a pre-specified, agreed study protocol.
Modeling after Colilert: This study design is modeled on the validation framework used by IDEXX to obtain EPA approval for Colilert, which included: (1) precision studies within and across laboratories, (2) comparative paired testing against existing approved methods across multiple matrices, (3) interference characterization, (4) method detection limit determination, and (5) pre-specified statistical acceptance criteria. Those same elements are incorporated here, adapted for continuous in-situ monitoring.
Demonstrate that the Virridy Lume tryptophan-like fluorescence sensor, using a multivariate linear regression model with TLF intensity, turbidity, and temperature as inputs, produces E. coli concentration estimates that are statistically equivalent to IDEXX Colilert results at Boulder Creek’s 6 Regulation 93 TMDL monitoring sites across all seasons, flow conditions, and concentration ranges relevant to Reg 93 compliance.
The primary regulatory claim is direct replacement of Colilert grab sampling for Reg 93 compliance reporting. Fallback claims are listed in order of preference if the primary claim cannot be fully supported by the dataset:
The Lume reading at the time of a scheduled compliance sampling event replaces the Colilert result as the official Reg 93 compliance value. City of Boulder no longer needs to submit grab samples for laboratory Colilert analysis at approved monitoring sites. This is the strongest regulatory claim and requires the tightest statistical equivalence demonstration.
A rolling geometric mean of Lume readings over a defined period constitutes the compliance measurement, independent of grab sampling. Requires agreement with EPA and CDPHE on the aggregation period and how continuous data maps to the Reg 93 geometric mean standard.
The Lume is the primary continuous monitor. A Lume exceedance above a defined threshold triggers an immediate Colilert grab sample for confirmation before any official compliance action. The Lume is the compliance tool; Colilert is the safety net.
Lume data supplements but does not replace Colilert for official compliance reporting. Used for real-time public advisories, operational decisions, and interim monitoring between scheduled Colilert events. Lowest regulatory bar; does not require equivalence demonstration.
The Lume sensor measures tryptophan-like fluorescence (TLF) in situ at 275 nm excitation and 340 nm emission. Tryptophan is an amino acid present in the cell walls of bacteria, including E. coli, and its fluorescence signal is a proxy for fecal contamination. The sensor simultaneously measures turbidity (NTU) and water temperature (°C) as co-variates. A multivariate linear regression model, fit to paired Lume–Colilert calibration data and validated on a held-out test set, converts the three sensor outputs to an E. coli concentration estimate in CFU/100 mL.
The regulatory prediction equation takes the following form, fit in log10 space to account for the log-normal distribution of environmental E. coli concentrations:
Where TLF is in ppb tryptophan equivalents, turbidity is in NTU, and temperature is in °C. The model coefficients (β0–β3) will be estimated by ordinary least squares regression on the training partition of the study dataset (see Section 8). The fitted coefficients will be locked before evaluation on the holdout test set and will not be modified thereafter. The final coefficient values will be reported in the ATP application and become fixed parameters of the approved method.
Why linear regression, not machine learning: A multivariate linear regression model was selected over more complex ML approaches (gradient-boosted trees, neural networks) because: (1) the equation is fully transparent and auditable by EPA on a spreadsheet; (2) there are no versioning or retraining concerns once coefficients are fixed; (3) the regulatory precedent for linear correction equations in approved methods is well-established (e.g., turbidity corrections in UV absorbance methods, temperature corrections for dissolved oxygen); and (4) no site-specific calibration is required, making the method universally applicable.
| Parameter | Specification |
|---|---|
| TLF excitation wavelength | 275 nm |
| TLF emission wavelength | 340 nm |
| TLF units | ppb tryptophan equivalents |
| Turbidity measurement | Nephelometric (NTU) |
| Temperature measurement | °C (thermistor) |
| Measurement interval | 15 minutes (configurable) |
| Deployment type | Submersible in-situ sensor |
| Data transmission | Cellular (primary); on-board SD card (backup) |
| Power | Solar or line power with battery backup |
| Operating temperature range | 0–40 °C |
This method is applicable to freshwater surface water for the measurement of E. coli in the range of approximately 1–10,000 CFU/100 mL. The method is designed for continuous in-situ monitoring and is not applicable to discrete grab sample laboratory analysis. The method as validated is specific to Boulder Creek, Colorado under the 40 CFR 136.5 limited-use approval; nationwide applicability under 40 CFR 136.4 will require additional multi-site validation.
The study will be conducted at the 6 City of Boulder Utilities TMDL compliance monitoring locations on Boulder Creek. These are the same sites at which Boulder currently collects Colilert grab samples for Regulation 93 reporting to CDPHE. Using existing compliance monitoring sites ensures that the validation dataset directly reflects the conditions and concentration ranges relevant to the regulatory claim.
| Site | Description | Expected Range | Regulatory Significance |
|---|---|---|---|
| BC-1 | Upstream reference / background | Low (<10 CFU/100 mL) | Establishes baseline TLF and natural organic matter background; critical for MDL characterization |
| BC-2 | Above WWTP discharge | Low–moderate | Mixed urban/natural signal; no direct WWTP influence |
| BC-3 | Below WWTP discharge | Variable — low to high | Critical compliance point; demonstrates performance at and above the 126 CFU/100 mL Reg 93 threshold |
| BC-4 | Mid-creek compliance point | Moderate | Mixed WWTP effluent + stormwater signal; seasonal variability important |
| BC-5 | Downstream compliance point | Moderate–low | Natural attenuation zone; demonstrates performance during dilution and recovery |
| BC-6 | Lower Boulder Creek / watershed outlet | Variable | Integrates full watershed; irrigation diversions may affect flow dynamics seasonally |
Note: Site descriptions and names are provisional. Exact site IDs, GPS coordinates, and Reg 93 permit associations to be confirmed with City of Boulder Utilities prior to study start.
The study will run for 12 months from the formal study start date (defined as the date all 6 sensors are deployed and the reference method sampling program is operating at the required frequency). Twelve months is the minimum duration required to capture all four seasons and, critically, two complete spring runoff events on Boulder Creek. Spring runoff (typically April–June) produces the highest E. coli concentrations of the year and is the period most likely to generate exceedances of the Reg 93 threshold. It must be represented in both the training and holdout partitions of the dataset.
Minimum duration rationale: A 9-month study (fallback) is statistically achievable in terms of sample counts but risks missing adequate representation of spring runoff in the holdout period. EPA is likely to scrutinize performance at high concentrations near the Reg 93 threshold specifically. A 12-month study captures spring runoff twice and eliminates this vulnerability. The existing 856-observation dataset may be submitted to EPA Region 8 as supplementary evidence if a reduced prospective study duration is requested.
| Task | Timing | Details |
|---|---|---|
| Deploy Lume sensors at all 6 sites | Pre-study | Each sensor co-located within 2 meters of the designated Colilert grab sample collection point at the same depth and cross-section position. GPS coordinates, installation photographs, and sensor serial numbers documented at deployment. |
| Firmware and model version lock | Study Day 0 | Firmware version and prediction equation coefficients (once fit per Section 8) are locked for the study duration. Any firmware updates require a protocol amendment and notification to EPA Region 8. |
| Data pipeline validation | Week 1 | Confirm all 6 sensors are transmitting reliably. Verify measurement interval, data completeness, and sensor-to-server latency. Automated alerts configured for signal dropout, out-of-range readings, and battery/power status. |
| Party | Role |
|---|---|
| Virridy | Sensor deployment, maintenance, calibration verification, data management, and statistical analysis. Contracted to City of Boulder for the duration of the study. |
| City of Boulder Utilities | Applicant on the 136.5 ATP application. Oversight of the study. Coordinates Colilert grab sample collection at the 6 monitoring sites per existing Reg 93 SOPs. |
| Reference Laboratory | State-certified laboratory for E. coli analysis. Processes all Colilert samples for the study. [See Open Question B-1: confirm certified lab and willingness to increase sampling frequency] |
| CDPHE | State regulatory oversight. Reviews study design and provides letter of support for the 136.5 application. |
The reference method for this study is IDEXX Colilert for E. coli quantification in freshwater, analyzed using the Quantitray/2000 MPN method per the manufacturer’s protocol and consistent with 40 CFR Part 136 Table IA. The specific Colilert formulation (standard 24-hour vs. Colilert-18 18-hour) will be confirmed with City of Boulder Utilities and will match the version currently used for Reg 93 compliance reporting. Both are EPA-approved; consistency with Boulder’s existing compliance program is required so that the ATP validation dataset is directly comparable to the regulatory baseline.
Critical prerequisite — Laboratory Certification: All Colilert reference samples used in the regulatory validation dataset must be analyzed by a laboratory holding current state certification under the Colorado Laboratory Certification Program (CLCP) for E. coli analysis. This is a prerequisite, not a preference. Uncertified reference data is grounds for application rejection. [See Open Question B-1 and B-2]
| Parameter | Requirement |
|---|---|
| Collection frequency | Minimum 3 times per week per site during the study period. Event-based enhanced sampling (see 5.3) increases this frequency during high-flow events. |
| Collection timing | Exact time of grab sample recorded to the minute by the collector. Matched against nearest Lume 15-minute reading (pairing window: ≤15 minutes; see Section 6). |
| Collection location | Within 2 meters of the Lume sensor, same depth and cross-section position. |
| Collection containers | Sterile, single-use 100 mL bottles per Part 136 / Colilert protocol. |
| Preservation and holding time | Per approved Colilert protocol: chilled at 4 °C, analyzed within 6 hours of collection. |
| Chain of custody | Formal chain of custody documentation for every sample, from collection through laboratory reporting. COC retained for minimum 10 years. |
| Field conditions log | At each sampling event: weather (clear/rain/snow/runoff), visual water clarity, estimated stream flow condition (baseflow/elevated/flood), any visible upstream disturbance, air temperature, and collector initials. |
| Laboratory duplicate | Every 10th paired sample: collect a field duplicate grab (two bottles from the same sampling event). Submitted blind to the laboratory. Provides reference method precision estimate (relative percent difference). |
Spring runoff and storm events are the primary drivers of Reg 93 exceedances on Boulder Creek. The Lume’s continuous monitoring will automatically detect elevated TLF signals. When any Lume sensor reads above a pre-defined alert threshold (to be determined during initial deployment, approximately 2× baseline TLF), the Colilert sampling frequency at that site will be increased to daily for the duration of the event. This generates additional paired observations at high concentrations, which are the most important for demonstrating performance at and above the 126 CFU/100 mL threshold.
A paired observation is defined as a Colilert result and a Lume sensor reading that meet all of the following criteria:
Pairing rationale: The ±15 minute window is more conservative than the ±30 minute window used in preliminary analyses. In a flowing stream, water conditions can change meaningfully over 30 minutes during events. A tighter pairing window ensures that the Lume reading and the Colilert sample represent the same parcel of water, which is the fundamental assumption underlying the equivalence claim.
| Metric | Minimum Target | Rationale |
|---|---|---|
| Total valid paired observations | ≥600 | 6 sites × ≥100 per site. Substantially exceeds Colilert’s original multi-lab validation sample count. Provides sufficient power for equivalence testing and holdout evaluation. |
| Paired observations per site | ≥80 | Minimum for site-level performance statistics. Lower-bound sites may have fewer observations due to site-specific access constraints. |
| Observations above 126 CFU/100 mL | ≥30 total (≥5 per site) | The Reg 93 threshold must be represented in the dataset. Classification performance at the threshold is meaningless without samples above it. Enhanced event sampling (Section 5.3) is designed to meet this requirement. |
| Observations above 235 CFU/100 mL | ≥10 total | The Reg 93 single-sample maximum. High-end performance must be characterized. |
| Seasonal coverage | All 4 seasons at all 6 sites | Spring runoff, summer baseflow, fall recession, and winter low-flow represent distinct hydrologic and biological conditions. All must be represented. |
| Continuous data completeness | ≥90% uptime per sensor per month | Demonstrates operational reliability of the method as deployed. |
All paired observations will be stored in a centralized, version-controlled database. Each record will include: site ID, sample date and time (UTC), Lume TLF reading (ppb), turbidity (NTU), temperature (°C), raw Lume E. coli estimate (CFU/100 mL), Colilert MPN result (CFU/100 mL), data quality flags, field conditions log entries, laboratory COC reference, and collector/operator ID. Raw data will be archived in original format with all transformations and QA flag applications fully documented and reproducible. Minimum retention: 10 years.
| QC Check | Frequency | Details |
|---|---|---|
| Field blank | Monthly (per site) | Submerge sensor in laboratory-grade DI water before and after each maintenance visit. TLF reading must be ≤[MDL to be determined]. Any reading above MDL triggers investigation and possible recalibration before data collection resumes. |
| Tryptophan standard verification | Monthly (per site) | Verify sensor response against a prepared tryptophan standard at a known concentration (e.g., 100 ppb). Acceptable response: within ±15% of the expected value. Out-of-range triggers recalibration. |
| Optical window cleaning | Bi-weekly | Clean optical window with approved cleaning solution. Verify post-cleaning blank reading before returning sensor to service. |
| Flag | Trigger | Action |
|---|---|---|
| Fouling suspected | TLF drift >20% over 24 hours without corresponding Colilert change; post-cleaning blank >MDL | Flag all readings during fouling period. Exclude from paired dataset. Document in maintenance log. |
| Out-of-range turbidity | Turbidity >[upper operating limit NTU — to be determined from initial deployment] | Flag readings. High turbidity may saturate optical signal. Include flagged readings in sensitivity analysis but exclude from primary dataset. |
| Maintenance window | Any period during which sensor was removed, cleaned, or calibrated | Exclude all readings within ±30 minutes of maintenance activity from paired dataset. |
| Power interruption | Gap in data >30 minutes | Flag readings immediately before and after gap. Investigate cause. Exclude interpolated values from paired dataset. |
| Temperature out of range | Water temperature <0 °C or >35 °C | Flag readings. Outside characterized operating range. Include in sensitivity analysis; exclude from primary dataset. |
| QC Check | Frequency | Details |
|---|---|---|
| Laboratory field duplicate | Every 10th sample | Blind field duplicate submitted to certified laboratory. Acceptable lab RPD: ≤[to be confirmed with reference laboratory; typically ≤30% for environmental E. coli samples]. |
| Laboratory positive control | Per laboratory QA plan | Certified laboratory runs positive and negative controls per their accredited QA plan. Control results reported with each analytical batch. |
| Colilert holding time compliance | Every sample | Sample analyzed within 6 hours of collection. Samples exceeding holding time excluded from paired dataset. |
For a calendar month’s data to count toward the study dataset, the following minimum completeness thresholds must be met: (a) ≥90% of expected Lume 15-minute readings must be present and unflagged for each site; (b) ≥80% of planned Colilert grab samples must have valid paired Lume readings within the ±15 minute pairing window. Months not meeting these thresholds will be documented and excluded from the primary dataset; if exclusion brings total paired observations below the Section 6.2 minimums, the study will be extended accordingly.
The prediction equation (Section 3.2) will be fit by ordinary least squares (OLS) regression to log10-transformed E. coli concentrations. Below-detection-limit Colilert results will be treated as one-half the detection limit for regression purposes (standard left-censored data convention). The model form is additive with three predictors — TLF, turbidity, and temperature — and no site-specific or temporal covariates. No interaction terms will be included unless pre-specified below and justified by exploratory analysis on the training partition only.
Critical: The train/test split methodology described below is pre-specified prior to data collection and may not be modified after data collection begins. This prevents any risk of split selection that optimizes apparent model performance. EPA will be provided this document before the study start date as evidence that the split was pre-defined.
| Partition | Data Period | Purpose |
|---|---|---|
| Training set | Months 1–9 of the study (all 6 sites) | Fit the OLS regression coefficients (β0–β3). Model is built entirely on this partition. The coefficients are fixed after training and not updated. |
| Holdout test set | Months 10–12 of the study (all 6 sites) | Independent evaluation of the fixed model. All primary performance statistics (see Section 9) are calculated on this partition only. The holdout period is chosen to include spring runoff (if the study starts in January, this is the second spring; otherwise adjusted accordingly). |
A temporal split is used rather than a random split because it mimics real-world deployment: the model is trained on historical data and applied forward in time. This is more conservative than a random split and eliminates any possibility of the training data "seeing" future conditions.
Before the model is locked for holdout evaluation, it must meet the following minimum acceptance criteria on the training set:
If these criteria are not met on the training set, the model form will be reviewed (e.g., log10 transformation of predictors, addition of one pre-specified interaction term: TLF × turbidity) with notification to EPA Region 8. The holdout set will not be examined until the model form is finalized.
The final fitted coefficients (β0, β1, β2, β3), standard errors, 95% confidence intervals, and p-values will be reported in the ATP application. The complete prediction equation with numerical coefficients will be included in the method documentation (Section 3.2) such that any laboratory can implement the method using a standard spreadsheet without proprietary software.
Pre-specified SAP: All statistical tests and acceptance thresholds in this section are pre-specified. No additional tests will be added and no thresholds will be modified after data collection begins. This section is the binding statistical analysis plan for the ATP application.
Evaluated on the holdout test set only (Months 10–12). All analyses on log10-transformed values unless otherwise specified.
| Test | Method | Pre-Specified Threshold |
|---|---|---|
| Pearson correlation | r between log10(Lume) and log10(Colilert) on holdout set | r ≥ 0.80 [proposed; to be confirmed with EPA Region 8] |
| Coefficient of determination | R² on holdout set | R² ≥ 0.65 [proposed] |
| Mean absolute percentage error | MAPE in log-transformed space on holdout set | MAPE ≤ 15% [proposed] |
| Systematic bias | Geometric mean ratio: median(Lume/Colilert) on holdout set. Test for bias using paired t-test on log differences. | Geometric mean ratio between 0.67 and 1.50 (i.e., within a factor of 1.5×); paired t-test p > 0.05 for no significant bias [proposed] |
| Limits of agreement | Bland-Altman analysis: mean difference ± 1.96 SD of differences (log10 scale) | 95% of differences within ±1.0 log10 unit (factor of 10×); reported for transparency [proposed] |
| Formal equivalence test | Two one-sided t-tests (TOST) on log10 differences. Equivalence margin: ±0.5 log10 units (≈ factor of 3×; comparable to Colilert’s own inter-lab variability of 20–30%) | Both one-sided tests significant at p < 0.05 within ±0.5 log10 equivalence margin [proposed] |
Binary classification performance at the Regulation 93 geometric mean threshold of 126 CFU/100 mL, evaluated on the holdout test set.
| Metric | Definition | Pre-Specified Threshold |
|---|---|---|
| Sensitivity (true positive rate) | P(Lume ≥126 | Colilert ≥126) | ≥0.85 [proposed — protective; missed exceedances are the higher public health risk] |
| Specificity (true negative rate) | P(Lume <126 | Colilert <126) | ≥0.80 [proposed] |
| Cohen’s kappa | Agreement beyond chance on binary classification | ≥0.70 [proposed — "substantial agreement"] |
| Overall accuracy | (TP + TN) / N | ≥0.85 [proposed] |
| Area under ROC curve | AUC from continuous Lume predictions vs. binary Colilert threshold outcome | ≥0.80 [proposed] |
| False negative rate | P(Lume <126 | Colilert ≥126) | ≤0.15 [proposed — complement of sensitivity; protects against under-reporting exceedances] |
Classification performance will also be evaluated at the Reg 93 single-sample maximum (235 CFU/100 mL) and reported as supplementary analyses. Confusion matrices for each site and season will be reported in the full comparability report.
| Analysis | Data Source | Metric |
|---|---|---|
| Temporal within-sensor precision | Readings during stable baseflow periods (identified post-hoc by <5% CV over 24 hours) | Coefficient of variation (CV) of consecutive readings; characterizes instrument noise floor vs. real environmental signal. |
| Reference method precision | Field duplicate Colilert samples (every 10th sample) | RPD between field duplicates. Documents Colilert’s own measurement uncertainty at Boulder Creek, providing context for the equivalence margin. |
All primary and secondary endpoint analyses will be repeated for the following stratifications, reported as supplementary analyses in the comparability report:
All statistical analyses will be performed in R (version ≥4.3) or Python (scikit-learn, scipy, statsmodels). Analysis code will be version-controlled and provided to EPA Region 8 upon request. All results will be reproducible from the archived raw data using the provided code. No analyses will be performed in Excel alone; Excel may be used for data entry and visualization only.
The instrument detection limit for TLF will be determined per 40 CFR Part 136 Appendix B: minimum 7 replicate measurements of a tryptophan standard near the expected detection limit, run in both laboratory (controlled tryptophan solution in DI water) and field (low-concentration upstream reference site BC-1) conditions. MDL = t-value (α = 0.01, n−1 degrees of freedom) × standard deviation of replicates.
The TLF MDL will be translated to an E. coli MDL in CFU/100 mL by applying the fitted prediction equation. The method reporting limit (MRL) will be set at 3× the MDL, consistent with standard analytical chemistry practice. Colilert results below the Lume MRL will be reported as <MRL in the paired dataset and treated as left-censored in regression analyses.
The MDL and MRL will be compared against the Reg 93 thresholds (126 CFU/100 mL geometric mean; 235 CFU/100 mL single-sample maximum) and the existing literature MDL of approximately 10 CFU/100 mL for TLF-based methods. The method must demonstrate that the MRL is substantially below the regulatory threshold to support the compliance monitoring claim.
| Interference | Mechanism | Mitigation in This Method |
|---|---|---|
| Dissolved organic carbon (DOC) / humic-like fluorescence | Humic and fulvic acids can fluoresce at wavelengths overlapping the TLF signal, causing false positive E. coli estimates | Partially mitigated by turbidity correction in the prediction equation. The magnitude of humic interference at Boulder Creek is characterized during the study by comparing TLF at low-E. coli / high-DOC conditions (e.g., late fall leaf litter input) against Colilert results. |
| High turbidity | Very high turbidity (>[threshold NTU]) can attenuate fluorescence signal (inner filter effect) or scatter light into detector | Turbidity is included as a model predictor. Readings above the identified interference threshold are flagged (Section 7.2). Performance at high turbidity conditions is characterized separately. |
| Temperature | TLF signal intensity is temperature-dependent; fluorescence efficiency decreases with increasing temperature | Temperature included as a model predictor. Study spans all four seasons, ensuring temperature effects are represented in the calibration dataset. |
| Algal fluorescence | Phytoplankton can produce fluorescence signals in the TLF spectral region, particularly during algal blooms | Algal events documented in field conditions log. Performance during identified algal bloom periods analyzed separately. Boulder Creek turbid spring runoff typically limits algal growth. |
| Biofouling | Biological growth on the optical window attenuates signal over time | Bi-weekly cleaning protocol (Section 7.1); fouling detection via daily drift monitoring (Section 7.2); maintenance rotation minimizes gaps. |
| Chlorinated effluent | WWTP effluent may carry chlorine residual that partially inactivates E. coli but does not immediately destroy TLF signal, causing over-prediction | Site BC-3 (below WWTP) receives residual-chlorinated effluent intermittently. Performance at BC-3 analyzed separately; any systematic bias at this site documented and disclosed. |
The study clock does not start until all prerequisites are met. Paired data collected before these conditions are satisfied does not count toward the regulatory dataset.
Confirm certified reference lab; confirm Colilert version; deploy all 6 sensors; submit study design to EPA Region 8; receive written acknowledgment; define study Day 0.
Continuous Lume data + 3×/week Colilert grabs at all 6 sites. Monthly QC checks. Event-based enhanced sampling. Quarterly interim reports to EPA Region 8 and CDPHE. All data to training partition.
Fit OLS regression to training data. Evaluate model acceptance criteria (Section 8.3). Lock coefficients. Report locked equation to EPA Region 8 before holdout evaluation begins. Holdout set remains unseen until this step is complete.
Continue Colilert + Lume paired collection. At Month 12, evaluate locked model on holdout set against pre-specified SAP thresholds (Section 9). Holdout evaluation is blinded — model coefficients are fixed, analysis plan is fixed.
Compile full statistical analysis report on holdout set. Draft Boulder Creek site-specific comparability report. Prepare complete 40 CFR 136.5 application package: method documentation, comparability report, QA/QC records, SAP results.
City of Boulder submits limited-use ATP application to EPA Region 8 Administrator.
| Deliverable | Recipient | Timing |
|---|---|---|
| Study design document (this document) | EPA Region 8, CDPHE, City of Boulder | Pre-study |
| Quarterly interim data reports | EPA Region 8, CDPHE | Months 3, 6, 9 |
| Model fitting report (locked coefficients) | EPA Region 8 | Month 9 |
| Full statistical analysis report (holdout) | EPA Region 8, CDPHE, City of Boulder | Month 13 |
| Boulder Creek comparability report | EPA Region 8, CDPHE | Month 14 |
| Complete method documentation (EPA format) | EPA Region 8 | Month 15 |
| 40 CFR 136.5 application package | EPA Region 8 | Month 16 |
The following questions require answers before the study clock can start. They are directed to EPA Region 8, CDPHE, and City of Boulder respectively. Answers will be incorporated as amendments to this document.
E-1. Study design pre-registration: Will EPA Region 8 provide written acknowledgment of this study design document prior to data collection, confirming that data collected under this protocol will be accepted for the 136.5 application? What is the formal process for pre-registering a study design under the ATP program?
E-2. Statistical acceptance thresholds: The thresholds in Section 9 are proposed by Virridy based on existing performance data and comparable method validation precedents. Will EPA Region 8 confirm, modify, or substitute these thresholds before data collection begins? Which metrics are most important to EPA for the primary regulatory claim (Option A — direct replacement)?
E-3. Independent operator requirement: Under 40 CFR 136.5, is it acceptable for Virridy (as City of Boulder’s contractor) to operate the Lume sensors during the study period, or must City of Boulder staff serve as the independent operator? What level of Virridy involvement is permissible?
E-5. Continuous-to-discrete compliance mapping: Virridy proposes that the Lume reading nearest in time to the scheduled Reg 93 compliance sampling event (within ±15 minutes) constitute the official compliance value, replacing the Colilert grab sample on a 1-for-1 basis. Does EPA Region 8 accept this approach, or does it require a different aggregation method (e.g., geometric mean over a defined window)?
E-6. ML / regression model documentation: The Lume uses a multivariate linear regression equation (not machine learning). EPA has not previously evaluated a continuous in-situ sensor using a regression model as a determinative step. Is the proposed model documentation approach (published equation with fixed coefficients, as described in Sections 3.2 and 8.4) acceptable? Are there additional documentation or transparency requirements?
E-7. Minimum study duration: Is a 12-month study duration required, or will EPA Region 8 consider a 9-month study supplemented by the existing 856-observation dataset for a limited-use 136.5 approval? What seasonal coverage is the minimum requirement?
E-8. Method document format: There is no existing EPA template for continuous in-situ sensor methods. Virridy will prepare the method documentation following the format of existing EPA microbiological methods (e.g., EPA Method 1603), adapted for continuous monitoring. Please confirm this approach is acceptable, or advise on any Region 8-specific formatting preferences.
C-4. Colorado-specific certification requirements: Beyond the federal ATP approval, are there any Colorado-specific certification, approval, or rulemaking steps required before City of Boulder can use Lume data for official Reg 93 compliance reporting?
B-1. Certified reference laboratory: Which state-certified laboratory (CLCP-certified for E. coli analysis) currently analyzes Boulder’s official Reg 93 Colilert samples? Is this laboratory willing and able to increase sampling frequency to a minimum of 3 times per week at all 6 monitoring sites for the 12-month study duration? What is the cost differential and is it covered under the existing contract or study budget?
B-2. Colilert version: Does Boulder’s existing Reg 93 monitoring program use standard IDEXX Colilert (24-hour, 35 °C incubation) or IDEXX Colilert-18 (18-hour, 44.5 °C incubation)? The study must use the same version as Boulder’s existing compliance program for comparability.
B-3. Existing chain-of-custody and SOPs: Can Boulder’s existing Colilert sampling SOPs and chain-of-custody documentation be used as-is for the study, or do they need to be upgraded to meet ATP validation standards?