We have made some significant updates to the BRAKES platform, with two major releases of functionality. The first lets a regulator test BRAKES itself, against its own data, without either side disclosing what it needs to protect. The second, BRAKES-Strata, identifies a pre-specified subgroup that is suppressing a trial’s result and quantifies what excluding it would recover. 

Verify it yourself: mass-testing BRAKES against your own data

In our response to FDA’s Request for Information on AI-Enabled Optimization of Early-Phase Clinical Trials (Docket No. FDA-2026-N-4390), we argued that a regulator should not have to take a vendor’s performance claims on trust. Key performance characteristics ought to be verifiable by the Agency itself, in silico, against data with established ground truth, at decision points of the Agency’s own choosing, before anything is deployed on a live trial.

We are delighted to say we have now released a tool for exactly this. Regulators can be issued with their own BRAKES accounts, have specific sponsor trials assigned to them, generate their own trial datasets, and run mass simulations against the exact configuration and algorithm version deployed on that trial. The regulator holds the ground truth throughout. We never receive it, and the platform cannot score itself against it.

Why vendor validation isn’t enough

The claim we make about BRAKES is a claim about error rates: that it detects failing trials substantially earlier than a single-midpoint interim analysis, and that our own mass testing shows a false-negative rate roughly five times lower than conventional methods. That is exactly the kind of claim a reviewer should be able to verify independently rather than accept. 

The fix is straightforward in principle: let the regulator run the test. The obstacle has always been that doing so appears to require one of two disclosures neither side wants to make. Either we hand over the algorithm, or the regulator hands over its evaluation data and the party being tested ends up holding the answer key. Neither is necessary.

What “zero trust” means here

The verification is zero trust in both directions.

The regulator learns only what is needed about the algorithm. BRAKES executes inside its secure enclave, exactly as it does on a live trial. The regulator sees inputs, outputs and configuration, not the code, and not the internals of the futility or success determination.

We learn nothing about the answer key. The data generator produces two files: the dataset that gets uploaded, and a separate ground truth file recording how each simulated trial would in fact have ended. Only the first is ever submitted to the platform, and it can be truncated to reveal as little as the regulator chooses, e.g. 40% of the information fraction, or whatever figure they consider a fair test. The ground truth file stays on the regulator's own systems. BRAKES does not receive it, at any point, and therefore cannot be tuned against it, cannot report against it, and cannot mark its own homework.

Scoring happens on the regulator's side of the boundary. That is the whole design.

How it works

1. Trial assignment. Assignment is initiated by the sponsor: a sponsor nominates one of its trials for regulator access, and the assignment is then executed and recorded by Presentient. Regulator admins assign individual users within their own organisation to the trials their organisation holds. Regulator users see configuration-level data only for the trials assigned to them; never patient data, never the sponsor's live outcome uploads. Assignment and unassignment are both written to the sponsor's trial audit log.

2. Dataset generation. Data is generated by yourselves or an open-source R package published on GitHub under the MIT licence that you can use directly. It takes the design parameters required for the trial type, sample size, endpoint type (continuous or time-to-event), arm count, significance level, effect size or hazard ratio, median survival, enrolment period, together with the information fraction to truncate to, and emits two CSVs per fraction, the subject-level outcome data for upload, and the trial-level efficacy or futility classification for each run. Generating up to ten thousand simulated trials.

The second file is the ground truth, and it is worth being precise about what it contains: for each simulated trial, the conclusion a conventional endpoint analysis reaches on the complete dataset, a two-sample t-test for continuous outcomes, a log-rank test for time-to-event, classified as efficacy where p < alpha and futility otherwise. That is the comparator an early call has to be judged against, and it is computed on the regulator's machine, not ours.

One call can produce several truncation levels at once, which turns a binary question into a more useful one: not merely whether BRAKES reaches the right conclusion, but how little of the accumulating data it needs before it does.

Publishing the generator openly is the point. A reviewer can read it line by line, interrogate the distributional assumptions, alter the effect size or direction of benefit, or fork it and write something they trust more. There is no version of this feature in which the party being audited controls the data generation and asks to be believed about it. Three interfaces are provided that can be used, an R function, a command-line script, and a Shiny app for reviewers who would rather complete a web form than write R.

3. Truncated execution. The regulator uploads the truncated dataset and runs the simulation. Data is submitted in chunks at each projection slot, mimicking how a sponsor's data arrives during a live trial, and evaluated by the same algorithm version and configuration as the sponsor deployment. 

4. Comparison, on your side of the line. The results view shows the outcome of every analysis package for every run, what BRAKES concluded, and when. At the moment those results are displayed, the platform has no knowledge of what the correct answer was. The regulator downloads the results as a CSV and compares it against the ground truth file the generator produced in step 2, computing false-stop rate, missed-stop rate, decision timing against a conventional interim, or any other statistic they consider material.

Every regulator action is audited and exportable. Simulated trials are stored under the regulator’s own account, never the sponsor’s: a sponsor cannot see a regulator’s runs or results, by construction rather than by permission setting.

BRAKES-Strata: when one subgroup is holding the trial back

BRAKES now has a third analysis package.

BRAKES-Success tells you when a trial has already made its case. BRAKES-Futility tells you when it is not going to. BRAKES-Strata answers a different question: is one pre-specified subgroup responding so poorly that it is suppressing the result for everyone else, and by how much would the probability of success improve if that subgroup were dropped from recruitment?

The package identifies an underperforming stratum, quantifies the effect its inclusion is having, and surfaces the gain available from excluding it. What it does not do is decide anything. The stratum, the threshold and the consequent action are all fixed in the statistical analysis plan before any data is unblinded. BRAKES tells you that a decision point you defined has been reached at exactly the right moment.

Why we built it

Adaptive design is the most consistent request we get from prospective clients, and it is almost always the same specific version of it: the ability to detect a subgroup that responds poorly to treatment, early enough to steer recruitment toward subgroups that respond well.

The value is not only within the trial in front of you. A stratum identified as underperforming in phase 2 is an input to the phase 3 design, a narrower, better-targeted population, a higher likelihood of success, and for a biotech, a materially stronger position going into a follow-on round. Finding it late can cost a phase 3 from starting, pushing back development times or stopping the development process in its tracks. 

Pre-specification is the whole design

Subgroup analysis has a deservedly poor reputation, and for a good reason: run enough cuts of the data after the fact and one of them will look encouraging. So the constraints we have built in are structural:

Only strata locked in the SAP can be displayed. Stratum definitions are registered at configuration and fixed. There is no facility for defining a new stratum mid-trial and looking at it, because that is post-hoc subgroup analysis with extra steps.

A single stratum of interest, named in advance. The trial declares which stratum it is watching. Not whichever one turns out to look the worst.

The threshold is set before the data arrives. A Success Delta Percentage is configured up front: the improvement in probability of success that, if achieved by excluding the stratum of interest, constitutes a flag. BRAKES applies the rule you wrote.

Nothing is visible until the flag is raised. Projection data for the strata package does not become available until the stratum has actually been identified as underperforming. 

And not before there is enough data to justify it. Projections run only once the configured minimum information fraction for the stratum of interest has been met. 

The consequence of all this is that BRAKES-Strata is not making a decision and is not designed to. It highlights the decision point attached to a pre-specified change, typically a DMC meeting. Success and futility calculations are not silently rewritten in the background, because a change in recruitment is a protocol matter, not a platform matter.

What you see

Two views.

Whole trial view: the trial as it stands, all strata included. Violin plots show the distribution of outcome data for the control and treatment arms, the y-axis carrying the quantified outcome and the x-axis the density of participants at each value, with quartile overlays. The projection chart shows the trial's progression to the current point with current probability of success, p-value, and effect size, hazard ratio or odds ratio as appropriate. If the stratum of interest has crossed the underperformance threshold, that is stated plainly here.

Exclusion view: the same picture with the stratum of interest removed. Updated effect estimate, p-value, effective sample size, revised probability of success, and the headline number: the percentage-point gain in probability of success attributable to excluding that stratum.

Projection view: excluding 'High' stratum raises PoS from 26.5% to 62.0%: STRATA_ADAPTATION flagged

Availability

Regulator accounts are provisioned by Presentient, and the data generation package is available now on GitHub under the MIT licence. BRAKES-Strata is standalone; it does not require BRAKES-Success or BRAKES-Futility on the same trial, though it sits naturally alongside them.

To talk to us about evaluating BRAKES under your own conditions, or about how BRAKES-Strata could apply to a trial you are running, please get in touch using the contact form below.