Vecura Biotech Insiders Spotlight #16: The Score at the Top Was One Sample Out of Ten
For Vecura Biotech Insiders Spotlight #16, Darren Hao shares the campaign, the full set of structures, and what they establish about where these two molecules actually bind.

Designing nanobodies against PSMA, and what thirty predicted structures said that two did not
Two nanobody designs came out of this campaign, and thirty predicted structures were produced to test them. Read the top-level score of each job and the two candidates look different from each other. Open every structure underneath those scores and they do not.
A structure prediction job returns one headline confidence value, and that value belongs to a single sample. When the job produced ten, the headline is a sample statistic being read as a job result, and nothing in the output marks the difference. The reported numbers were never wrong. They were answering a narrower question than the one being asked of them, which is harder to notice than an incorrect value.
The target and the design pass
Prostate-specific membrane antigen is a cell-surface target in prostate cancer. The campaign began from PDB 5O5T, a 1.43 Å human PSMA structure, which carries a caveat worth stating early: it is bound to the small-molecule inhibitor PSMA-1007 and is not a PSMA to nanobody complex. It gives the shape of the protein in that state and says nothing about which surfaces are reachable on native PSMA, which is glycosylated, dimeric and membrane-presented. A surface assessment shortlisted three residue-defined patches, and RFantibody generated ten backbone designs against each on a supplied h-NbBCII10 framework.
Figure 1. The campaign path. Patch 2 stopped after backbone design, for a reason worth recording.
Patch 2 was the largest of the three at 21 residues and the only one that did not reach the downstream workflow, for a geometric reason rather than an administrative one. It spans two segments roughly 90 residues apart in sequence, around 614 to 625 and around 710 to 727. The backbone designs reached the second segment and largely ignored the first, producing interfaces of only three to six target residues, the smallest of the three patches by a clear margin. A patch defined across two distant segments is a specification the designer has to bridge, these designs did not bridge it, and a large patch with a small achieved interface is worse than a small patch with a complete one. That is a useful result rather than a dead end, because it says something about what this design method will do with a discontinuous epitope definition.
The validator and the generator were related
Darren Hao carried two Patch 1 sequences forward, both 114 residues. An in-house RF2 refold scored both designs strong, with an interaction predicted alignment error of 4.4 to 4.9 Å, and that result carries less weight than it appears to.
RF2 and RFantibody descend from the same modeling lineage and share assumptions about what a plausible protein interface looks like. When a generator from that family proposes a pose and a validator from that family approves it, the approval may be reporting internal consistency rather than independent support. The two can be confidently wrong in the same direction, because they learned to be wrong in the same direction. A validator that shares the generator's assumptions is a consistency check wearing a validator's clothes.
So the question falls to Boltz-2.1 and Chai-1, two co-folding models of a different lineage from RFantibody. Both were run on both candidates with the same settings, so that any difference found between the two molecules is a difference in the molecules rather than in how closely each was examined.
A low interface score means two different things
Boltz-2.1 returned interface confidences below the rough working threshold of 0.6 for both candidates, which on its own decides nothing. A weak interface confidence is consistent with two quite different situations: the model places the binder somewhere entirely different from the designed site, which indicates a design that does not reproduce, or it places the binder on the right epitope but is unsure of the exact register, which indicates a plausible binder with an uncertain pose. Those call for opposite decisions and the score cannot separate them.
One step separates them. Superpose the predicted complex onto the original design using the target, then check whether the VHH is still sitting on Patch 1.
| C1, des_5_design_39 | C2, des_9_design_32 | |
| PSMA alignment quality | 0.77 Å RMSD | 0.82 Å RMSD |
| VHH displacement after superposition | 18.7 Å RMSD, 11.9 Å centroid | 33.4 Å RMSD, 32.6 Å centroid |
| Patch 1 hotspots still contacted | 7 of 11 | 2 of 11 |
| Which hotspots | 232, 237, 317, 318, 320, 322, 324 | 317, 318 |
| VHH residues contacting PSMA | 14 | 22 |
Table 1. Sample 0 of each Boltz-2.1 job, superposed on the design. This is the structure the headline score describes.
On these two structures the candidates split cleanly. C1 is the second case, shifted about 12 Å and still on the epitope, contacting seven of the eleven Patch 1 residues where the original design contacted four. C2 is the first, relocated about 33 Å with two hotspots left. That is where a reading built on one structure per candidate stops, and it is not where the evidence stops.
Five Chai-1 seeds per candidate
Darren Hao ran Chai-1 five times on each candidate, with identical sequences and settings and only the random seed changed. All ten structures were superposed on the design and contact-analyzed on a 5 Å heavy-atom criterion. The residue numbering was checked rather than assumed, since the design numbers this target as canonical PSMA plus 60 while the Chai-1 target begins at canonical residue 55.
| Seed | C1 centroid shift | C1 hotspots | C2 centroid shift | C2 hotspots |
| 42 | 45.9 Å | 0 of 11 | 32.8 Å | 1 of 11, Ser318 |
| 500 | 43.3 Å | 0 of 11 | 37.9 Å | 0 of 11 |
| 1,000 | 16.6 Å | 1 of 11, Ser318 | 38.1 Å | 0 of 11 |
| 10,000 | 42.5 Å | 0 of 11 | 57.1 Å | 0 of 11 |
| 100,000 | 43.6 Å | 0 of 11 | 35.7 Å | 0 of 11 |
Table 2. All ten Chai-1 runs, superposed on PSMA. Target alignments were 0.37 to 0.77 Å RMSD, so misalignment does not explain the displacements.
Neither candidate recovers its designed interface in any run. For C1, seeds 500, 10,000 and 100,000 form a tight off-epitope cluster with centroids within about 1.3 Å of one another, contacting PSMA around canonical residues 605 to 633, while seed 1,000 sits some 26 Å from that cluster. For C2, seeds 500, 1,000 and 100,000 group within 1.5 to 3.0 Å around residues 605 to 623, and seed 10,000 lands on a distinct surface roughly 73 Å away at residues 121 to 144. Both VHHs fold well throughout, at mean pLDDT near 89 with over 95 percent of residues above 70, and both still dock somewhere on PSMA. Neither docks where it was designed to.
Ten Boltz-2.1 samples per candidate
Each Boltz-2.1 job was submitted for ten samples and returned ten. The values quoted earlier, 0.431 and 0.485, describe sample 0 of each job, and the rest of each ensemble describes something else.
| C1, des_5_design_39 | C2, des_9_design_32 | |
| Samples produced | 10 | 10 |
| Mean structure confidence | 0.819 | 0.833 |
| Mean ipTM | 0.291 | 0.324 |
| ipTM range across samples | 0.224 to 0.431 | 0.187 to 0.485 |
| Top-level ipTM, sample 0 | 0.431 | 0.485 |
| Largest pose cluster | 8 of 10 samples | 4 of 10 samples |
| Is sample 0 in that cluster | no, about 31 Å away | yes |
Table 3. The Boltz-2.1 ensembles. For each candidate the headline value is the highest in its range, and for C1 it belongs to an outlier.
For C1, samples 1 to 7 and 9 converge tightly on one orientation, with a median pairwise Cα RMSD of 3.16 Å after aligning their PSMA chains, repeatedly contacting a region that includes Arg127, Lys153, Asp262 and Glu488. Samples 0 and 8 sit roughly 31 Å and 28 Å from that group, and sample 0 is the structure in Table 1. For C2 the pattern is different and in one respect worse: samples 0 to 3 share a surface with Jaccard overlap of 0.56 to 0.77, samples 6 to 8 cluster on a second surface near Arg127 to Lys139 and Gly257 to Arg266, and samples 4, 5 and 9 go elsewhere again. Four of ten is not convergence on an interface.
The 7-of-11 result for Candidate 1 was measured on the single sample out of ten that its own ensemble does not reproduce. A pose 31 Å away from that sample is not sitting on the same eleven residues.
There is a second lesson inside the same table. For C2, ranking by ipTM picks a sample from the dominant cluster. For C1, ranking by ipTM picks the outlier, because sample 0 carries the highest ipTM in its job at 0.431 while the eight-sample cluster runs from 0.224 to 0.336. The same ranking rule selected a representative pose in one job and an unrepresentative one in the other, and nothing on the face of either result says which case applies.
Where that leaves the two candidates
For C2 the answer is consistent across everything run. Ten Chai structures put it off Patch 1 and the Boltz ensemble does not settle on any single interface, so it is not a Patch 1 candidate, which is what it was built to be. That does not rule out binding somewhere on PSMA.
For C1 the on-epitope result has lost its support without being replaced by a clean rejection. The Boltz ensemble's contacts were recorded in that job's own chain numbering and have not been transferred to the canonical frame in which the Patch 1 hotspots are defined, so the recurring Boltz pose has not been tested against Patch 1. It has only been shown to lie far from the one sample that was.
That mapping is worth doing for a reason beyond bookkeeping. The Chai ensembles for both candidates cluster around canonical residues 605 to 633. If the recurring Boltz surfaces map into the same region, then two independent architectures have converged on a common alternative epitope and the campaign has found a reproducible binding site that nobody designed for. If they map elsewhere, the predictions are simply unstable. Those are different results, and the analysis that separates them is an afternoon's work on files that already exist.
Two reasons to hold this loosely
The yardstick is a patch definition rather than a design. The eleven Patch 1 hotspots are the surface the campaign chose to target, and the RFantibody design itself contacts only four of them, namely Pro237, Ser317, Arg320 and Ser322. Reporting a prediction as 0 of 11 is a stricter test than asking whether it reproduced the design, and the counts should be read with that denominator in mind.
Co-folding models are also being asked a question they were not trained on. Boltz-2.1 and Chai-1 learned from natural complexes, and their accuracy on de novo designed binders is an open question in the field rather than a settled strength. A design these models cannot place is a design that has not been computationally corroborated. It is not a design that has been shown not to bind, and none of these scores is calibrated against measured affinity. No binding-affinity prediction was requested in either Boltz job and none appears in the output.
What each assessment could answer
Eight distinct computational assessments were run on these two sequences and they do not carry equal weight on the question being asked.
Figure 2. The assessments arranged by what they can establish, with the family relationship and the unmapped samples marked.
The five sequence-level predictors never see the target at all. They answer whether a molecule would be workable if it bound, which is a real question and a different one. Only the interface-geometry band can say whether a binder reached Patch 1, and it currently covers twelve of the thirty predicted complexes.
| Assessment | C1 | C2 | Favors |
| Boltz-2.1 mean ipTM, ten samples | 0.291 | 0.324 | ranges overlap, no separation |
| Chai-1 mean confidence, five seeds | 0.422 | 0.349 | C1 |
| NetSolP-1.0 predicted solubility | 0.5003 | 0.6074 | C2 |
| NetSolP-1.0 predicted usability | 0.3844 | 0.4280 | C2 |
| BioPhi Sapiens heavy-chain mean | 0.8239 | 0.8197 | effectively tied |
| ImmunoSeq hit rate | 0.56 | 0.5352 | no reference value exists |
| Patch 1 hotspots, Boltz sample 0 | 7 of 11 | 2 of 11 | C1, on an outlier sample |
| Patch 1 hotspots, best of five Chai poses | 1 of 11 | 1 of 11 | neither |
Table 4. The full comparison. The two bottom rows are the only ones that test the design objective.
Nothing in that table separates the two molecules in a way that would justify choosing between them. The sequence predictors favor C2 by margins with no stated threshold and no measured anchor. The two interface-confidence means come from different models, neither calibrated against affinity, and their sample ranges overlap heavily. The measurements that test the design objective favor neither. Retaining both candidates is therefore the defensible decision, and it should be recorded as a decision made on insufficient evidence rather than on a balance of it.
What would settle it
Map the recurring Boltz interfaces onto canonical PSMA numbering and test them against Patch 1. It is the one remaining analysis that could change the reading rather than confirm it, the structures are finished, and the same superposition has already been performed twelve times in this campaign.
Then address accessibility, because every structural conclusion here was drawn against an inhibitor-bound monomer, and whether any of these surfaces is reachable on glycosylated dimeric PSMA at a cell surface has not been examined by anything in this campaign.
Then measure binding, against intact cell-surface PSMA with matched negative controls and a purified-protein assay alongside, followed by epitope competition if binding is detected. No affinity, selectivity, expression, solubility or functional result exists for either molecule, and a binding assay answers in an afternoon what no further prediction can answer at all.
The part that transfers
A top-level confidence score describes one sample. If the job produced ten, that score is a sample statistic standing in for a job result, and the gap between those two things does not appear anywhere in the output. In this campaign the gap was the difference between a candidate that held its epitope and a candidate whose best structure was the one its own ensemble contradicted.
The habit worth building is not skepticism about models. It is opening every structure a job returns, superposing the set on the design, and asking whether the ensemble agrees with the number printed above it. Pose recurrence across samples and seeds is a more useful signal than any single confidence value, and it costs nothing beyond the analysis, because the structures have already been generated and paid for.
Agreement between a generator and a validator from the same model family is weaker evidence than it looks. A single independent model that agrees is better and still thin. A single sample drawn from an independent model is thinner still, and that is what the strongest claim in this campaign turned out to rest on.
Scope
All results described here are computational. Neither candidate has been synthesized or tested. Complex predictions, confidence fields, contact counts and sequence-property scores are model outputs and require experimental validation. Contacts are geometric, defined as at least one interchain heavy-atom distance within 5 Å, and are not evidence of a measured interaction. Centroid displacements are measured after superposing the predicted PSMA chain on the design and are not predicted binding errors. Residue numbers quoted from the Boltz-2.1 ensembles refer to chain B of those jobs and have not been transferred to canonical PSMA numbering. PDB 5O5T is an inhibitor-bound human PSMA structure and is not a PSMA to nanobody complex.
立即试用 Vecura。
带上您自己的输入,开始探索 Vecura 的能力。


