🔬 The Scientific Defensibility Dashboard¶
Duration: ~30 minutes | Level: ⭐⭐⭐ Advanced
What makes a protein structure scientifically defensible?¶
This question sits at the heart of every structural biology publication. When researchers deposit a structure in the Protein Data Bank (PDB), it must pass rigorous validation checks before acceptance. The same standards apply to computationally generated structures.
synth-pdb's get_quality_report() applies five independent criteria, each
rooted in a different branch of physical chemistry:
| # | Criterion | Scientific standard | Threshold |
|---|---|---|---|
| 1 | Bond Z-score | Engh & Huber (1991) geometry dictionary | mean < 3.0 |
| 2 | Rotamer quality | Dunbrack backbone-dependent library (1997) | favored > 80% |
| 3 | Ramachandran | MolProbity / Top8000 dataset | outliers < 5% |
| 4 | Potential energy | OpenMM / AMBER14 force field | < 1 × 10⁵ kJ/mol |
| 5 | Hydrophobic burial | Kauzmann Oil Drop model (1959) | burial_ratio ≥ 0.8 |
A structure must pass all five to be is_overall_scientifically_defensible = True.
In this tutorial we generate three structures, each designed to fail a different subset of criteria, demonstrating that the checks are genuinely independent.
import os
import sys
IN_COLAB = 'google.colab' in sys.modules
if IN_COLAB:
# @title Environment Setup
!pip install -q synth-pdb biotite matplotlib numpy scipy py3Dmol openmm
else:
sys.path.append(os.path.abspath('../../'))
print("✅ Environment configured!")
🔧 Setup¶
import os
import sys
try:
import google.colab # noqa: F401
IN_COLAB = True
except ImportError:
IN_COLAB = False
if IN_COLAB:
print('Running in Google Colab — installing dependencies...')
import subprocess
subprocess.run(['pip', 'install', '-q', 'synth-pdb', 'biotite'], check=True)
else:
repo_root = os.path.abspath(os.path.join(os.path.dirname('__file__'), '..', '..'))
if repo_root not in sys.path:
sys.path.insert(0, repo_root)
print(f'Running locally — {repo_root}')
import matplotlib.pyplot as plt
import numpy as np
from synth_pdb.generator import generate_pdb_content
from synth_pdb.validator import PDBValidator
print('All imports successful.')
📐 Step 1: The Five Criteria — Scientific Background¶
Criterion 1 — Bond Z-score (Engh & Huber, 1991)¶
Rainer Engh and Robert Huber analysed 985 high-resolution X-ray structures to establish ideal bond lengths and angles. The Z-score measures how many standard deviations a bond length deviates from this ideal:
$$Z = \frac{\text{observed} - \text{ideal}}{\sigma_{\text{ideal}}}$$
A mean Z-score < 3.0 means no bonds are dramatically wrong.
Criterion 2 — Rotamer Quality (Dunbrack, 1997)¶
Side-chain conformations are not random. Roland Dunbrack built a backbone-dependent rotamer library from thousands of PDB structures — a probability map of which Chi angles are physically preferred. Structures with > 80% favored rotamers are adopting low-energy, experimentally observed conformations.
Criterion 3 — Ramachandran Outliers (MolProbity)¶
The Ramachandran plot (phi/psi dihedral space) has forbidden regions where backbone atoms sterically clash. MolProbity's Top8000 dataset defines outlier boundaries from ultra-high-resolution structures. < 5% outliers is the wwPDB deposition standard.
Criterion 4 — Potential Energy (AMBER14 / OpenMM)¶
The AMBER14 force field computes potential energy as a sum of bonded and non-bonded terms. Negative energies are typical for well-packed structures. Extremely positive energies (> 1 × 10⁵ kJ/mol) indicate severe steric clashes.
Criterion 5 — Hydrophobic Burial (Kauzmann, 1959)¶
The Oil Drop model predicts that hydrophobic residues bury in the core (low SASA) while polar residues remain surface-exposed (high SASA). burial_ratio ≥ 0.8 confirms this pattern. See the companion tutorial The Oil Drop Model for a deep dive.
🧬 Step 2: Three Structures, Three Failure Modes¶
We design each structure to have a specific defect:
| Structure | Design intent | Expected failure |
|---|---|---|
| S1 — Minimized helix | Amphipathic helix, energy minimized | None — should PASS all 5 |
| S2 — Drifted decoy | Same sequence, torsion noise drift=120 |
Ramachandran (backbone scrambled) |
| S3 — Hydrophobic strand | All-hydrophobic beta strand, unminimized | Burial ratio + energy |
print('Generating 3 structures (seed=42 for reproducibility)...')
# S1: energy-minimized amphipathic alpha helix
pdb_s1 = generate_pdb_content(
sequence_str='AAKLLLAAKLLLAAK',
structure='1-15:alpha',
minimize_energy=True,
seed=42,
)
# S2: same sequence, large torsion perturbation scrambles Ramachandran
pdb_s2 = generate_pdb_content(
sequence_str='AAKLLLAAKLLLAAK',
structure='1-15:alpha',
minimize_energy=False,
drift=120.0,
seed=42,
)
# S3: all-hydrophobic beta strand — no polar surface, no minimization
pdb_s3 = generate_pdb_content(
sequence_str='VIVVIVVIVVI',
structure='1-11:beta',
minimize_energy=False,
seed=42,
)
print('S1 — Minimized helix: ready')
print('S2 — Drifted decoy: ready')
print('S3 — Hydrophobic strand: ready')
📊 Step 3: Run get_quality_report() on Each Structure¶
One call, five criteria, a single composite verdict:
structures = [
('S1 Minimized helix', pdb_s1),
('S2 Drifted decoy', pdb_s2),
('S3 Hydrophobic strand', pdb_s3),
]
reports = []
for label, pdb in structures:
r = PDBValidator(pdb).get_quality_report()
reports.append((label, r))
verdict = 'DEFENSIBLE' if r['is_overall_scientifically_defensible'] else 'NOT defensible'
print(f'{label}: {verdict}')
print(f' Bond Z-score : {r["geometric_z_scores"]["mean_bond_zscore"]:.3f}')
print(f' Rotamers %% : {r["rotamer_stats"]["favored_rotamers_pct"]:.1f}')
print(f' Rama outliers%%: {r["ramachandran_stats"]["outlier_pct"]:.1f}')
print(f' Energy kJ/mol : {r["potential_energy_kj_mol"]:.0f}')
print(f' Burial ratio : {r["hydrophobic_burial_ratio"]:.3f}')
print()
🎨 Step 4: The Dashboard Heatmap¶
Each cell shows the raw value and a green (pass) / red (fail) background. Reading down each column shows which criterion fails, and reading across each row shows which structures fail.
Key insight: S2 fails only Ramachandran; S3 fails only burial and energy. The criteria are truly independent — a structure can have perfect geometry but wrong chemistry, or correct secondary structure but bad energetics.
# ── Criterion definitions ────────────────────────────────────────────────
CRITERIA = [
# (display_name, report_key_fn, threshold_fn, format_str, invert_color)
# invert_color=True means LOWER is better
('Bond Z-score\n(< 3.0)',
lambda r: r['geometric_z_scores']['mean_bond_zscore'],
lambda v: v < 3.0, '.3f', False),
('Rotamers\n(> 80 %)',
lambda r: r['rotamer_stats']['favored_rotamers_pct'],
lambda v: v > 80.0, '.0f', False),
('Ramachandran\noutliers (< 5 %)',
lambda r: r['ramachandran_stats']['outlier_pct'],
lambda v: v < 5.0, '.0f', True),
('Potential energy\n(< 1e5 kJ/mol)',
lambda r: r['potential_energy_kj_mol'],
lambda v: v < 1e5, '.0f', True),
('Burial ratio\n(>= 0.8)',
lambda r: r['hydrophobic_burial_ratio'],
lambda v: v >= 0.8, '.3f', False),
]
struct_labels = [label for label, _ in reports]
n_structs = len(reports)
n_criteria = len(CRITERIA)
fig, ax = plt.subplots(figsize=(11, 4.5))
ax.set_xlim(-0.5, n_criteria - 0.5)
ax.set_ylim(-0.5, n_structs - 0.5)
ax.invert_yaxis()
PASS_COLOR = '#d5f5e3' # soft green
FAIL_COLOR = '#fadbd8' # soft red
PASS_EDGE = '#1e8449'
FAIL_EDGE = '#922b21'
for col, (_crit_name, value_fn, pass_fn, fmt, _) in enumerate(CRITERIA):
for row, (struct_label, report) in enumerate(reports):
val = value_fn(report)
passes = pass_fn(val)
bg = PASS_COLOR if passes else FAIL_COLOR
edge = PASS_EDGE if passes else FAIL_EDGE
verdict = 'PASS' if passes else 'FAIL'
rect = plt.Rectangle(
(col - 0.48, row - 0.48), 0.96, 0.96,
facecolor=bg, edgecolor=edge, linewidth=1.5, zorder=1
)
ax.add_patch(rect)
# Value
display_val = format(val, fmt)
if abs(val) > 9999:
display_val = f'{val:.2e}'
ax.text(col, row - 0.13, display_val,
ha='center', va='center', fontsize=10,
fontweight='bold', zorder=2,
color=PASS_EDGE if passes else FAIL_EDGE)
ax.text(col, row + 0.22, verdict,
ha='center', va='center', fontsize=7.5, zorder=2,
color=PASS_EDGE if passes else FAIL_EDGE)
# Axes labels
ax.set_xticks(range(n_criteria))
ax.set_xticklabels([c[0] for c in CRITERIA], fontsize=9)
ax.set_yticks(range(n_structs))
ax.set_yticklabels(struct_labels, fontsize=9.5, fontweight='bold')
ax.tick_params(length=0)
ax.set_title(
'Scientific Defensibility Dashboard\n'
'Green = criterion passes | Red = criterion fails',
fontsize=12, fontweight='bold', pad=12
)
# Verdict column on the right
for row, (_, report) in enumerate(reports):
ok = report['is_overall_scientifically_defensible']
ax.text(n_criteria - 0.5 + 0.65, row,
'ALL PASS' if ok else 'FAILED',
ha='center', va='center', fontsize=9, fontweight='bold',
color=PASS_EDGE if ok else FAIL_EDGE)
ax.text(n_criteria - 0.5 + 0.65, -0.48 - 0.12, 'Verdict',
ha='center', va='center', fontsize=9, style='italic', color='#555')
ax.set_xlim(-0.5, n_criteria + 0.4)
ax.spines[:].set_visible(False)
plt.tight_layout()
plt.show()
🔍 Step 5: Why Each Failure Mode Is Distinct¶
S2 — Drifted decoy: only Ramachandran fails¶
Adding drift=120 perturbs backbone torsion angles far from their ideal values,
scattering phi/psi angles into sterically forbidden regions.
But the bond lengths (determined by Biotite templates, not torsions) remain
correct — so the Z-score still passes. And since the sequence is the same as S1,
the burial_ratio is identical.
Take-away: A Ramachandran failure catches backbone geometry errors that bond-length checks completely miss. Both are necessary.
S3 — Hydrophobic strand: burial and energy fail¶
The all-Val sequence has no polar residues, so mean_polar_sasa ≈ 0 and
burial_ratio ≈ 0. The unminimized extended strand also has severe Van der Waals
clashes (positive energy). Yet the Ramachandran angles for a beta strand are
exactly in the allowed region, and bond lengths are template-correct.
Take-away: Burial ratio catches sequence-level design flaws that are invisible to geometry and backbone checks.
The composite flag is conservative by design¶
A structure passes is_overall_scientifically_defensible only if all five
criteria pass simultaneously. This mirrors wwPDB deposition requirements, where
a structure must clear geometry, rotamer, and Ramachandran checks independently.
📈 Step 6: Per-Criterion Score Comparison¶
A radar/bar view showing all five criterion values normalized to their thresholds:
# Normalise each criterion value to [0, 1] where 1.0 = exactly at threshold
# Values > 1 are above threshold (good for maximising criteria, bad for minimising)
NORM_FUNS = [
('Bond Z-score', lambda r: r['geometric_z_scores']['mean_bond_zscore'] / 3.0),
('Rotamers %', lambda r: r['rotamer_stats']['favored_rotamers_pct'] / 100.0),
('Rama outliers %', lambda r: 1.0 - r['ramachandran_stats']['outlier_pct'] / 100.0),
('Energy / 1e5', lambda r: max(0, 1.0 - r['potential_energy_kj_mol'] / 1e5)),
('Burial ratio', lambda r: r['hydrophobic_burial_ratio'] / 0.8),
]
crit_names = [c[0] for c in NORM_FUNS]
colors = ['#2ecc71', '#e67e22', '#9b59b6']
x = np.arange(len(crit_names))
width = 0.25
fig, ax = plt.subplots(figsize=(11, 4.5))
for i, (struct_label, report) in enumerate(reports):
vals = [norm_fn(report) for _, norm_fn in NORM_FUNS]
bars = ax.bar(x + (i - 1) * width, vals, width,
label=struct_label.strip(), color=colors[i], alpha=0.82)
ax.axhline(1.0, color='#2c3e50', linestyle='--', linewidth=1.0,
alpha=0.6, label='Pass threshold')
ax.set_xticks(x)
ax.set_xticklabels(crit_names, fontsize=10)
ax.set_ylabel('Normalised score (> 1.0 = passes threshold)', fontsize=9)
ax.set_title('All Five Criteria — Normalised to Their Pass Thresholds',
fontsize=12, fontweight='bold')
ax.legend(fontsize=8.5, loc='lower right')
ax.set_ylim(0, max(2.2, ax.get_ylim()[1]))
plt.tight_layout()
plt.show()
print('\nNormalised scores (> 1.0 means criterion passes):')
for struct_label, report in reports:
vals = [norm_fn(report) for _, norm_fn in NORM_FUNS]
print(f'{struct_label.strip()}: {", ".join(f"{n}={v:.2f}" for n, v in zip(crit_names, vals, strict=False))}')
🎓 Key Takeaways¶
The five criteria are orthogonal¶
Each measures a genuinely different aspect of structural quality:
- Geometry (bonds) ← scale: ångström
- Conformation (rotamers) ← rotational degrees of freedom
- Backbone topology (Ramachandran) ← steric clash in phi/psi space
- Thermodynamics (energy) ← kJ/mol force field
- Composition (burial) ← Ų surface area per residue type
A structure that cheats on one dimension will be caught by a different check.
When to use get_quality_report() in practice¶
| Workflow | What to look for |
|---|---|
| Before MD simulation | All criteria should pass; high energy = run minimization first |
| After homology modeling | Check Ramachandran and rotamers in modeled loops |
| Evaluating AlphaFold outputs | Burial_ratio failures in low-pLDDT regions = likely disordered |
| Synthetic dataset generation | Use is_overall_scientifically_defensible as a quality filter |
What to explore next¶
- The Oil Drop Model tutorial — deep dive into criterion 5 (burial ratio)
- The Hard Decoy Challenge — use all five criteria to score a real decoy set
- Co-evolutionary Fitness Landscape (incubator) — scan all single-point mutants through this scorecard to build a structural fitness map