Geno diagnosis accuracy
Every AI plant-diagnosis tool claims accuracy. This is Pheno's — measured on a labeled, versioned benchmark, published per release, with the full methodology and known failure modes. Receipts, not claims.
Benchmark in progress
The labeled corpus is being assembled.
We publish measured scores here per release — not marketing claims. When the first benchmark run is blessed, current and per-release scores appear below. The methodology is final and published today.
Methodology
- What we measure. Geno's live photo-diagnosis path is run over a curated, labeled image set. Each image has one primary ground-truth condition — a category (one of 12) plus a free-text condition and severity.
- Sourcing & labeling. Images are first-party captures, community submissions with permission, or openly-licensed datasets — each with documented provenance. At least two cultivation-literate labelers apply labels; disagreements are reconciled and recorded (inter-rater discipline).
- Contamination guard. Pheno does not train models — it calls a hosted model API. The eval set is a separate corpus, stored apart from user data, and is never injected into the diagnosis prompt. That isolation is enforced in the test suite, so "never used in prompting" is provable, not asserted.
- Scoring. A per-condition score is recall: of the labeled images for a condition, the share Geno correctly identifies (category coarse, condition title fine). Healthy images measure the false-positive rate — Geno inventing a problem.
- Calibration. Confidence is categorical (high/medium/low), so "calibration" here means accuracy within each confidence bucket — not a probabilistic score.
- Versioned & per-release. Every score is stamped with a diagnosis-config version (model + prompt). A model or prompt change is a new release with its own benchmark run; a regression gate blocks releases that drop accuracy beyond a threshold.
Known failure modes & limits
- Fuzzy condition matching. Geno's finding titles are free text, matched to the labeled condition by category + salient tokens. The rule is documented; edge cases can mis-match.
- Run-to-run variance. Vision calls are nondeterministic (adaptive reasoning). Treat each score as a distribution, not a fixed point; the regression threshold leaves room for it.
- Small-sample conditions. Conditions with few labeled images carry wide error bars and are shown with their sample size (n) so a small number can't masquerade as precision.
- Confounders. pH lockout mimics multi-nutrient deficiency; spider-mite stippling mimics magnesium; light burn mimics nutrient burn. The set deliberately includes these so precision is honest.
- Coverage. A condition not in the benchmark has no score — the app says "not yet measured" rather than implying one.