Independent validation of a model using cell line chemosensitivity to predict response to therapy.

{"sci_version": "1.2", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:23964133", "kind": "science-record", "revision": 2, "content_hash": "sha256:ac4ec37374063186108dc90262923d7dfbcb60cf969abb328877e7e740d98a83", "created_at": "2026-11-01T15:40:03Z", "updated_at": "2026-11-01T18:46:56Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "Independent validation of a model using cell line chemosensitivity to predict response to therapy.", "doi": "10.1093/jnci/djt202", "pmid": "23964133", "authors": ["Wenting Wang", "Keith A Baggerly", "Steen Knudsen", "Jon Askaa", "Wiktor Mazin", "Kevin R Coombes"], "journal": "Journal of the National Cancer Institute", "year": 2013, "abstract": "BACKGROUND: Methods using cell line microarray and drug sensitivity data to predict patients' chemotherapy response are appealing, but groups may be reluctant to release details to preserve intellectual property. Here we describe a case study to validate predictions while treating the methods as a \"black box.\"\nMETHODS: Medical Prognosis Institute (MPI) constructed cell-line-derived sensitivity scores (SSs) and combined scores (CSs) that incorporate clinical variables. MD Anderson researchers evaluated their predictions. We searched the Gene Expression Omnibus (GEO) to identify validation datasets, and we performed statistical evaluation of the agreement between prediction and clinical observation.\nRESULTS: We identified 3 suitable datasets: GSE16446 (n = 120; binary outcome), GSE17920 (n = 130; binary outcome), and GSE10255 (n = 161; continuous and time-to-event outcomes). The SS was statistically significantly associated with primary treatment responses for all studies (GSE16446: P = .02; GSE17920: P = .02; GSE10255: P = .02). Dichotomized SSs performed no better than chance for GSE16446 and GSE17920, and categorized SSs did not predict disease-free survival (GSE10255). SSs sometimes improved on predictions using clinical variables (GSE16446: P = .05; GSE17920: P = .31; GSE10255: P = .045), but gains were limited (95% confidence intervals for GSE16446 and GSE17920 include 0). The CS did not predict treatment response for GSE16446 (P = .55), but it did for GSE17920 (P < .001). Coefficients of clinical variables provided by MPI for CSs agree with estimates for GSE17920 better than estimates for GSE16446.\nCONCLUSIONS: Model predictions were better than chance in all three datasets. However, these scores added little to existing clinical predictors; statistically significant contributions were likely to be too small to change clinical practice. These findings suggest that discovering better predictors will require both cell line data and a clinical training dataset of patient samples.", "claims": [{"id": "claim-target-result", "statement": "The model predictions were better than chance in all three datasets but added little to existing clinical predictors, with statistically significant contributions likely too small to change clinical practice.", "kind": "qualitative", "target": null, "evidence": [{"asset": "source:pmid:23964133", "locator": "sections[0] \"Abstract\" paragraph 4", "quote": "Model predictions were better than chance in all three datasets. However, these scores added little to existing clinical predictors; statistically significant contributions were likely to be too small to change clinical practice."}, {"asset": "source:pmid:23964133", "locator": "sections[0] \"Abstract\" paragraph 2", "quote": "We searched the Gene Expression Omnibus (GEO) to identify validation datasets, and we performed statistical evaluation of the agreement between prediction and clinical observation."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "The model predictions were better than chance in all three datasets but added little to existing clinical predictors, with statistically significant contributions likely too small to change clinical practice.", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:23964133"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:23964133", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1093/jnci/djt202", "sha256": "sha256:52ad6431ebf800d4963f7637ac96084e155a1a2b0fedd36defce8e606b3fca54", "bytes": 1986, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1093/jnci/djt202", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC3955959", "source": "ncbi_pmc_fulltext", "coverage": "whole_body", "text_hash_scope": "rendered_section_body"}}], "gaps": []}, "analysis": {"state": "absent", "methods": [], "targets": [], "gaps": ["method", "operations"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R4", "expressible": false, "required_operations": [], "missing_operations": [], "basis": "The reader quoted the method the paper names (not named), but the corpus's method extractor could not map it onto a DSL operation, so the method is published but not expressible."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": [], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:23964133"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-10-31T18:21:30Z", "independence": "single_reader"}, "evidence_check": {"method": null, "availability": null}, "calls": 2, "usage": {"prompt_tokens": 1842, "completion_tokens": 467, "cached_tokens": 520}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:23964133", "type": "paper", "label": "pmid:23964133"}, {"id": "source:pmid:23964133", "type": "manuscript", "label": "PMID 23964133 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:23964133", "type": "paper", "label": "pmid:23964133"}, {"id": "source:pmid:23964133", "type": "manuscript", "label": "PMID 23964133 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-23964133", "type": "extraction", "used": ["source:pmid:23964133"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:23964133", "parent_hash": "sha256:87dbba4f538d29a829c4e60a292a0164737de6cdbe88a18a7356b4c0057ab3f7", "fork_of": null, "generation": 1}, "gaps": ["execution.not_run", "method", "operations", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent"], "assays": [{"role": "data_generation", "definition_id": "microarray", "definition_version": "99e1195c7d6f", "definition_digest": null, "required_operations": [], "support_state": "partial", "source_hashes": {"assay_lexicon": "99e1195c7d6f"}, "confidence": 0.93, "evidence": ["BACKGROUND: Methods using cell line microarray and drug sensitivity data to predict patient"], "order_basis": "paper_text_order:99", "id": "assay:1"}]}