Benchmarking Sepsis Gene Expression Diagnostics Using Public Data.

{"sci_version": "1.2", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:27681387", "kind": "science-record", "revision": 7, "content_hash": "sha256:fb9a552de50a5e5f0e6c3a3b194406ce276b541b66f40828a631d62ed307ec6d", "created_at": "2026-11-01T15:40:20Z", "updated_at": "2026-11-01T21:59:10Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "Benchmarking Sepsis Gene Expression Diagnostics Using Public Data.", "doi": "10.1097/CCM.0000000000002021", "pmid": "27681387", "authors": ["Timothy E Sweeney", "Purvesh Khatri"], "journal": "Critical care medicine", "year": 2017, "abstract": "OBJECTIVE: In response to a need for better sepsis diagnostics, several new gene expression classifiers have been recently published, including the 11-gene \"Sepsis MetaScore,\" the \"FAIM3-to-PLAC8\" ratio, and the Septicyte Lab. We performed a systematic search for publicly available gene expression data in sepsis and tested each gene expression classifier in all included datasets. We also created a public repository of sepsis gene expression data to encourage their future reuse.\nDATA SOURCES: We searched National Institutes of Health Gene Expression Omnibus and EBI ArrayExpress for human gene expression microarray datasets. We also included the Glue Grant trauma gene expression cohorts.\nSTUDY SELECTION: We selected clinical, time-matched, whole blood studies of sepsis and acute infections as compared to healthy and/or noninfectious inflammation patients. We identified 39 datasets composed of 3,241 samples from 2,604 patients.\nDATA EXTRACTION: All data were renormalized from raw data, when available, using consistent methods.\nDATA SYNTHESIS: Mean validation areas under the receiver operating characteristic curve for discriminating septic patients from patients with noninfectious inflammation for the Sepsis MetaScore, the FAIM3-to-PLAC8 ratio, and the Septicyte Lab were 0.82 (range, 0.73-0.89), 0.78 (range, 0.49-0.96), and 0.73 (range, 0.44-0.90), respectively. Paired-sample t tests of validation datasets showed no significant differences in area under the receiver operating characteristic curves. Mean validation area under the receiver operating characteristic curves for discriminating infected patients from healthy controls for the Sepsis MetaScore, FAIM3-to-PLAC8 ratio, and Septicyte Lab were 0.97 (range, 0.85-1.0), 0.94 (range, 0.65-1.0), and 0.71 (range, 0.24-1.0), respectively. There were few significant differences in any diagnostics due to pathogen type.\nCONCLUSIONS: The three diagnostics do not show significant differences in overall ability to distinguish noninfectious systemic inflammatory response syndrome from sepsis, though the performance in some datasets was low (area under the receiver operating characteristic curve, < 0.7) for the FAIM3-to-PLAC8 ratio and Septicyte Lab. The Septicyte Lab also demonstrated significantly worse performance in discriminating infections as compared to healthy controls. Overall, public gene expression data are a useful tool for benchmarking gene expression diagnostics.", "claims": [{"id": "claim-target-result", "statement": "Paired t-tests of validation datasets showed no significant differences in AUCs among the three gene-expression diagnostics for distinguishing non-infectious SIRS from sepsis.", "kind": "qualitative", "target": null, "evidence": [{"asset": "source:pmid:27681387", "locator": "sections[0] \"Abstract\" paragraph 5", "quote": "Paired t-tests of validation datasets showed no significant differences in AUCs."}, {"asset": "source:pmid:27681387", "locator": "sections[0] \"Abstract\" paragraph 6", "quote": "The three diagnostics do not show significant differences in overall ability to distinguish non-infectious SIRS from sepsis, though the performance in some datasets was low (AUC<0.7) for the FAIM3:PLAC8 ratio and Septicyte lab."}, {"asset": "source:pmid:27681387", "locator": "sections[0] \"Abstract\" paragraph 2", "quote": "We searched NIH GEO and EBI ArrayExpress for human gene expression microarray datasets."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "Paired t-tests of validation datasets showed no significant differences in AUCs among the three gene-expression diagnostics for distinguishing non-infectious SIRS from sepsis.", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:27681387"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "blocked", "blocked_on": "method:group_comparison_wilcoxon", "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:run_analysis blocked on method:group_comparison_wilcoxon", "step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:27681387", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1097/CCM.0000000000002021", "sha256": "sha256:ca6246ccc4a7f4e82623152c99e03a21fae534fcfe7a33892d89b159e6cb8b3c", "bytes": 28329, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1097/CCM.0000000000002021", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC5518756", "source": "ncbi_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}], "gaps": []}, "analysis": {"state": "partial", "methods": [{"id": "method-1", "family": "group_comparison", "specific": "wilcoxon", "label": "Group comparison", "dsl_operations": [], "missing_operations": ["group_comparison_wilcoxon"], "expressible": false, "parameters": {}, "data_shapes": ["two_or_more_groups_continuous"], "evidence": {"score": 0.8, "band": "high", "quote": "Wilcoxon"}}, {"id": "method-2", "family": "meta_analysis", "specific": "random_effects", "label": "Meta-analysis / evidence synthesis", "dsl_operations": ["random_effects_pool"], "missing_operations": [], "expressible": true, "parameters": {}, "data_shapes": ["study_level_effect_estimates"], "evidence": {"score": 0.8, "band": "high", "quote": "random effects"}}, {"id": "method-3", "family": "meta_analysis", "specific": "fixed_effects", "label": "Meta-analysis / evidence synthesis", "dsl_operations": [], "missing_operations": ["fixed_effects_pool"], "expressible": false, "parameters": {}, "data_shapes": ["study_level_effect_estimates"], "evidence": {"score": 0.8, "band": "high", "quote": "fixed-effects"}}], "targets": [], "gaps": ["method.group_comparison_wilcoxon"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R1", "expressible": false, "required_operations": [], "missing_operations": ["group_comparison_wilcoxon"], "basis": "The paper names the method it used and the corpus's method extractor maps it onto the DSL catalogue: wilcoxon."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": [], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:27681387"], "steps": []}, "must_build": {"dsl_operations": [{"operation": "group_comparison_wilcoxon", "unlocks_papers": null, "priority": null}], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-10-31T18:29:29Z", "independence": "single_reader"}, "evidence_check": {"method": null, "availability": null}, "calls": 2, "usage": {"prompt_tokens": 12823, "completion_tokens": 797, "cached_tokens": 521}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:27681387", "type": "paper", "label": "pmid:27681387"}, {"id": "source:pmid:27681387", "type": "manuscript", "label": "PMID 27681387 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:27681387", "type": "paper", "label": "pmid:27681387"}, {"id": "source:pmid:27681387", "type": "manuscript", "label": "PMID 27681387 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-27681387", "type": "extraction", "used": ["source:pmid:27681387"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:27681387", "parent_hash": "sha256:6985be8a2ddf8a8b7a588aced110f256e59446f9de24ecbafd75a98f3abd84cb", "fork_of": null, "generation": 6}, "gaps": ["execution.not_run", "method.group_comparison_wilcoxon", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent", "step:run_analysis blocked on method:group_comparison_wilcoxon"], "assays": [{"role": "data_generation", "definition_id": "microarray", "definition_version": "99e1195c7d6f", "definition_digest": null, "required_operations": [], "support_state": "partial", "source_hashes": {"assay_lexicon": "99e1195c7d6f"}, "confidence": 0.99, "evidence": ["d EBI ArrayExpress for human gene expression microarray datasets. We also included the Glue Grant tr", ", and surgery. We automatically excluded non-microarray, non-human data. Next, using the abstracts o", "ion: GSE20346, GSE40012, GSE57065, GSE68310. Microarray data exists in both ‘raw’ form (fluorescence", "gh the two datasets were run using different microarray types (Affymetrix HG 2.0 vs. Affymetrix Exon"], "order_basis": "paper_text_order:632", "id": "assay:1"}, {"role": "analysis", "definition_id": "group_comparison", "definition_version": "f2132e96ae6c7a4e", "definition_digest": null, "required_operations": [], "support_state": "partial", "source_hashes": {"dsl_catalogue": "f2132e96ae6c7a4e"}, "confidence": 0.8, "evidence": ["Wilcoxon"], "order_basis": "consumes_earlier_output", "id": "assay:2"}]}