A curated benchmark dataset for molecular identification based on genome skimming.

{"sci_version": "1.2", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:40442190", "kind": "science-record", "revision": 2, "content_hash": "sha256:76f6a0b3dbbac7490a122e3eccfe1c433cedf362b8e59554189e8aa8e8206d02", "created_at": "2026-11-01T20:30:34Z", "updated_at": "2026-11-01T21:35:14Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "A curated benchmark dataset for molecular identification based on genome skimming.", "doi": "10.1038/s41597-025-05230-2", "pmid": "40442190", "authors": ["Renata C Asprino", "Liming Cai", "Yujing Yan", "Peter J Flynn", "Lucas C Marinho", "Xiaoshan Duan", "Christiane Anderson", "Goia M Lyra", "Charles C Davis", "Bruno A S de Medeiros"], "journal": "Scientific data", "year": 2025, "abstract": "Genome skimming is a promising sequencing strategy for DNA-based taxonomic identification. However, the lack of standardized datasets for benchmarking genome skimming tools presents a challenge in comparing new methods to existing ones. As part of the development of varKoder, a new tool for DNA-based identification, we curated four datasets designed for comparing molecular identification tools using low-coverage genomes. These datasets comprise vast phylogenetic and taxonomic diversity from closely related species to all taxa currently represented on NCBI SRA. One of them consists of novel sequences from taxonomically verified samples in the plant clade Malpighiales, while the other three datasets compile publicly available data. All include raw genome skim sequences to enable comprehensive testing and validation of a variety molecular species identification methods. We also provide the two-dimensional graphical representations of genomic data used in varKoder. These datasets represent a reliable resource for researchers to assess the accuracy, efficiency, and robustness of new tools to varKoder and other methods in a consistent and reproducible manner.", "claims": [{"id": "claim-method", "statement": "We applied leave-one-out cross-validation in all tests following de Medeiros et al.1, so the dataset has not been split into training and validation sets.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:40442190", "locator": "sections[11] \"Malpighiales\" paragraph 2", "quote": "We applied leave-one-out cross-validation in all tests following de Medeiros et al.1, so the dataset has not been split into training and validation sets."}], "state": "complete"}, {"id": "claim-data-access", "statement": "The dataset is available at Harvard Dataverse and the NCBI Sequence Read Archive. The Harvard Dataverse repository57 includes metadata tables, processed conventional DNA barcodes, and DNA signature images (varKodes and rfCGRs). New sequences (i.e., Malpighiales) have been uploaded to NCBI SRA under SRP47912858. All remaining sequence data were already publicly available on NCBI SRA and can be retrieved from the accession numbers in the metadata tables.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:40442190", "locator": "sections[15] \"Data Records\" paragraph 2", "quote": "The dataset is available at Harvard Dataverse and the NCBI Sequence Read Archive. The Harvard Dataverse repository57 includes metadata tables, processed conventional DNA barcodes, and DNA signature images (varKodes and rfCGRs). New sequences (i.e., Malpighiales) have been uploaded to NCBI SRA under SRP47912858. All remaining sequence data were already publicly available on NCBI SRA and can be retrieved from the accession numbers in the metadata tables."}], "state": "complete"}, {"id": "claim-target-result", "statement": "The paper curated four standardized datasets for benchmarking molecular identification tools using low-coverage genomes.", "kind": "qualitative", "target": null, "evidence": [{"asset": "source:pmid:40442190", "locator": "sections[0] \"Abstract\" paragraph 1", "quote": "As part of the development of varKoder, a new tool for DNA-based identification, we curated four datasets designed for comparing molecular identification tools using low-coverage genomes."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "The paper curated four standardized datasets for benchmarking molecular identification tools using low-coverage genomes.", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:40442190"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_unresolved", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": ["data:geo:SRP479128"], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": ["data:geo:SRP479128"], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_unresolved"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:40442190", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1038/s41597-025-05230-2", "sha256": "sha256:d80e3a843e9535fdd305a6a7b48cd4e32c2ef4e9c7bf97a7c2a3ff8ad1ddd19e", "bytes": 35961, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1038/s41597-025-05230-2", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC12122930", "source": "europe_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}, {"id": "data:geo:SRP479128", "role": "raw", "format": "geo-series", "media_type": "application/vnd.ncbi.geo-series+json", "uri": "https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=SRP479128", "sha256": null, "bytes": null, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "SRP479128", "version": null, "note": "The dataset is available at Harvard Dataverse and the NCBI Sequence Read Archive. The Harvard Dataverse repository57 includes metadata tables, processed conventional DNA barcodes, and DNA signature images (varKodes and rfCGRs). New sequences (i.e., Malpighiales) have been uploaded to NCBI SRA under SRP47912858. All remaining sequence data were already publicly available on NCBI SRA and can be retrieved from the accession numbers in the metadata tables.", "structure": {"repository": "GEO", "access_route": "GEO accession URL", "access_statement_verbatim": "The dataset is available at Harvard Dataverse and the NCBI Sequence Read Archive. The Harvard Dataverse repository57 includes metadata tables, processed conventional DNA barcodes, and DNA signature images (varKodes and rfCGRs). New sequences (i.e., Malpighiales) have been uploaded to NCBI SRA under SRP47912858. All remaining sequence data were already publicly available on NCBI SRA and can be retrieved from the accession numbers in the metadata tables.", "access_statement_kind": "data_availability_statement", "sample_accessions": [], "sample_count": null, "group_sizes": null, "group_sizes_basis": null, "platform": null, "taxon": null, "title": null, "resolution_checked_at": "2026-11-01T20:16:09Z", "resolution_check": "identifier_resolution_only_no_download_no_licence_read", "downloaded": false, "licence_read": false}}], "gaps": ["asset:data:geo:SRP479128 not downloaded", "asset:data:geo:SRP479128 licence not read"]}, "analysis": {"state": "partial", "methods": [{"id": "method-1", "family": null, "specific": null, "label": "leave-one-out cross-validation", "dsl_operations": [], "missing_operations": [], "expressible": null, "parameters": {}, "data_shapes": [], "evidence": {"score": 1.0, "band": "verbatim", "quote": "We applied leave-one-out cross-validation in all tests following de Medeiros et al.1, so the dataset has not been split into training and validation sets."}}], "targets": [], "gaps": ["method.reader_named_unmapped"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R4", "expressible": false, "required_operations": [], "missing_operations": [], "basis": "The reader quoted the method the paper names (leave-one-out cross-validation), but the corpus's method extractor could not map it onto a DSL operation, so the method is published but not expressible."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": ["SRP479128"], "resolved_accessions": [], "statements": ["The dataset is available at Harvard Dataverse and the NCBI Sequence Read Archive. The Harvard Dataverse repository57 includes metadata tables, processed conventional DNA barcodes, and DNA signature images (varKodes and rfCGRs). New sequences (i.e., Malpighiales) have been uploaded to NCBI SRA under SRP47912858. All remaining sequence data were already publicly available on NCBI SRA and can be retrieved from the accession numbers in the metadata tables."], "basis": "The paper names SRP479128, but the accession does not resolve in the repository (not_in_ncbi_sra), so the data cannot be reached from here. No dataset was downloaded and no licence was read."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:40442190", "data:geo:SRP479128"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:SRP479128"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-11-01T19:53:23Z", "independence": "single_reader"}, "evidence_check": {"method": "verbatim", "availability": "verbatim"}, "calls": 5, "usage": {"prompt_tokens": 9737, "completion_tokens": 1458, "cached_tokens": 128}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:40442190", "type": "paper", "label": "pmid:40442190"}, {"id": "source:pmid:40442190", "type": "manuscript", "label": "PMID 40442190 full text"}, {"id": "data:geo:SRP479128", "type": "dataset", "label": "GEO SRP479128"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:40442190", "type": "paper", "label": "pmid:40442190"}, {"id": "source:pmid:40442190", "type": "manuscript", "label": "PMID 40442190 full text"}, {"id": "data:geo:SRP479128", "type": "dataset", "label": "GEO SRP479128"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-40442190", "type": "extraction", "used": ["source:pmid:40442190"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-SRP479128", "type": "accession_resolution", "used": ["data:geo:SRP479128"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:40442190", "parent_hash": "sha256:73103dca664da81213e028c467bc728076725b7e594ec7d9eb61a751a36f8cf7", "fork_of": null, "generation": 1}, "gaps": ["asset:data:geo:SRP479128 licence not read", "asset:data:geo:SRP479128 not downloaded", "execution.not_run", "method.reader_named_unmapped", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_unresolved"], "assays": [{"role": "data_generation", "definition_id": "qpcr", "definition_version": "99e1195c7d6f", "definition_digest": null, "required_operations": [], "support_state": "partial", "source_hashes": {"assay_lexicon": "99e1195c7d6f"}, "confidence": 0.93, "evidence": [".7 nM or 1.0 nM and pooled together. We used Real-Time PCR (BioRad CFX96 Touch, BioRad Laboratories, He"], "order_basis": "paper_text_order:3235", "id": "assay:1"}]}