PlasmoFAB: a benchmark to foster machine learning for Plasmodium falciparum protein antigen candidate prediction.

{"sci_version": "1.2", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:37387133", "kind": "science-record", "revision": 4, "content_hash": "sha256:64a178e09701de93024086c578325cefa0523062b40f884b74463abaa1cdbda2", "created_at": "2026-11-01T19:02:08Z", "updated_at": "2026-11-01T22:52:14Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "PlasmoFAB: a benchmark to foster machine learning for Plasmodium falciparum protein antigen candidate prediction.", "doi": "10.1093/bioinformatics/btad206", "pmid": "37387133", "authors": ["Jonas C Ditz", "Jacqueline Wistuba-Hamprecht", "Timo Maier", "Rolf Fendel", "Nico Pfeifer", "Bernhard Reuter"], "journal": "Bioinformatics (Oxford, England)", "year": 2023, "abstract": "MOTIVATION: Machine learning methods can be used to support scientific discovery in healthcare-related research fields. However, these methods can only be reliably used if they can be trained on high-quality and curated datasets. Currently, no such dataset for the exploration of Plasmodium falciparum protein antigen candidates exists. The parasite P.falciparum causes the infectious disease malaria. Thus, identifying potential antigens is of utmost importance for the development of antimalarial drugs and vaccines. Since exploring antigen candidates experimentally is an expensive and time-consuming process, applying machine learning methods to support this process has the potential to accelerate the development of drugs and vaccines, which are needed for fighting and controlling malaria.\nRESULTS: We developed PlasmoFAB, a curated benchmark that can be used to train machine learning methods for the exploration of P.falciparum protein antigen candidates. We combined an extensive literature search with domain expertise to create high-quality labels for P.falciparum specific proteins that distinguish between antigen candidates and intracellular proteins. Additionally, we used our benchmark to compare different well-known prediction models and available protein localization prediction services on the task of identifying protein antigen candidates. We show that available general-purpose services are unable to provide sufficient performance on identifying protein antigen candidates and are outperformed by our models that were trained on this tailored data.\nAVAILABILITY AND IMPLEMENTATION: PlasmoFAB is publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation of PlasmoFAB and the training and evaluation of machine learning models are open source and publicly available on GitHub here: https://github.com/msmdev/PlasmoFAB.", "claims": [{"id": "claim-method", "statement": "The used methods include a kernelized support vector machine (SVM) utilizing the oligo kernel (Meinicke et al. 2004), the protein language model embedding ESM-1b (Rives et al. 2021) combined with a logistic regression (LR) classifier as well as an SVM, and the protein language model embedding ProtT5 (Elnaggar et al. 2020), which we also combined with an LR classifier and an SVM.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:37387133", "locator": "sections[8] \"3 Utilizing machine learning for Plasmodium falciparum protein antigen candidate exploration\" paragraph 2", "quote": "The used methods include a kernelized support vector machine (SVM) utilizing the oligo kernel (Meinicke et al. 2004), the protein language model embedding ESM-1b (Rives et al. 2021) combined with a logistic regression (LR) classifier as well as an SVM, and the protein language model embedding ProtT5 (Elnaggar et al. 2020), which we also combined with an LR classifier and an SVM."}], "state": "complete"}, {"id": "claim-data-access", "statement": "PlasmoFAB is publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation of PlasmoFAB and the training and evaluation of machine learning models are open source and publicly available on GitHub here: https://github.com/msmdev/PlasmoFAB.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:37387133", "locator": "sections[0] \"Abstract\" paragraph 3", "quote": "PlasmoFAB is publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation of PlasmoFAB and the training and evaluation of machine learning models are open source and publicly available on GitHub here: https://github.com/msmdev/PlasmoFAB."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:37387133"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:37387133", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1093/bioinformatics/btad206", "sha256": "sha256:fd43c326a76a7a5f73d6c1a9fe515d3fb63c87198bb3cd9e1e87497bf9d40869", "bytes": 36311, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1093/bioinformatics/btad206", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC10311333", "source": "europe_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}], "gaps": []}, "analysis": {"state": "complete", "methods": [{"id": "method-1", "family": "machine_learning", "specific": "support_vector_machine", "label": "Machine learning", "dsl_operations": ["train_classifier", "cross_validate", "roc_auc"], "missing_operations": [], "expressible": true, "parameters": {}, "data_shapes": ["labelled_feature_matrix"], "evidence": {"score": 1.0, "band": "verbatim", "quote": "The used methods include a kernelized support vector machine (SVM) utilizing the oligo kernel (Meinicke et al. 2004), the protein language model embedding ESM-1b (Rives et al. 2021) combined with a logistic regression (LR) classifier as well as an SVM, and the protein language model embedding ProtT5 (Elnaggar et al. 2020), which we also combined with an LR classifier and an SVM."}}, {"id": "method-2", "family": "regression_modelling", "specific": "logistic", "label": "Regression modelling", "dsl_operations": ["glm_logit_fit"], "missing_operations": [], "expressible": true, "parameters": {}, "data_shapes": ["predictors_and_outcome"], "evidence": {"score": 0.8, "band": "high", "quote": "logistic regression"}}], "targets": [], "gaps": []}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R1", "expressible": true, "required_operations": ["train_classifier", "cross_validate", "roc_auc"], "missing_operations": [], "basis": "The paper names the method it used and the corpus's method extractor maps it onto the DSL catalogue: support_vector_machine."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": ["PlasmoFAB is publicly available on Zenodo with DOI 10.5281/zenodo.7433087. Furthermore, all scripts that were used in the creation of PlasmoFAB and the training and evaluation of machine learning models are open source and publicly available on GitHub here: https://github.com/msmdev/PlasmoFAB."], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": ["train_classifier", "cross_validate", "roc_auc"], "assets": ["source:pmid:37387133"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-11-01T18:45:28Z", "independence": "single_reader"}, "evidence_check": {"method": "verbatim", "availability": "verbatim"}, "calls": 5, "usage": {"prompt_tokens": 9487, "completion_tokens": 2150, "cached_tokens": 128}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:37387133", "type": "paper", "label": "pmid:37387133"}, {"id": "source:pmid:37387133", "type": "manuscript", "label": "PMID 37387133 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:37387133", "type": "paper", "label": "pmid:37387133"}, {"id": "source:pmid:37387133", "type": "manuscript", "label": "PMID 37387133 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-37387133", "type": "extraction", "used": ["source:pmid:37387133"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:37387133", "parent_hash": "sha256:84905fd773bf8a9baa3df43cee9a3dc9b0864fb3d8f5c7050c7e6106590c0083", "fork_of": null, "generation": 3}, "gaps": ["execution.not_run", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent"], "assays": [{"role": "analysis", "definition_id": "machine_learning", "definition_version": "f2132e96ae6c7a4e", "definition_digest": null, "required_operations": ["train_classifier", "cross_validate", "roc_auc"], "support_state": "supported", "source_hashes": {"dsl_catalogue": "f2132e96ae6c7a4e"}, "confidence": 1.0, "evidence": ["The used methods include a kernelized support vector machine (SVM) utilizing the oligo kernel (Meinicke et al. 2004), the protein language model embedding ESM-1b (Rives et al. 2021) combined with a logistic regression (LR) classifier as well as an SVM, and the protein language model embedding ProtT5 (Elnaggar et al. 2020), which we also combined with an LR classifier and an SVM."], "order_basis": "analysis_only", "id": "assay:1"}]}