Benchmarking optimization methods for parameter estimation in large kinetic models.

{"sci_version": "1.0", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:30816929", "kind": "science-record", "revision": 4, "content_hash": "sha256:cf97beb0b794dc89faff5f7f73f8742df1ef1e09a431e8cdb9f833c4fc81adeb", "created_at": "2026-11-01T15:40:42Z", "updated_at": "2026-11-01T21:22:02Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "Benchmarking optimization methods for parameter estimation in large kinetic models.", "doi": "10.1093/bioinformatics/bty736", "pmid": "30816929", "authors": ["Alejandro F Villaverde", "Fabian Fröhlich", "Daniel Weindl", "Jan Hasenauer", "Julio R Banga"], "journal": "Bioinformatics (Oxford, England)", "year": 2019, "abstract": "MOTIVATION: Kinetic models contain unknown parameters that are estimated by optimizing the fit to experimental data. This task can be computationally challenging due to the presence of local optima and ill-conditioning. While a variety of optimization methods have been suggested to surmount these issues, it is difficult to choose the best one for a given problem a priori. A systematic comparison of parameter estimation methods for problems with tens to hundreds of optimization variables is currently missing, and smaller studies provided contradictory findings.\nRESULTS: We use a collection of benchmarks to evaluate the performance of two families of optimization methods: (i) multi-starts of deterministic local searches and (ii) stochastic global optimization metaheuristics; the latter may be combined with deterministic local searches, leading to hybrid methods. A fair comparison is ensured through a collaborative evaluation and a consideration of multiple performance metrics. We discuss possible evaluation criteria to assess the trade-off between computational efficiency and robustness. Our results show that, thanks to recent advances in the calculation of parametric sensitivities, a multi-start of gradient-based local methods is often a successful strategy, but a better performance can be obtained with a hybrid metaheuristic. The best performer combines a global scatter search metaheuristic with an interior point local method, provided with gradients estimated with adjoint-based sensitivities. We provide an implementation of this method to render it available to the scientific community.\nAVAILABILITY AND IMPLEMENTATION: The code to reproduce the results is provided as Supplementary Material and is available at Zenodo https://doi.org/10.5281/zenodo.1304034.\nSUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.", "claims": [{"id": "claim-method", "statement": "Therefore, in the fourth step, we quantify the trade-off between success rate and average computation time using a novel metric called overall efficiency (OE).", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:30816929", "locator": "sections[7] \"2.5 Comparison of optimization methods\" paragraph 7", "quote": "Therefore, in the fourth step, we quantify the trade-off between success rate and average computation time using a novel metric called overall efficiency (OE)."}], "state": "complete"}, {"id": "claim-data-access", "statement": "Supplementary data are available at Bioinformatics online.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:30816929", "locator": "sections[0] \"Abstract\" paragraph 4", "quote": "Supplementary data are available at Bioinformatics online."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": null, "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:30816929"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:30816929", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1093/bioinformatics/bty736", "sha256": "sha256:9d9f784261c7b46fbc94943a38119ebd61c189054985c2bf7ecd307cd8f5985d", "bytes": 39891, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1093/bioinformatics/bty736", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC6394396", "source": "europe_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}, {"id": "asset:table:pmid:30816929:1", "role": "table", "format": "jats-table", "media_type": "application/vnd.jats.table+json", "uri": null, "sha256": "sha256:8b1af1e89f9e9e64f060905347dc1a7ccbc99647a9e5c0e46c8b9086c0b15f37", "bytes": 675, "access": "open", "state": "verified", "reader": "read_table", "reader_status": "available", "identifier": "section[5]/table-wrap[1]#bty736-T1", "version": null, "note": "Classification of the hybrid optimization methods considered in the benchmarking", "structure": {"pmcid": "PMC6394396", "section_index": 5, "section_title": "2.3 Choice of optimization methods for benchmarking", "row_count": 6, "column_count": 6}, "table": {"label": "Table 1.", "caption": "Classification of the hybrid optimization methods considered in the benchmarking", "rows": [["Global strategy", "Local method & gradient calculation", "Parameter scaling"], ["", "FMINCON-ADJ", "NL2SOL-FWD", "DHC", "None", ""], ["MS", "MS-FMINCON-ADJ-LOG", "MS-NL2SOL-FWD-LOG", "MS-DHC-LOG", "–", "LOG"], ["", "MS-FMINCON-ADJ-LIN", "MS-NL2SOL-FWD-LIN", "MS-DHC-LIN", "–", "LIN"], ["eSS", "eSS-FMINCON-ADJ-LOG", "eSS-NL2SOL-FWD-LOG", "eSS-DHC-LOG", "eSS-NOLOC-LOG", "LOG"], ["", "eSS-FMINCON-ADJ-LIN", "eSS-NL2SOL-FWD-LIN", "eSS-DHC-LIN", "eSS-NOLOC-LIN", "LIN"]], "row_count": 6, "column_count": 6, "numeric_cells": 0, "header_rows": 2, "locator": "section[5]/table-wrap[1]#bty736-T1"}}, {"id": "asset:table:pmid:30816929:2", "role": "table", "format": "jats-table", "media_type": "application/vnd.jats.table+json", "uri": null, "sha256": "sha256:837bdb1e53acd91fa1b2cb3e8a1ecf1711e235a42f5198c6a5737cef145bfe21", "bytes": 1415, "access": "open", "state": "verified", "reader": "read_table", "reader_status": "available", "identifier": "section[8]/table-wrap[1]#bty736-T2", "version": null, "note": "Main features of the benchmarks.The model IDs follow the nomenclature in Villaverde et al. (2015) and Fröhlich et al. (2017a)", "structure": {"pmcid": "PMC6394396", "section_index": 8, "section_title": "2.6 Benchmark problems", "row_count": 14, "column_count": 8}, "table": {"label": "Table 2.", "caption": "Main features of the benchmarks.The model IDs follow the nomenclature in Villaverde et al. (2015) and Fröhlich et al. (2017a)", "rows": [["ID", "B2", "B3", "B4", "B5", "BM1", "BM3", "TSP"], ["Original ref.", "Chassagnole et al. (2002)", "Kotte et al. (2010)", "Villaverde et al. (2014)", "MacNamara et al. (2012)", "Smith and Shanley (2013)", "Chen et al. (2009)", "Moles et al. (2003)"], ["Organism", "Escherichia coli", "Escherichia coli", "Chinese hamster", "Generic", "Mouse", "Human", "Generic"], ["Description", "Metabolic", "Metabolic", "Metabolic", "Signaling", "Signaling", "Signaling", "Metabolic"], ["level", "", "& transcription", "", "", "", "", ""], ["Parameters", "116", "178", "117", "86", "383", "219", "36"], ["Upper bounds", "10·pref", "10·pref", "10·pref", "varying", "10·pref", "103·pref", "105·pref"], ["Lower bounds", "0.1·pref", "0.1·pref", "0.1·pref", "varying", "0.1·pref", "10−3·pref", "10−5·pref"], ["Dynamic states", "18", "47", "34", "26", "104", "500", "8"], ["Observed states", "9", "47", "13", "6", "12", "5", "8"], ["Experiments", "1", "1", "1", "10", "1", "4", "16"], ["Data points", "110", "7567", "169", "960", "120", "105", "2688"], ["Data type", "Measured", "Simulated", "Simulated", "Simulated", "Measured", "Measured", "Simulated"], ["Noise level", "Reala", "No noise", "Variableb", "Uniformc", "Reala", "Reala", "σ=10%d"]], "row_count": 14, "column_count": 8, "numeric_cells": 55, "header_rows": 1, "locator": "section[8]/table-wrap[1]#bty736-T2"}}], "gaps": []}, "analysis": {"state": "partial", "methods": [{"id": "method-1", "family": null, "specific": null, "label": "overall efficiency (OE)", "dsl_operations": [], "missing_operations": [], "expressible": null, "parameters": {}, "data_shapes": [], "evidence": {"score": 1.0, "band": "verbatim", "quote": "Therefore, in the fourth step, we quantify the trade-off between success rate and average computation time using a novel metric called overall efficiency (OE)."}}], "targets": [], "gaps": ["method.reader_named_unmapped"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R4", "expressible": false, "required_operations": [], "missing_operations": [], "basis": "The reader quoted the method the paper names (overall efficiency (OE)), but the corpus's method extractor could not map it onto a DSL operation, so the method is published but not expressible."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": ["Supplementary data are available at Bioinformatics online."], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:30816929", "asset:table:pmid:30816929:1", "asset:table:pmid:30816929:2"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-10-31T18:14:03Z", "independence": "single_reader"}, "evidence_check": {"method": "verbatim", "availability": "verbatim"}, "calls": 3, "usage": {"prompt_tokens": 28384, "completion_tokens": 1413, "cached_tokens": 792}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:30816929", "type": "paper", "label": "pmid:30816929"}, {"id": "source:pmid:30816929", "type": "manuscript", "label": "PMID 30816929 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:30816929", "type": "paper", "label": "pmid:30816929"}, {"id": "source:pmid:30816929", "type": "manuscript", "label": "PMID 30816929 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-30816929", "type": "extraction", "used": ["source:pmid:30816929"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:30816929", "parent_hash": "sha256:fb48bc12015cc2c9ca28eed3d175d11b483c8729e26476206286224a6e9d9b88", "fork_of": null, "generation": 3}, "gaps": ["execution.not_run", "method.reader_named_unmapped", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent"]}