Improving dictionary-based named entity recognition with deep learning.

{"sci_version": "1.0", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:39230709", "kind": "science-record", "revision": 2, "content_hash": "sha256:7afee70523ef1833206c1b5bbefd5c4362dbc937085eb34e958e6cc1aba64566", "created_at": "2026-11-01T20:29:22Z", "updated_at": "2026-11-01T21:33:49Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "Improving dictionary-based named entity recognition with deep learning.", "doi": "10.1093/bioinformatics/btae402", "pmid": "39230709", "authors": ["Katerina Nastou", "Mikaela Koutrouli", "Sampo Pyysalo", "Lars Juhl Jensen"], "journal": "Bioinformatics (Oxford, England)", "year": 2024, "abstract": "MOTIVATION: Dictionary-based named entity recognition (NER) allows terms to be detected in a corpus and normalized to biomedical databases and ontologies. However, adaptation to different entity types requires new high-quality dictionaries and associated lists of blocked names for each type. The latter are so far created by identifying cases that cause many false positives through manual inspection of individual names, a process that scales poorly.\nRESULTS: In this work, we aim to improve block list s by automatically identifying names to block, based on the context in which they appear. By comparing results of three well-established biomedical NER methods, we generated a dataset of over 12.5 million text spans where the methods agree on the boundaries and type of entity tagged. These were used to generate positive and negative examples of contexts for four entity types (genes, diseases, species, and chemicals), which were used to train a Transformer-based model (BioBERT) to perform entity type classification. Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked. Introducing this into our system doubled the size of the previous list of corpus-wide blocked names. In addition, we generated a document-specific list that allows ambiguous names to be blocked in specific documents. These changes boosted text mining precision by ∼5.5% on average, and over 8.5% for chemical and 7.5% for gene names, positively affecting several biological databases utilizing this NER system, like the STRING database, with only a minor drop in recall (0.6%).\nAVAILABILITY AND IMPLEMENTATION: All resources are available through Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360.", "claims": [{"id": "claim-method", "statement": "We have selected the cased BioBERT base model v1.1 (Lee et al. 2019), a biomedical domain-specific pre-trained deep language representation model based on BERT (Devlin et al. 2018), to train a multi-class entity type classifier on the consensus dataset described above.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:39230709", "locator": "sections[5] \"2.3 Experimental setup\" paragraph 2", "quote": "We have selected the cased BioBERT base model v1.1 (Lee et al. 2019), a biomedical domain-specific pre-trained deep language representation model based on BERT (Devlin et al. 2018), to train a multi-class entity type classifier on the consensus dataset described above."}], "state": "complete"}, {"id": "claim-data-access", "statement": "The data underlying this article are available in Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360.", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:39230709", "locator": "sections[18] \"Data availability\" paragraph 2", "quote": "The data underlying this article are available in Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360."}], "state": "complete"}, {"id": "claim-target-result", "statement": "Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked.", "kind": "quantitative", "target": {"value": 96.7, "unit": "per cent", "tolerance": null, "source": "Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked."}, "evidence": [{"asset": "source:pmid:39230709", "locator": "sections[0] \"Abstract\" paragraph 2", "quote": "Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "Application of the best model (F1-score = 96.7%) allowed us to generate a list of problematic names that should be blocked.", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:39230709"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:39230709", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.1093/bioinformatics/btae402", "sha256": "sha256:e5dcdfbf2680e621cb3c7ebaa973e94d1f0d2174f46e0e56104b8a03045ff9d5", "bytes": 36268, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.1093/bioinformatics/btae402", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC11373323", "source": "europe_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}], "gaps": []}, "analysis": {"state": "partial", "methods": [{"id": "method-1", "family": null, "specific": null, "label": "Consensus of PubTator, EVEX, and tagger followed by training a cased BioBERT base model v1.1 as a multi-class entity type classifier", "dsl_operations": [], "missing_operations": [], "expressible": null, "parameters": {}, "data_shapes": [], "evidence": {"score": 1.0, "band": "verbatim", "quote": "We have selected the cased BioBERT base model v1.1 (Lee et al. 2019), a biomedical domain-specific pre-trained deep language representation model based on BERT (Devlin et al. 2018), to train a multi-class entity type classifier on the consensus dataset described above."}}], "targets": [{"id": "target-published-estimate", "claim_id": "claim-target-result", "operation": null, "expected_value": 96.7, "unit": "per cent", "tolerance": null, "state": "complete", "note": "The published estimate as the reader quoted it from the paper. No tolerance is published because the paper states none."}], "gaps": ["method.reader_named_unmapped"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R4", "expressible": false, "required_operations": [], "missing_operations": [], "basis": "The reader quoted the method the paper names (Consensus of PubTator, EVEX, and tagger followed by training a cased BioBERT base model v1.1 as a multi-class entity type classifier), but the corpus's method extractor could not map it onto a DSL operation, so the method is published but not expressible."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": ["The data underlying this article are available in Zenodo https://doi.org/10.5281/zenodo.11243139 and GitHub https://doi.org/10.5281/zenodo.10289360."], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:39230709"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-11-01T19:30:17Z", "independence": "single_reader"}, "evidence_check": {"method": "verbatim", "availability": "verbatim"}, "calls": 5, "usage": {"prompt_tokens": 9640, "completion_tokens": 1442, "cached_tokens": 264}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:39230709", "type": "paper", "label": "pmid:39230709"}, {"id": "source:pmid:39230709", "type": "manuscript", "label": "PMID 39230709 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:39230709", "type": "paper", "label": "pmid:39230709"}, {"id": "source:pmid:39230709", "type": "manuscript", "label": "PMID 39230709 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-39230709", "type": "extraction", "used": ["source:pmid:39230709"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:39230709", "parent_hash": "sha256:d12840d54a1a2365e0e1feb1c40d9f0bdfbe6e0615e64dbdaa3afc366ddfd0e7", "fork_of": null, "generation": 1}, "gaps": ["execution.not_run", "method.reader_named_unmapped", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent"]}