Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication.

{"sci_version": "1.0", "schema": "https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json", "id": "pmid:33511044", "kind": "science-record", "revision": 2, "content_hash": "sha256:0232184ad9e33914eada1fc1d2aab165211d36d4af188f62888ed01176c5b5bd", "created_at": "2026-11-01T17:34:39Z", "updated_at": "2026-11-01T18:54:22Z", "generator": {"name": "corpus-assay-engine", "run_id": "reading-sci-2026-11-01"}, "paper": {"state": "complete", "title": "Assessing the Reliability of Relevant Tweets and Validation Using Manual and Automatic Approaches for Flood Risk Communication.", "doi": "10.3390/ijgi9090532", "pmid": "33511044", "authors": ["Xiaohui Liu", "Bandana Kar", "Francisco Alejandro Montiel Ishino", "Chaoyang Zhang", "Faustine Williams"], "journal": "ISPRS international journal of geo-information", "year": 2020, "abstract": "While Twitter has been touted as a preeminent source of up-to-date information on hazard events, the reliability of tweets is still a concern. Our previous publication extracted relevant tweets containing information about the 2013 Colorado flood event and its impacts. Using the relevant tweets, this research further examined the reliability (accuracy and trueness) of the tweets by examining the text and image content and comparing them to other publicly available data sources. Both manual identification of text information and automated (Google Cloud Vision, application programming interface (API)) extraction of images were implemented to balance accurate information verification and efficient processing time. The results showed that both the text and images contained useful information about damaged/flooded roads/streets. This information will help emergency response coordination efforts and informed allocation of resources when enough tweets contain geocoordinates or location/venue names. This research will identify reliable crowdsourced risk information to facilitate near real-time emergency response through better use of crowdsourced risk communication platforms.", "claims": [{"id": "claim-data-access", "statement": "Historical tweets were identified and then purchased from Twitter Inc. using two types of keywords: (1) location names (Colorado, Boulder, Front Range, El Paso County and Boulder County, Denver metro), and (2) hazard event/impacts (flash flooding, flooding, rain 2013, emergency, impact, damaged bridges and roads, damaged houses, financial losses, evacuate, and evacuation).", "kind": "methodological", "target": null, "evidence": [{"asset": "source:pmid:33511044", "locator": "sections[6] \"Tweets of 2013 Colorado floods\" paragraph 2", "quote": "Historical tweets were identified and then purchased from Twitter Inc. using two types of keywords: (1) location names (Colorado, Boulder, Front Range, El Paso County and Boulder County, Denver metro), and (2) hazard event/impacts (flash flooding, flooding, rain 2013, emergency, impact, damaged bridges and roads, damaged houses, financial losses, evacuate, and evacuation)."}], "state": "complete"}, {"id": "claim-target-result", "statement": "The results showed that both the text and images contained useful information about damaged/flooded roads/streets.", "kind": "qualitative", "target": null, "evidence": [{"asset": "source:pmid:33511044", "locator": "sections[0] \"Abstract\" paragraph 1", "quote": "The results showed that both the text and images contained useful information about damaged/flooded roads/streets."}, {"asset": "source:pmid:33511044", "locator": "sections[12] \"Evaluation of Text Content\" paragraph 2", "quote": "As a result, 584 out of 720 relevant tweets were verified to have reliable information."}, {"asset": "source:pmid:33511044", "locator": "sections[13] \"Evaluation of Image Content\" paragraph 2", "quote": "Results showed that the manual approach identified 60 (19%) reliable images compared to the AI approach which detected only 34 (11%)."}], "state": "complete"}], "gaps": []}, "experiment": {"state": "partial", "objective": "The results showed that both the text and images contained useful information about damaged/flooded roads/streets.", "steps": [{"id": "read_published_estimate", "order": 1, "name": "Read the published estimate from the source", "kind": "other", "state": "complete", "blocked_on": null, "description": "The claim, method and evidence span published with the source article.", "duration_hours": null, "parameters": {}, "inputs": ["source:pmid:33511044"], "outputs": ["the published estimate and its evidence"], "instruments": [], "reagents": []}, {"id": "obtain_source_data", "order": 2, "name": "Obtain the source data from the named repository", "kind": "data-preparation", "state": "blocked", "blocked_on": "data:accession_absent", "description": "The GEO series accession resolves and its sample accessions are listed. This lane did not download the dataset or read its licence.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the source data with its repository provenance"], "instruments": [], "reagents": []}, {"id": "run_analysis", "order": 3, "name": "Run the published analysis", "kind": "analysis", "state": "complete", "blocked_on": null, "description": "Apply the paper's statistical method to the resolved series and compare the result with the published target.", "duration_hours": null, "parameters": {}, "inputs": [], "outputs": ["the replicated result and its comparison"], "instruments": [], "reagents": []}], "materials": [], "instruments": [], "reagents": [], "conditions": {}, "gaps": ["step:obtain_source_data blocked on data:accession_absent"]}, "data": {"state": "partial", "assets": [{"id": "source:pmid:33511044", "role": "manuscript", "format": "xml", "media_type": "application/xml", "uri": "https://doi.org/10.3390/ijgi9090532", "sha256": "sha256:84a26b2cc2fbea92b2787eab29e5384663aa07d5cc399597c8c99b8538e16be6", "bytes": 35745, "access": "open", "state": "resolved", "reader": null, "reader_status": "reader_required", "identifier": "10.3390/ijgi9090532", "version": null, "note": "Full article fetched from PMC; the checksum covers the rendered section body, not the original XML bytes.", "structure": {"pmcid": "PMC7839990", "source": "ncbi_pmc_fulltext", "coverage": "whole_body_without_references", "text_hash_scope": "rendered_section_body"}}], "gaps": []}, "analysis": {"state": "absent", "methods": [], "targets": [], "gaps": ["method", "operations"]}, "capability": {"state": "complete", "requirements": [{"capability_id": "capability:computational_analysis", "label": "Computational analysis platform", "assay_families": ["meta_analysis", "machine_learning", "survival_analysis", "statistical_regression", "pca", "molecular_docking", "network_analysis", "differential_expression", "gwas"], "availability_state": "owned", "tier_unlocked": "R1", "candidate": "Automated Assays Laravel application and simulated adapters"}], "gaps": []}, "readiness": {"state": "blocked", "tier": "R4", "blocking_axis": "data", "blocked_on": "data:accession_absent", "axes": {"method": {"tier": "R4", "expressible": false, "required_operations": [], "missing_operations": [], "basis": "The reader quoted the method the paper names (not named), but the corpus's method extractor could not map it onto a DSL operation, so the method is published but not expressible."}, "capability": {"tier": "R1", "capability_id": "capability:computational_analysis", "availability_state": "owned", "basis": "The computational analysis is owned. The physical bench the paper used is not, so a physical validation cannot be run here."}, "data": {"tier": "R4", "reachable": false, "accessions": [], "resolved_accessions": [], "statements": ["Historical tweets were identified and then purchased from Twitter Inc. using two types of keywords: (1) location names (Colorado, Boulder, Front Range, El Paso County and Boulder County, Denver metro), and (2) hazard event/impacts (flash flooding, flooding, rain 2013, emergency, impact, damaged bridges and roads, damaged houses, financial losses, evacuate, and evacuation)."], "basis": "The paper names no accession to its source data, so the data has to come from the authors."}}, "can_run": {"dsl_operations": [], "assets": ["source:pmid:33511044"], "steps": ["run_analysis"]}, "must_build": {"dsl_operations": [], "converters": [], "readers": [], "capabilities": [], "data_gaps": ["data:geo:None"]}, "gaps": ["data:accession_absent"]}, "review": {"state": "partial", "machine_critique": {"kind": "extraction_review", "reader": {"model": "MiniMax-M3.1-Flash", "provider": "minimax", "provider_family": "minimax", "read_at": "2026-11-01T17:01:27Z", "independence": "single_reader"}, "evidence_check": {"method": null, "availability": null}, "calls": 5, "usage": {"prompt_tokens": 8820, "completion_tokens": 1923, "cached_tokens": 264}}, "human_critique": null, "stages": [{"id": "extraction", "state": "complete", "reader": "MiniMax-M3.1-Flash", "provider": "minimax"}], "gaps": ["review.independent_reader", "review.machine_critique", "review.human_critique"]}, "execution": {"state": "absent", "status": "not_run", "mode": null, "runs": [], "gaps": ["execution.not_run"]}, "provenance": {"entities": [{"id": "pmid:33511044", "type": "paper", "label": "pmid:33511044"}, {"id": "source:pmid:33511044", "type": "manuscript", "label": "PMID 33511044 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "activities": [{"id": "activity-reading-sci-2026-11-01", "type": "classification", "used": [{"id": "pmid:33511044", "type": "paper", "label": "pmid:33511044"}, {"id": "source:pmid:33511044", "type": "manuscript", "label": "PMID 33511044 full text"}, {"id": "data:geo:None", "type": "dataset", "label": "GEO None"}], "agent": "agent:corpus-assay-engine"}, {"id": "activity-read-33511044", "type": "extraction", "used": ["source:pmid:33511044"], "agent": "agent:minimax-reader"}, {"id": "activity-resolve-None", "type": "accession_resolution", "used": ["data:geo:None"], "agent": "agent:corpus-assay-engine"}], "agents": [{"id": "agent:corpus-assay-engine", "type": "software", "label": "Corpus assay engine"}, {"id": "agent:minimax-reader", "type": "software", "label": "MiniMax literature reader"}], "source_run_id": "reading-sci-2026-11-01", "dsl_catalogue_fingerprint": "343fb30756ba6ec3", "capability_table_fingerprint": "2cb3e42f047d2ecf", "snapshot_id": "ba34b49490d9bfaf"}, "lineage": {"parent_id": "pmid:33511044", "parent_hash": "sha256:64fedacceddf9258c4ea0e0b4714d1e852df6cab47e910013f21f6fbd516fbb3", "fork_of": null, "generation": 1}, "gaps": ["execution.not_run", "method", "operations", "review.human_critique", "review.independent_reader", "review.machine_critique", "step:obtain_source_data blocked on data:accession_absent"]}