SCI v1.5

One file for the paper and the experiment behind it

Keep a paper and the experiment it describes in one portable record. The format is JSON, the schema is published, and every record carries its own checksum.

Read the specification Fetch the JSON Schema See a published record

The 21 top-level fields

A record is one JSON object with these fields and no others. The schema sets additionalProperties to false, so a record that carries an unknown field is rejected rather than silently accepted.

Field	What it carries	Required
sci_version	The version of the format this document is written in.	yes
schema	The absolute URI of the JSON Schema this document validates against.	optional
id	The .science file identifier, for example pmid:26981379.	yes
kind	What kind of record this is. Always science-record in v1.	yes
revision	The counter that changes when the body of the record changes.	yes
content_hash	SHA-256 over the canonical JSON of the record body. Reproducible, and checkable by any consumer.	yes
created_at	When the record was first published.	optional
updated_at	When the record was last changed.	optional
generator	The tool and converter that wrote the record, and the dialect it was read from.	optional
paper	The source: title, authors, journal, abstract and the claims the paper makes.	yes
experiment	The experiment behind the paper: objective, steps, materials, instruments, reagents and conditions.	yes
data	The data the work rests on: every file, table or repository route, with its checksum and access state, and the size of each comparison group.	yes
analysis	The analysis: named methods, the operations they need, and the published target a run would check.	yes
capability	The capabilities required to run the experiment.	yes
readiness	How ready the record is for execution, and the requirements still to resolve.	yes
review	What the readers made of the paper, and where that critique is published.	yes
execution	Whether the experiment has been run, and the evidence if it has.	yes
provenance	Where every part of the record came from: entities, activities and agents.	yes
lineage	The revision this record supersedes, its checksum, and the generation it belongs to.	optional
assays	Every assay the experiment ran, in the order it ran them. An experiment is a set of processes and each process is an assay: the measurement, the analysis, the validation, the normalisation. Every one of them is required, so there is no primary assay and no optional one.	optional
gaps	Missing information and unresolved questions.	yes

The four layers

Each layer records its completion state and identifies the information still needed.

paper

What the paper says: its claims, its authors, its abstract.

state: absent, partial, complete or blocked

experiment

The work behind the paper: steps, materials, instruments, conditions.

state: absent, partial, complete or blocked

data

The data route: every asset with its checksum, format and access state.

state: absent, partial, complete or blocked

analysis

The analysis: named methods, the operations needed, the published target.

state: absent, partial, complete or blocked

What the record carries beyond the four layers

The schema at the format’s URI accepts 1.0, 1.1 and 1.2 records, so a client that reads the address it has always read sees all of them. A 1.0 record is unchanged and still valid.

data.group_sizes

The number of samples in each comparison group, as the repository metadata states it. A consumer holding the samples can check them against the paper’s design instead of running the paper’s method over a different set.

assays

Every assay the experiment ran, in the order it ran them. Each one names the definition it used, the operations it needs and whether the platform supports them. The catalogue that owns the definitions keeps them. Version 1.2 replaced the single binding of version 1.1 with this array, because an experiment is a set of processes and every one of them is required.

How a record changes

A published record is immutable. A corrected record is published as the next revision of the same record and names the revision it replaces, so a consumer can prove what it holds and can fetch what supersedes it. A rebuild that changes nothing leaves the published bytes, the checksum and the revision number exactly as they were.

The checksum rule

content_hash is a SHA-256 over the canonical JSON encoding of the record body with the hash field itself removed: sorted keys, compact separators, UTF-8. The same canonical content always hashes to the same value. A consumer can recompute it to check the record’s content; a separate byte checksum checks the exact file received.

A number is written with the fewest digits that read back to the same value, in plain decimal between 0.0001 and 10^16 and in scientific notation otherwise, with an exponent of at least two digits. PHP and JavaScript write the same number differently, so a consumer that re-encodes a document follows the rule as the specification states it.

The engine’s own validator checks the schema, the hash, that every step cites an asset that exists, and that every method claims only operations the catalogue knows. A consumer is free to check the first and the second on its own.

Where to get one

Endpoint	What it returns
GET /api/v1/sci	The format contract: the schema URI and the status of each input dialect.
GET /api/v1/sci/schema	The JSON Schema inside the published envelope.
GET /api/v1/sci/documents	Every published .science file, filterable and pageable.
GET /api/v1/sci/documents/{id}	One whole .science file.
GET /api/v1/sci/documents/{id}/versions	Every published version of that file, each with its own hash.
GET /api/v1/sci/versions/{content_hash}	One exact version, by the hash of its own body.
GET /api/v1/papers/{pmid}/sections	The section index for a paper: every section, its role and its size.

The specification

The full text of the format, published as Markdown so it can be read, cited and versioned alongside the schema it describes.

Read the specification →

Download the Markdown

Every field

250 keys, each with its own address. The schema says what is valid; this says what each key is for, so a .science file can be read, edited or proposed against.

sci_version enum required one of 1.0, 1.1, 1.2, 1.3, 1.4

The minor version of the format this document is written in. 1.0, 1.1 and 1.2 are accepted at the same schema URI.

schema any optional one of https://corpus.bntrg.co.uk/sci/1.0/sci.schema.json

The absolute URI of the JSON Schema the document validates against. It resolves to the schema and to this specification.

id string required

The record identifier. A corpus record is `pmid:<pmid>`; a user-authored fork is `fork:<parent-id>:<slug>`.

kind any required one of science-record

What kind of record this is. Always `science-record` in version 1.

revision integer required

The counter that increases each time the body of the record changes. The first publication is revision 1.

content_hash string or null required

SHA-256 over the canonical JSON of the record body, as `sha256:<hex>`. A consumer recomputes it to check the record.

created_at string or null optional

When the record was first published, in UTC.

updated_at string or null optional

When the record body last changed, in UTC.

generator object or null optional

The tool and converter that wrote the record, and the input dialect it was read from.

paper object required

The source: what the paper says, and the claims it makes.

paper.state enum required one of absent, partial, complete, blocked

How complete the paper layer is: absent, partial, complete or blocked.

paper.title string or null required

The title of the paper.

paper.doi string or null required

The DOI of the paper.

paper.pmid string or null required

The PubMed identifier of the paper.

paper.authors array or null required

The authors of the paper, as a list of names.

paper.journal string or null required

The journal the paper was published in.

paper.year integer or null required

The year of publication.

paper.abstract string or null required

The abstract of the paper.

paper.claims array required

The claims the paper makes, each with its target and its evidence.

paper.claims[].id string required

The identifier of the claim within the record.

paper.claims[].statement string or null required

The claim, as the paper states it.

paper.claims[].kind enum required one of quantitative, qualitative, methodological

Whether the claim is quantitative, qualitative or methodological.

paper.claims[].state enum required one of absent, partial, complete, blocked

How complete the claim is.

paper.claims[].target object or null required

The published value the claim is about, where the paper states one.

paper.claims[].target.value number or null required

The published value, as a number.

paper.claims[].target.unit string or null required

The unit of the published value.

paper.claims[].target.tolerance number or null required

The tolerance a recomputed value must fall inside for the claim to be reproduced.

paper.claims[].target.source string or null required

Where the published value came from: a table, a figure or the text.

paper.claims[].evidence array required

The evidence for the claim: where in the record it came from and what was quoted.

paper.claims[].evidence[].asset string or null required

The asset the quotation was taken from.

paper.claims[].evidence[].locator string or null required

Where in the asset the quotation sits.

paper.claims[].evidence[].quote string or null required

The quoted text, checked against the asset it names.

paper.gaps array required

What is missing from the paper layer.

experiment object required

The experiment behind the paper: what was done, with what, in what order.

experiment.state enum required one of absent, partial, complete, blocked

How complete the experiment layer is.

experiment.objective string or null required

What the experiment set out to do.

experiment.steps array required

The steps of the experiment, in order.

experiment.steps[].id string required

The identifier of the step within the record.

experiment.steps[].order integer required

The position of the step in the sequence.

experiment.steps[].name string or null required

The name of the step.

experiment.steps[].kind enum required one of physical, computational, data-preparation, analysis, other

Whether the step is physical, computational, data preparation, analysis or other.

experiment.steps[].state enum required one of absent, partial, complete, blocked

How complete the step is.

experiment.steps[].blocked_on string or null optional

What stops the step running: a missing datum, a missing method, a missing capability, or null.

experiment.steps[].description string or null optional

What the step does.

experiment.steps[].duration_hours number or null required

How long the step takes, in hours.

experiment.steps[].parameters object required

The parameters of the step, keyed by name.

experiment.steps[].inputs array required

The assets the step consumes.

experiment.steps[].outputs array required

The assets the step produces.

experiment.steps[].instruments array optional

The instruments the step uses.

experiment.steps[].reagents array optional

The reagents the step consumes.

experiment.steps[].reagents[].name string or null required

The name of the reagent.

experiment.steps[].reagents[].quantity number or null required

How much of the reagent is used.

experiment.steps[].reagents[].unit string or null required

The unit of that quantity.

experiment.steps[].reagents[].catalogue string or null optional

The catalogue the reagent comes from, where it is named.

experiment.materials array required

The materials the experiment consumes.

experiment.instruments array required

The instruments the experiment uses.

experiment.reagents array required

The reagents the experiment consumes.

experiment.conditions object required

The conditions the experiment runs under, keyed by name.

experiment.gaps array required

What is missing from the experiment layer.

data object required

The data the work rests on: every file, table or repository route, with its checksum and access state.

data.state enum required one of absent, partial, complete, blocked

How complete the data layer is.

data.group_sizes object or null optional

The number of samples in each comparison group, as the repository metadata states it. It is what separates a reproduction from a re-analysis.

data.assets array required

Every file, table or repository route the record rests on.

data.assets[].id string required

The identifier of the asset within the record.

data.assets[].role enum required one of manuscript, protocol, raw, processed, derived, code, supplementary, figure, table, reference

What the asset is: the manuscript, a protocol, raw or processed data, derived data, code, supplementary material, a figure, a table or a reference.

data.assets[].format string required

The format of the asset, as the engine names formats: `jats-table`, `geo-series`, `vcf`, `csv`, `xlsx` and so on.

data.assets[].media_type string required

The declared media type of the asset.

data.assets[].uri string or null required

Where the asset can be fetched, or null when it lives inside the record.

data.assets[].sha256 string or null required

The checksum of the asset's own bytes, as `sha256:<hex>`, or null when the asset was not read.

data.assets[].bytes integer or null required

The size of the asset in bytes, or null when it is not known.

data.assets[].access enum required one of open, restricted, embargoed, unknown

Whether the asset is open, restricted, embargoed or of unknown access.

data.assets[].state enum required one of declared, resolved, verified, unavailable

How far the asset was resolved: declared, resolved, verified or unavailable.

data.assets[].reader string or null required

The engine component that reads the asset.

data.assets[].reader_status enum required one of available, converter_required, reader_required, unknown

Whether that reader exists today: available, converter required, reader required or unknown.

data.assets[].identifier string or null optional

The accession, DOI or locator that names the asset elsewhere.

data.assets[].version string or null optional

The version of the asset, where the source states one.

data.assets[].note string or null optional

A note about the asset, in the words of the source.

data.assets[].structure object or null required

What the asset contains: its shape, its parts and where they sit.

data.assets[].table object or null optional

The table itself, for an asset whose role is `table`.

data.assets[].table.label string or null required

The label of the table in the article, for example `Table 1`.

data.assets[].table.caption string or null required

The caption of the table.

data.assets[].table.rows array required

The rows of the table, each a list of cells.

data.assets[].table.row_count integer required

How many rows the table has.

data.assets[].table.column_count integer required

How many columns the table has.

data.assets[].table.numeric_cells integer required

How many cells hold a number. A table with none cannot carry a published result.

data.assets[].table.header_rows integer required

How many rows at the top of the table are headings.

data.assets[].table.locator string required

Where the table sits in the article, for example `section[1]/table-wrap[1]#T1`.

data.gaps array required

What is missing from the data layer: every asset that was not read, downloaded or licensed.

analysis object required

The analysis: the methods named, the operations they need and the published target a run would check.

analysis.state enum required one of absent, partial, complete, blocked

How complete the analysis layer is.

analysis.methods array required

The methods the paper used.

analysis.methods[].id string required

The identifier of the method within the record.

analysis.methods[].family string or null required

The family the method belongs to, for example `statistical_test`.

analysis.methods[].specific string or null required

The specific method, as the paper names it.

analysis.methods[].label string or null required

A short human-readable name for the method.

analysis.methods[].dsl_operations array required

The engine operations the method needs.

analysis.methods[].missing_operations array required

The operations the engine does not have yet.

analysis.methods[].expressible boolean or null required

Whether the method can be expressed with the operations the engine has.

analysis.methods[].parameters object required

The parameters of the method, keyed by name.

analysis.methods[].data_shapes array required

The shapes of data the method consumes.

analysis.methods[].evidence object required

How the method was identified, and how sure the engine is.

analysis.methods[].evidence.score number or null required

The measured agreement score behind the identification.

analysis.methods[].evidence.band string or null required

The interval the agreement score is reported with.

analysis.methods[].evidence.quote string or null required

The text the method was identified from.

analysis.targets array required

The published values a run would check itself against.

analysis.targets[].id string required

The identifier of the target within the record.

analysis.targets[].claim_id string or null required

The claim the target belongs to.

analysis.targets[].operation string or null required

The engine operation that produces the value.

analysis.targets[].expected_value number or null required

The published value.

analysis.targets[].unit string or null required

The unit of the published value.

analysis.targets[].tolerance number or null required

The tolerance a recomputed value must fall inside.

analysis.targets[].state enum required one of absent, partial, complete, blocked

How complete the target is.

analysis.targets[].note string or null optional

A note about the target, in the words of the source.

analysis.gaps array required

What is missing from the analysis layer.

capability object required

The capabilities the experiment needs to run.

capability.state enum required one of absent, partial, complete, blocked

How complete the capability layer is.

capability.requirements array required

Each capability the record needs.

capability.requirements[].capability_id string required

The identifier of the capability in the engine's catalogue.

capability.requirements[].label string or null required

The human-readable name of the capability.

capability.requirements[].assay_families array required

The assay families the capability belongs to.

capability.requirements[].availability_state string or null required

Whether the capability is owned, candidate or missing.

capability.requirements[].tier_unlocked string or null required

The readiness tier the capability unlocks when it exists.

capability.requirements[].candidate string or null optional

Whether the capability is a candidate for building.

capability.gaps array required

What is missing from the capability layer.

readiness object required

How ready the record is to run, and what has to be resolved first.

readiness.state enum required one of absent, partial, complete, blocked

How complete the readiness layer is.

readiness.tier enum required one of R0, R1, R2, R3, R4, R5,

The readiness tier: R0 runs today, R5 needs the most.

readiness.blocking_axis enum required one of method, capability, data, none,

Which axis blocks the record: method, capability, data or none.

readiness.blocked_on string or null required

The specific blocker, in the form `axis:reason`.

readiness.axes object required

The state of each axis: method, capability and data.

readiness.can_run object required

What can run today: the operations, the assets and the steps that are available.

readiness.can_run.dsl_operations array required

The engine operations that can run today.

readiness.can_run.assets array required

The assets that can be read today.

readiness.can_run.steps array required

The protocol steps that can run today.

readiness.must_build object required

What has to be built before the record can run.

readiness.must_build.dsl_operations array required

The engine operations that have to exist.

readiness.must_build.dsl_operations[].operation string required

The operation that has to exist.

readiness.must_build.dsl_operations[].unlocks_papers integer or null required

How many papers that one operation would unblock.

readiness.must_build.dsl_operations[].priority integer or null required

The order the operation should be built in.

readiness.must_build.converters array required

The converters that have to exist, named by the format they read.

readiness.must_build.readers array required

The readers that have to exist, named by the format they read.

readiness.must_build.capabilities array required

The capabilities that have to be built.

readiness.must_build.data_gaps array required

The data that has to be obtained before the record can run.

readiness.gaps array required

What is missing from the readiness layer.

review object required

What the readers made of the paper, and where that critique is published.

review.state enum required one of absent, partial, complete, blocked

How complete the review layer is.

review.machine_critique object or null required

The machine critique of the record.

review.human_critique object or null required

The human critique of the record.

review.stages array required

The review stages the record has been through.

review.gaps array required

What is missing from the review layer.

execution object required

Whether the experiment has been run, and the evidence if it has.

execution.state enum required one of absent, partial, complete, blocked

How complete the execution layer is.

execution.status string or null required

The execution status, as the run reports it.

execution.mode string or null required

Whether the run was real or synthetic.

execution.runs array required

The runs of the experiment, each with its evidence.

execution.gaps array required

What is missing from the execution layer.

provenance object required

Where every part of the record came from.

provenance.entities array required

The things the record is about: papers, datasets, instruments, people.

provenance.activities array required

The acts that produced the record: reading, extraction, conversion, publication.

provenance.agents array required

Who or what carried out each activity.

provenance.source_run_id string or null required

The corpus run the record was read from.

provenance.dsl_catalogue_fingerprint string or null optional

The fingerprint of the operation catalogue the record was built against.

provenance.capability_table_fingerprint string or null optional

The fingerprint of the capability table the record was built against.

provenance.snapshot_id string or null optional

The corpus snapshot the record was read from.

lineage object or null optional

The version this record supersedes, and the generation it belongs to.

lineage.parent_id string or null optional

The identifier of the record this one supersedes.

lineage.parent_hash string or null optional

The exact content hash of the revision this one replaces.

lineage.fork_of string or null optional

The record this one is a fork of, for a user-authored fork.

lineage.generation integer or null optional

How many revisions deep the record is.

assays array optional

Every assay the experiment ran, in the order it ran them. An experiment is a set of processes and each process is an assay: the measurement, the analysis, the validation, the normalisation. Every binding here is required for the experiment as described; there is no primary assay and no optional one. A consumer that wants to run the experiment walks the list in order.

assays[].id string required

This entry's identifier within the record. It is required, because a request or a result names it and a reference that may be missing is not a reference. The order of the list is the order of the experiment, so the identifier is the position: `assay:1` is the first assay the experiment ran.

assays[].role string or null optional

What this process is: the measurement, the analysis, the validation, the normalisation. Free text. It names the process; it does not make the process optional, because every assay in the list is required.

assays[].order_basis string or null optional

How this assay's place in the order was established: `paper_text_order:<offset>` when the paper's own text describes this process at that offset, `consumes_earlier_output` when it consumes what the earlier ones produced. Free text. The order of the list is the order of the experiment.

assays[].depends_on array or null optional

The assay entries of this document whose output this entry needs, named by their identifiers within this document. It is a statement about the experiment as it ran, and it is written, never inferred from the order of the array: an entry that states no dependency states no dependency, and an ordered array does not say that each entry needs the one before it. A dependency names an entry in this document; it does not cross to another record, and the dependencies of a document do not form a cycle.

assays[].definition_id string optional

The identifier of the assay definition.

assays[].definition_version string optional

The version of the assay definition.

assays[].definition_digest string or null optional

The checksum of the assay definition.

assays[].required_operations array optional

The operations the record needs from that assay definition.

assays[].support_state enum optional one of supported, partial, unsupported, unknown

Whether the platform supports the record's operations: supported, partial, unsupported or unknown.

assays[].source_hashes object optional

The checksums of the artefacts the binding was made against.

assays[].confidence number or null optional

How strongly the evidence supports this binding, in [0,1]. An evidence score, not a probability.

assays[].evidence array optional

The passages of the paper that support this binding, so the binding can be checked against the paper's own text.

assays[].definition object or null optional

The assay definition this record belongs to, with the contract the record relies on. The definition itself is owned by the catalogue; this is the binding.

assays[].definition.definition_id string required

The identifier of the assay definition.

assays[].definition.definition_version string required

The version of the assay definition this record is bound to.

assays[].definition.definition_digest string or null optional

The checksum of the assay definition, so the binding names the exact definition it was made against.

assays[].definition.required_operations array optional

The operations the record needs from that assay definition. The list is an array of operation names.

assays[].definition.support_state enum optional one of supported, partial, unsupported, unknown

Whether the platform supports the record's operations: supported, partial, unsupported or unknown.

assays[].definition.input_contract object or null optional

What the assay definition takes as input, as the catalogue states it.

assays[].definition.parameter_contract object or null optional

The parameters the assay definition accepts, and their types and defaults, as the catalogue states them.

assays[].definition.output_contract object or null optional

What the assay definition returns, as the catalogue states it.

assays[].definition.limitations array or null optional

The standing limitations of the operation: what it cannot establish and the assumptions it rests on. They live here and not in `analysis.methods[]`, which forbids extra keys: an operation's limits are a property of the operation, and the catalogue owns the operation.

assays[].definition.interpretation string or null optional

How a result of this assay should be read: what a reproduction of it would and would not show. Free text, written by the assay's owner.

assays[].request object or null optional

A request to run one operation against one source record. The source record id and the hash of its body bind the request to the version that was asked for.

assays[].request.assay_id string required

The identifier of the assay entry in the source document's `assays[]` that this request runs. It is required. Together with `source_content_hash` it says exactly which entry, in which version of the record, the request concerns.

assays[].request.source_record_id string required

The identifier of the record the request runs against.

assays[].request.source_content_hash string required

The hash of that record's body when the request was made, so the request is tied to one version of it.

assays[].request.operation string required

The DSL operation the request asks for.

assays[].request.operation_version string or null optional

The version of that operation the request asks for.

assays[].request.inputs array optional

The input values the request carries, one object per input.

assays[].request.parameters object or null optional

The parameters the request sets, as the operation takes them.

assays[].request.request_id string or null optional

The identifier of the request itself, so the same request can be made twice and told apart.

assays[].request.provider string or null optional

The provider the request asks to run it.

assays[].result object or null optional

The outcome of one request. The input, result and evidence hashes bind the outcome to the input it came from and to the evidence for it.

assays[].result.assay_id string required

The identifier of the assay entry in the source document's `assays[]` that this result belongs to. It is required, so an outcome always names the assay it came from.

assays[].result.operation string or null optional

The operation this result is the result of. An assay entry may be run as more than one operation, so a result that does not say which operation produced it says less than the reader needs.

assays[].result.provider string or null optional

The provider that ran it.

assays[].result.execution_id string required

The identifier of the execution that produced the outcome.

assays[].result.input_hash string or null optional

The hash of the input the run was given.

assays[].result.result_hash string or null optional

The hash of the result the run produced.

assays[].result.evidence_hash string or null optional

The hash of the evidence the run produced with the result.

assays[].result.status string required

What the run did: succeeded, failed, refused, or whatever vocabulary the provider uses.

assays[].result.finding string or null optional

What the run found, in one sentence.

assays[].result.result object or null optional

The operation's own result body, as the operation returns it.

assays[].operations array or null optional

The operations this assay may be run as, each with the contract of that operation. An assay definition can implement several operations and the definition is still one assay, so the operations are stated inside the entry rather than by repeating the assay in the array. Optional: an entry that states no operations says nothing about how it may be run.

assays[].operations[].operation string required

The name of the operation, as the owner of the catalogue publishes it, so a consumer can look it up.

assays[].operations[].operation_version string or null optional

The version of the operation, when the owner publishes one. The name alone does not say which version of an operation was run.

assays[].operations[].support_state enum optional one of supported, partial, unsupported, unknown

How well this operation can be run today: supported, partial, unsupported or unknown. It is a property of this operation and can differ from the support state of the assay that contains it.

assays[].operations[].input_contract object or null optional

What this operation takes as input, as the catalogue states it.

assays[].operations[].parameter_contract object or null optional

The parameters this operation accepts, and their types and defaults, as the catalogue states them.

assays[].operations[].output_contract object or null optional

What this operation returns, as the catalogue states it.

assays[].operations[].limitations array or null optional

The standing limitations of this operation: what it cannot establish and the assumptions it rests on.

assays[].operations[].interpretation string or null optional

How a result of this operation should be read: what a reproduction of it would and would not show. Free text, written by the owner of the operation.

gaps array required

Every gap in the record, gathered from the layers.

generator.name free optional

The name of the tool that wrote the record, for example `corpus-assay-engine`.