The FormScript corpus

Real questionnaires, published by real organisations, converted to FormScript by scripts/corpus/convert-fhir.ts. This report is generated by npm run corpus:report, which re-reads every converted document and runs it through the lexer, the parser, the analyzer and the engine before writing a word of it.

The headline

• 73 sources from 24 publishers, covering 31 domains: 71 questionnaires and 2 answer sets.
• 69 forms converted from 4563 source questions into 3079 FormScript fields.
• 69 of 69 parse and analyse with no error.
• 69 of 69 evaluate: the engine builds a transaction and produces an output document from each one.
• 5 carry an advisory from the language itself (see below).
• 2 questionnaires rejected outright, with a code — so 69 + 2 + 2 = 73.
• 4624 gaps were recorded, of which 274 lossy — something the source said that FormScript cannot say.

The lossy gaps are the interesting half. They are where the language stops.

The rejects

The rejects come first, because they are the measurement.

The domains

The sources

The upstream files are not redistributed. corpus/manifest/fhir-sources.json pins each one by URL and SHA-256; npm run corpus:fetch reads them again and checks the bytes it gets against the pin.

What the toolchain said

The output document holds fewer keys than the form declares outputs in some documents. That is the language working: a field that is inapplicable — its when did not hold — is omitted from the output document, because "not asked" and "asked and left blank" are different facts (Spec-4.35). The Zika assessment and the cardiology form are the two with conditional questions.

What could not be carried over

Every gap names the FHIR path that produced it. A gap is either lossy — something meaningful was dropped — or informational, which records something that does not change what the form means.

Advisories from the language itself

5 of the forms carry W-VAL-01, which Spec-10.11 requires: a required constraint on a boolean field is reported at compile time with a message that suggests accepted. The converter keeps required, because that is what the source asked for: FHIR's required on a boolean means the question must be answered, not the answer must be yes. The advisory is the language steering, and the corpus keeps it visible.

• WHO-smart-ccc-registration-e — 1 advisory warning
• WHO-smart-ccc-measurements — 1 advisory warning
• WHO-smart-ccc-danger-signs — 7 advisory warnings
• WHO-smart-ccc-breastfeeding — 7 advisory warnings
• numde-generic — 5 advisory warnings

Document by document

Each row is one converted form. The gap codes say what was lost; corpus/reports/fhir-conversion.json holds every gap with the FHIR path that produced it.

A real answer set, replayed

The SDC implementation guide publishes the answers a real clinician gave in the cardiology form — MOMO ABBAS. This script pushes those answers through the FormScript document the converter produced and checks the output document against the answers that went in.

• 43 answers in the response
• 43 of them carried into the converted form
• 0 could not be carried, and the reason is recorded
• 42 output values matched the answers exactly, 0 did not

The corpus holds 2 answer sets. This one is replayed because the questionnaire it answers is in the corpus; the other belongs to a form that is not, so there is nothing to push it through.

Questions the corpus has raised for the specification

These are not defects the corpus fixes. They are design questions the corpus found, and each one is checked when this report is written rather than asserted from memory. The probe is what settles the question: the specification is checked against it, not the other way round.

Does a field have to be a leaf?

• the specification says: Spec-7.1 said "leaf"; Spec-7.16 says a field may contain blocks
• the probe is: a field that contains another field
• what the toolchain does: the analyzer reports 0 errors and 0 warnings; the engine evaluates it and projects 2 output keys
• what the corpus did: The corpus asked, and the specification answered in r9: a field may contain blocks, and a field that does is a composite field (Spec-7.16). A source question that has child questions is now converted into a composite field, so its own answer is kept instead of being dropped as GAP-NESTED-LEAF. The one case the language still cannot hold is a repeating question with children, and that is recorded as GAP-NESTED-REPEATING.

Reproducing this

npm run corpus:fetch     # read the upstream files, refresh the manifest
npm run corpus:convert   # convert them, write the gap log
npm run corpus:replay    # push a real answer set through a converted form
npm run corpus:report    # re-check every document and write this report