Concept Dictionary — How it works

Overview

The concept engine is the platform's controlled vocabulary (Zhenus_UHP-style): the single source of coded meaning that clinical data, form fields, program outcomes, and attributes bind to. Concepts are system-wide (centrally curated, with imported/provided ids), named in multiple locales, answerable and set-organised, and mapped to external standards. A read-only validation API lets other engines confirm a concept id/code is valid and active.

Data model & ownership

Table	Purpose
concept	The core concept (class + datatype), system-wide id.
concept_class / concept_datatype	Reference types (seeded; BIGINT ids, OpenMRS-compatible).
concept_name	Names/synonyms per locale; preferred name.
concept_answer	Allowed answers for a coded question (CODED only).
concept_set	Set membership (self/duplicate guarded).
terminology_source	The registry of terminologies whose codes we record — LOINC, SNOMED CT, ICD-10, RadLex, CIEL, UCUM (TERM-001).
concept_reference_term	One code within one terminology, with identity of its own so a retirement or a display can hang off it (TERM-009).
concept_mapping	Mapping of a concept to a concept_reference_term.
concept_map_type	The 71 relationships a mapping may assert — four equivalence types plus the SNOMED relationship qualifiers (TERM-016). Administrable at Admin → Terminology → Map types.
concept_version	Append-only version snapshots.
concept_numeric	Numeric metadata for a concept: unit (FK to a UCUM term), normal/critical/absolute ranges, precision.

Key rules & invariants

• source_system is a foreign key onto terminology_source, not free text (TERM-001). All three columns that carry it — on concept, concept_mapping and concept_version — reference terminology_source(code), ON UPDATE CASCADE. Before this, a typo did not fail: it quietly became a new terminology, and the mapping filed under it was never found again. 169,683 mappings depend on these strings.

  The registry is seeded verbatim and merges nothing. The 31 values look like drift and mostly are not: SNOMED CT / SNOMED NP / SNOMED MVP / SNOMED UK / SNOMED US, and ICD-10-WHO / ICD-10-WHO 2nd / NP / NP2, are distinct sources in the CIEL dictionary, not misspellings of each other. Collapsing any pair would silently re-point tens of thousands of mappings onto the wrong terminology — a clinical data decision needing its own migration, never a tidy-up.

  Consequently the registered spelling is authoritative: query SNOMED CT with the space, not SNOMED-CT. A registry row's active flag is distinct from voided — retiring a terminology stops new mappings being filed against it without invalidating those already recorded.

  ⚠ Domain modules must not build their own code-system tables. imaging_code_system did exactly that and is removed by TERM-003 (which absorbs TERM-002); imaging procedures now bind to concept_id and validate over Feign instead of a local registry.

• A numeric concept's unit is a UCUM code, not a string (TERM-006). concept_numeric.units was a free-text VARCHAR(60) written verbatim, with no UCUM binding, no normalisation and no validation. It is now units_term_id, a foreign key onto a concept_reference_term under the UCUM terminology_source, and the old text column is gone.

  Why this one mattered most. That column does two jobs: it labels the value a clinician reads and it parameterises the normal / critical / absolute range checks on the same row. A concept seeded mcg and later edited to ug or mg gives a range check passing on the wrong scale, displayed with a unit nobody flagged — the only finding in the terminology audit with a direct dosing-error path.

  The foreign key is not the whole rule. concept_reference_term also holds LOINC, SNOMED and RadLex codes, so the constraint only guarantees "a registered code". "…and it must be a UCUM one" is enforced in ConceptUnitServiceImpl#resolveForWrite, which answers 400 naming the terminology the term actually belongs to ("…belongs to LOINC, not to UCUM") rather than a bare "invalid unit".

  Nothing is ever guessed and case is never folded. UCUM is case-sensitive by design: mg is a milligram, Mg a megagram; u is the micro prefix, U an enzyme unit. So MG is refused, and mcg, IU and mmHg are refused rather than mapped onto ug, [IU] and mm[Hg]. Each of those is probably what the author meant, and "probably" is not a standard a dosing unit may be settled by. The concept/012 migration takes the same line: it resolves stored strings by exact match only and HALTs with the offending rows named if any does not resolve.

  A unitless numeric concept is legitimate — an Apgar score, a pain score, a plain count — so units_term_id stays nullable. What the ticket removed is an unvalidated unit, not the absence of one.

  GET /api/v1/concept/units?search= lists the seeded set for a picker. It is read-only: the seed covers the practical set (mass, volume, concentration, activity, pressure, time, rates, cell counts, dose, radiation, and the {annotation} forms), and a deployment adds a missing code as a concept_reference_term under UCUM — never as a second units table, which is the mistake TERM-003 (absorbing TERM-002) undid for imaging procedures. UCUM is a generative grammar, not a code list, so enumerating all of it is not possible.

• conceptId is optional on create, and an import can never overwrite one (TERM-012). concept.concept_id had no sequence behind it, so nothing could allocate an id and ConceptDto.conceptId was @NotNull — creating a concept meant the caller invented a primary key. It is now:

  Create carries	What happens
  a conceptId	used as given — a steward pinning a known dictionary row is a supported workflow
  no conceptId	allocated from concept_concept_id_seq via concept_next_id() and returned
  a conceptId already taken	409, naming the id (CONCEPT_ID_ALREADY_EXISTS) — including when the holder is voided, because voiding does not release an id

  There is no reserved id band and none is needed. concept_id is our own surrogate key and the standards never occupy it: SNOMED, LOINC, ICD and RxNorm attach through concept_mapping, and each row's own source id lives in concept.external_code. The seed already mixes 51,283 CIEL rows with 3,806 ZHENUS_UHP rows in one id space with no band between them. A local concept and an imported one may therefore share an external_code and are told apart by source_system — the partial unique index is on the pair, not on external_code alone.

  The sequence is seeded by reading MAX(concept_id), never from a literal. It is 167096 in today's seed and the next CSV refresh moves it, so a literal would be right on the day it was written and silently wrong afterwards. concept_next_id() re-checks the value it produced against the table, which covers the one way the two can drift: a deployment that starts without the concept-dictionary context seeds the sequence from an empty table, and turning the context on later loads dictionary rows the already-applied changeset will never re-seed against.

  A dictionary refresh must never write to concept directly. Load the CSV into concept_import_staging, then SELECT concept_import_promote(); — it refuses the whole batch and names the colliding ids if any concept_id is already present, and it only ever INSERTs. This is the clause that matters: a loadUpdateData straight into concept would UPDATE the colliding row, and every observation already recorded against that concept would silently change meaning, with no error and nothing in the data to notice it by. The 013-refuse-pending-concept-import-collisions changeset carries the same check as a sqlCheck / onFail: HALT precondition, so a batch staged and never resolved stops the next deployment instead of being promoted by it.

• A concept is written whole, in one transaction — and edits are not all the same edit (TERM-013). A concept is not one row: it is the concept row plus its names, its answers when CODED, its numeric metadata when NUMERIC, its set members and its standards mappings. Those used to be written through six separate endpoints, so building one was a POST followed by N sub-resource calls, and a failure part-way left a concept that was already visible, already selectable, and missing the parts that gave it meaning. POST and PUT /api/v1/concept/concepts now take the whole thing and persist it together or not at all. The sub-resource endpoints remain for targeted edits.

  names is required on every write. A concept without one is selectable, codeable, and blank in every picker — not a hypothetical, since the triage create-and-map path produced exactly those, already carrying coded clinical rows. Per locale the writer enforces: exactly one FULLY_SPECIFIED, at most one SHORT, exactly one localePreferred, and no repeated text; SYNONYM and INDEX_TERM repeat freely. FULLY_SPECIFIED and SHORT are both wanted — the unambiguous "Diabetes mellitus type 2" and the "T2DM" that fits a column header. ⚠ Several names flagged preferred at once is a real past defect: the preferred name then resolved alphabetically across languages, and concept 5089 came back with its Kinyarwanda name.

  What each change does, because "every change voids and re-inserts" is wrong here — the distinction is what the value is:

  Change	Action	Why
  which name displays in a locale	switch locale_preferred	both names are still true; voiding one destroys a valid synonym
  a name's text	void the old row, insert a new one	a different string — the old one may already be printed on a document and must stay answerable
  SYNONYM promoted to FULLY_SPECIFIED	switch the type	the text did not change, only its role
  a name added / dropped from the payload	insert / void	the payload is the complete name set
  code, concept_class_id, concept_datatype_id	supersede — retire the original, insert a successor, link them	these change what the concept means; observations already recorded must keep pointing at what was meant when they were written
  active	flip it in place	this is retire; it must never mint a new concept_id

  ⚠ Retire is active = false, never voided. Concept carries @SQLRestriction("voided = false"), so a voided concept disappears from ordinary reads and every observation coded against it resolves to nothing — silence, not "this was retired". Once clinical rows reference a concept, voiding is not available. concept has active and no retired column.

  ⚠ A superseded concept has a forwarding address. concept.superseded_by_concept_id (concept/014, self-FK, indexed, CHECK it is not itself) records which concept replaced it — without it a superseded concept is just an inactive row and a reader holding an old observation has nowhere to go. So a PUT that changes the code, class or datatype answers with the successor, under a different conceptId from the one in the path. Read the id back rather than assuming it.

  Child collections: absent and empty differ. A null collection is not managed by this request and is left as it stands; an empty one means remove them all. Collapsing the two would make every composite update destructive for a caller that only wanted to correct a code. names is the exception — it is required, so it has no null case.

• Concept ids are system-wide and stable so coded references are portable.

• Answers apply only to CODED questions; set membership rejects self/duplicate. Supplying answers for a non-CODED concept, or numeric metadata for a non-NUMERIC one, is refused (400), not ignored — dropping them silently would let an author leave the screen believing a TEXT concept has an answer set and discover otherwise when the form renders an empty picker in a consulting room.

• Versions are append-only; voided concepts fail validation — a consumer binding to a voided concept is rejected.

API

See the API Reference. Endpoint groups under /api/v1/concept: concepts, classes, datatypes, names, answers, sets, mappings, versions, terminology sources, read-only units of measure (GET /api/v1/concept/units), and read-only concept validation (resolve by id / source+code / mapped code, with deterministic reasons).

POST /api/v1/concept/concepts and PUT /api/v1/concept/concepts/{conceptId} are the composite write: they carry names (required), answers, numeric, setMembers and mappings alongside the concept's own fields and persist all of it in one transaction. GET /api/v1/concept/concepts/{id} takes ?includeChildren=true to read the same aggregate back for an editor; it is off by default because that endpoint is also the form builder's per-answer resolver, called once per answer in a set.

PUT /api/v1/concept/concepts/{conceptId}/numeric takes unitsTermId. units is still accepted as a compatibility path when unitsTermId is absent — matched exactly against UCUM, never case-folded, and rejected outright when it does not resolve — and is always resolved output on read, so existing readers (the patient summary panel, the observation field contract) see the same units string they did before.

Mapping coverage (TERM-015)

GET /api/v1/concept/concept-mappings/coverage?terminologySourceId=… answers one question: which concepts cannot yet be expressed in this vocabulary? It is the input to the mapping backlog screen (FE-388) and the go/no-go check before an integration that speaks LOINC or SNOMED goes live.

Parameter	Default	Meaning
terminologySourceId	—	The vocabulary to measure against. Required.
activeOnly	true	Exclude retired concepts. Nobody can select them, so counting them inflates the backlog and hides the real work.
page / size	0 / server default	The page of unmapped concepts.

The response carries unmapped (one page), plus unmappedCount and totalInScope (both totals). Read progress from the counts, never from unmapped.length — the seeded dictionary is ~55k concepts, so against an untouched source the first page is 20 rows out of tens of thousands.

Four properties are deliberate, and each of them is a way this endpoint could have lied:

• Coverage is per source. A concept mapped to SNOMED CT still appears under LOINC. Treating any mapping as coverage of all of them is how an integration goes live believing it can speak a vocabulary nobody mapped.
• Voiding a mapping returns its concept to the backlog. The query is native and asks NOT EXISTS (… AND m.voided = false) on purpose: a derived query would go through ConceptMapping, whose @SQLRestriction("voided = false") hides voided rows, so a retired mapping would keep counting as coverage — the concept would leave the work list at the exact moment it rejoins it. Retiring a bad mapping is someone saying this still needs doing.
• A source this deployment does not hold answers available: false, with a reason — never 0 unmapped. CPT-4 is AMA-licensed and dm+d is not seeded here, so "we never checked" and "nothing left to do" are opposite answers that must not share a representation.
• activeOnly=false widens both numbers, so the retired tail is visible when someone wants it rather than silently folded into the headline.

Saving a mapping from the screen is the existing write — POST /api/v1/concept/concepts/{conceptId}/mappings — so a concept leaves the backlog by the same path that any other mapping is recorded, and the coverage view has no write of its own to keep in step.

Configuration & feature flags

Property	Default	Purpose
spring.liquibase.contexts / LIQUIBASE_CONTEXTS	concept-dictionary	When concept-dictionary is present, Liquibase concept/006 loads the Zhenus_UHP/CIEL CSV dictionary (~55k concepts, names, answers, sets, mappings) at startup. Omit that context (e.g. integration tests use integration) to skip the bulk seed.

All concept dictionary tables use BIGINT primary keys (OpenMRS-compatible import ids). The uuid column remains a separate unique string for interchange.

Re-export from a local dictionary MySQL database:

python scripts/concept-import/export_zhenus_uhp_concepts.py \
  --host 127.0.0.1 --port 3306 --user creativity --password creativity --database emr

CSVs land under core/src/main/resources/db/changelog/concept/data/.

⚠ A refresh is not a re-run of concept/006. concept/006 is guarded by onFail: MARK_RAN and skips entirely once the dictionary is present, so it will not pick up new rows. A refresh is a new changeset that stages into concept_import_staging and calls concept_import_promote() — see scripts/concept-import/README.md for the procedure and concept/013 for the template. Never loadData or loadUpdateData straight into concept.

Related features

• Clinical, Form, Program, and Demographic attributes all bind to concept ids. Domain plug-ins read via exchange.client.concept (ConceptClient).

---

Why it is this way

Retirement is active = false, never voided. The dictionary distinguishes "this concept was a mistake" from "this concept is no longer in use". Voiding a concept that has been used would orphan every observation coded against it; retiring it keeps history readable while removing it from pickers.

Only SAME_AS mappings are exported as foreign codings (TERM-014). NARROWER_THAN and BROADER_THAN say something weaker than equality, and publishing them as if they were equivalent tells a receiving system that two things are the same when they are not.

The map-type vocabulary (TERM-016)

map_type is a four-value enum on concept_mapping, and the dictionary it imports from publishes 71 relationships. concept_map_type holds them all, and part 3 gave it an entity, CRUD at /api/v1/concept/concept-map-types and a screen — the second arm of modelling rule 1, since a seeded table nobody can administer is a free-text column with extra steps.

The 71 split into two kinds, and the difference is the whole point:

Kind	Count	Meaning
Equivalence	4	SAME-AS, NARROWER-THAN, BROADER-THAN, RELATED-TO. The external code may stand in for the concept — this is what makes a mapping usable for a clinical write.
Relationship	67	SNOMED qualifiers: Finding site, Laterality, Causative agent, … They say how two clinical ideas relate, and are never a substitution.

⚠ The kind is derived from the name, never stored and never accepted from a client. A stored flag would need maintaining every time the enum changed, and forgetting means a relationship type silently becoming write-safe; a client-settable one would let a new entry declare itself usable for clinical writes.

⚠ The four equivalence types cannot be retired or renamed out of the set. findWriteSafeCandidates gates FHIR writes on map_type = 'SAME_AS', so those names are load-bearing. Hide one from pickers instead — that withdraws it from new mappings without asserting the existing ones are wrong. The screen shows no Retire control on those rows rather than offering one and answering 400.

⚠ In practice all 171,887 mappings still use only the four enum values, because import flattened them. The other 67 types are seeded and unused: ConceptMapping.mapType is still the enum, and letting a mapping carry a relationship type touches the FHIR write path, so it is a separate change.

Concept ids are numeric, not UUIDs. They are referenced by clinical data in bulk, and a stable compact identifier matters at that volume.

Traps

⚠ Changing code, conceptClassId or conceptDatatypeId changes what existing data means — ConceptController. Observations already recorded against the concept are not migrated, so an edit here rewrites the meaning of history rather than correcting it.

⚠ Ambiguity is refused, not resolved — ConceptMappingServiceImpl. Two concepts claiming the same external code is a data problem; picking one would attach clinical data to whichever happened to sort first, and the wrong choice is invisible afterwards.

⚠ A local concept with no SAME_AS mapping is perfectly usable internally but cannot be exported as a standard code — nothing says what it means to an outside system. Worth knowing before creating local concepts in bulk.

⚠ Concept names are locale-resolved, and a concept can hold several locale-preferred names. A search that finds nothing may be searching the wrong locale rather than a missing concept.