Baselines

Track 1 · Subtrack 1 — global equivalence alignment

Repaired coherence-aware P/R/F1

Macro-averaged over the three pairs (NCIT–DOID, SNOMED–FMA, SNOMED–NCIT); sorted by the headline column. See evaluation metrics.

Track 1 · Subtrack 2 — local equivalence ranking

MRR and Hits@k (standard reference)

Macro-averaged over the three pairs (NCIT–DOID, SNOMED–FMA, SNOMED–NCIT); sorted by the headline column. See evaluation metrics.

About these baselines

Organiser baselines (macro over the three pairs). Global = semi-supervised test-slice (option a): repaired coherence-aware F1 headline. Local ranking is scored against the standard reference — coherence repair applies to the global task only. Participant results appear on the CodaBench leaderboards. The global-alignment headline is the repaired coherence-aware F1; local ranking is headlined by MRR against the standard (unrepaired) reference. Full data: leaderboard.json. Participant results appear on the results page, fed by the CodaBench leaderboards. How long each baseline took, and the machine it ran on, are in wall-clock times below. * Lexical-Baseline is a character tri-gram similarity matcher, not an exact-string baseline.

Baseline coherence

Baseline coherence — the alignments the systems produce

Alongside P/R/F1, each global alignment is checked with a reasoner: does the system's own alignment leave the merged ontology satisfiable? Each baseline's submitted mappings are merged with both ontologies and classified, giving the number of unsatisfiable named classes and the incoherence degree (that count over the merged named-class signature, in parentheses). Lower is better; 0 is coherent. Rows are in the same order as the P/R/F1 table above, so the two can be read across.

Counted in the merged ontology (O_src ∪ O_tgt ∪ M), where M is the system's own alignment. Every ELK figure is a sound lower bound; HermiT is exact but only terminates on NCIT–DOID at this scale, so the SNOMED pairs carry ELK bounds only. As with the reference figures below, correspondences involving owl:deprecated classes are excluded as out-of-task, so the two tables are on the same basis. Two cells are genuinely unavailable (—): AML and LogMapLt on SNOMED–FMA did not produce a result. Coherence is not a quality ranking and does not track F1 — a system that maps very little scores well here, and LogMap and AML repair internally by design. Machine-readable data: baseline_coherence.json.

Reference coherence

Reference coherence — before vs after repair

The Task-1 reference alignment is grounded in UMLS/Mondo and can be logically incoherent; the repair procedure removes or weakens the incoherence-inducing correspondences. This table reports the coherence of each pair's reference before and after repair — the number of unsatisfiable named classes in the merged ontology and the corresponding incoherence degree. All three repaired references are coherent (0 unsatisfiable classes).

Unsatisfiable classes are counted in the merged ontology (O_src ∪ O_tgt ∪ R) under the ELK reasoner; the incoherence degree is that count over the merged named-class signature. † denotes an ELK lower bound (a sound under-approximation; the exact figure is pending). The all-equivalence standard SNOMED–FMA merge does not classify under ELK even at a 128 GB heap, so its before-repair figure is unavailable (—). Machine-readable data: reference_coherence.json.

Runtime

Wall-clock times — and the machine that produced them

How long each baseline actually took, per pair and over all three. The Subtrack-1 figures cover the matching phase that produced the system's global alignment; the Subtrack-2 figures cover the ranking run behind the MRR / Hits@k above. Every run was on the one workstation specified below.

Subtrack 1 — global equivalence alignment

matching: producing each system's global alignment over the two full ontologies

LogMap: The 24-hour SNOMED–FMA cell is the outlier; the other two pairs finish in minutes to hours. BERTMap: Includes the in-run BERT fine-tune on each pair's own ontology synonyms.

Measured as: AML / LogMap / LogMapLt: per-cell wall-clock recorded, cells match, write and score. BERTMap / BERTMapLt: perform end-of-matching, excludes the ranking.

Subtrack 2 — local equivalence ranking

ranking: scoring every candidate list in the test split

BERTMap-ss: Fine-tunes fresh on the public train+valid maps for every split — the fine-tune must not be shared across splits. BERTMap: Reuses the fine-tuned checkpoint from its Subtrack-1 run, so the fine-tuning cost is counted in the matching table above and not here.

Measured as: Per-cell wall-clock recorded by the baseline runner for the ranking-only runs that produced the published MRR / Hits@k.

Hardware

Every baseline runs locally on a machine with the following specifications.

Every figure is a single run, measured values should be considered as follows: the BERTMap-family ranking runs were launched in pairs, one per GPU; a second matcher process was sharing the host's CPU and memory bandwidth throughout. The reasoner coherence checks reported above are separate steps that are not counted here. Measured June 2026. Machine-readable data: baseline_runtime.json.

Editions

Baselines and results by edition

OAEI Bio-ML 2026 is the current edition. As the track matures, each edition's final baselines and participant results will be archived here, and changes between editions are recorded in the changelog.