Linguistic interchange and transliteration

These native Rust profiles belong to Standard/Lite and Pro. Core can inspect and format their literal syntax, but needs a Standard parser binding to compile it. All three literal delimiters work; ordinary quotes process Grid escapes, while triple quotes and angle blocks retain the payload. Compilation expands literals into native constants. The compiled values need no parser to reload.

Tag / function	Native result	Deliberate scope
textgrid / PARSE_TEXTGRID	Tiers and ordered interval/point annotations, seconds, UTF-8 source spans	Praat full, short and mixed text. Doubled quotes, multiline strings, comments and empty labels. Positive adjacent intervals and increasing point times. Retains Praat backslash trigraphs literally. No binary TextGrid or legacy byte decoding.
eaf / PARSE_EAF	Tiers, linguistic types, time slots, annotations with inherited start/end, ordered document elements	ELAN EAF 2.8/3.0 XML; integer milliseconds. Validates IDs, tier/type/time references and cycles. Missing times stay null. Metadata and media references retained without opening resources. Structural profile, not complete XSD, controlled-vocabulary or stereotype validation.
conllup (alias conlluplus) / PARSE_CONLLUP	Declared columns, sentences, raw fields and basic dependency edges	CoNLL-U Plus header; reordered/subset core columns and uppercase namespace columns; unique sentence/token IDs. Keeps _, *, comments and custom values. No custom-column interpretation, full UD tree validation or external corpus loading.
transliteration / PARSE_TRANSLITERATION	Source, transformed text, scheme, canonical transform ID and engine identity	JSON object containing exactly text and scheme. Fixed bundled Unicode transforms, not arbitrary rules.

TRANSLITERATE(text, scheme) returns the same object as the transliteration literal. Its text field is the transformed string; source retains the exact input. reversible: false means Grid makes no reversibility guarantee. A scheme may leave characters outside its repertoire unchanged. Transliteration is not translation, language detection, pronunciation inference or phonetic analysis.

roman = TRANSLITERATE("Москва", "cyrillic-latin")
name = transliteration"""{"text":"Crème brûlée","scheme":"latin-ascii"}"""
corpus = conllup"""# global.columns = ID FORM HEAD DEPREL
# sent_id = s1
1	Zoë	2	nsubj
2	sings	0	root
"""

Pinned schemes (ICU transforms also accept case-insensitive BCP 47 IDs; the separate ALA-LC table has an edition identifier):

Scheme	Transform ID
russian-alalc-2012	ala-lc:ru:2012 (native Library of Congress 2012 Russian letter table)
cyrillic-latin	und-Latn-t-und-cyrl
greek-latin	und-Latn-t-und-grek
greek-latin-ungegn	und-Latn-t-und-grek-m0-ungegn
arabic-latin	und-Latn-t-und-arab
devanagari-latin	und-Latn-t-und-deva
hangul-latin	und-Latn-t-und-hang
hiragana-latin	und-Latn-t-und-hira
katakana-latin	und-Latn-t-und-kana
japanese-latin-bgn	ja-Latn-t-ja-hrkt-m0-bgn
ethiopic-latin-alaloc	und-Latn-t-und-ethi-m0-alaloc
latin-ethiopic-alaloc	und-Ethi-t-und-latn-m0-alaloc
latin-ascii	und-t-und-latn-d0-ascii
han-latin-proper-names	und-Latn-t-und-hani-m0-prprname
ethiopic-latin-alaloc-geminate	und-Latn-t-und-ethi-m0-alaloc-geminate

The ICU engine and its compiled data are pinned to icu_experimental = 0.6.0 in the lockfile. Changing transform data requires a parser contract revision and fresh expansion identity. ALALOC means that specific bundled CLDR Ethiopic transform. It is not a declaration of conformance to every current ALA-LC romanization table, or to an independently verified historical edition. Japanese BGN is not labeled Hepburn. The Han proper-name transform is explicitly selected and does not promise context-sensitive phrase disambiguation or every Pinyin convention. A certified Hepburn profile and other ALA-LC tables still require separate data and conformance fixtures. Fifteen schemes share one parser; its contract is revision 3 after adding the independent Russian 2012 table (revision 2 added the proper-name and Ethiopic gemination transforms). Bounded IPA/X-SAMPA conversion and Penn Treebank readers are documented in the notation corpus. Existing phonetics records preserve authored IPA text but do not validate the full IPA symbol/diacritic system.

Readers use the native parser input, node, depth and output budgets, with cancellation checkpoints. Transliteration additionally caps text at 8,192 UTF-8 bytes and checks cancellation before/after the library call. ICU exposes no interruptible call or bounded allocator: this is not hard preemption or an RSS guarantee. A local unoptimized 65 KB probe proved impractical; the v1 cap is conservative and whole-document transliteration is excluded. Result-size checks are post-call. No custom rule injection, download, external entity resolution or media retrieval is available. Error offsets are payload-relative UTF-8; TextGrid/CoNLL-U Plus locate lexical errors, XML syntax errors locate the reader position, and EAF semantic/transliteration contract errors currently point at the payload start.

Editor registration supplies tag completion, hover and literal boundaries. TextGrid has a stateful grammar for doubled quotes and multiline strings; EAF uses XML, CoNLL-U Plus the tabular CoNLL-U grammar, and transliteration JSON. Ordinary quoted literals retain Grid string highlighting. Native diagnostics require the editor host's existing injected Standard parser service. These profiles do not add payload symbol rename, tier navigation, media playback or annotation editing.

Dependencies and parser work are selected only in the Standard assembly and on explicit parser/function invocation. Core gains static catalog entries, not an ICU dependency. No worker, file/network I/O, new ordinary resolve branch or per-cell capability discovery is added. Transform construction currently occurs per explicit call, with no global cache. The explicit local linguistics-cost Performance observer reports paired parse and already-expanded native-value admission distributions, including 100/1000 record cases and 1.3/7.8 KB transliteration requests. It is not a historical regression baseline or proof of ordinary resolve overhead. The dependency lockfile also advances shared yoke to 0.8.4 because ICU pattern data requires it; no unmeasured latency or memory regression claim is made.

Primary contracts: Praat TextGrid, CoNLL-U Plus, EAF 3.0 schema, ICU4X transliteration, CLDR transforms, ALA-LC tables.

Local unoptimized observations are retained in the paired cost report: 1,000 TextGrid intervals at a 4.9 ms median; 1,000 CoNLL-U Plus rows at 13.4 ms; Cyrillic transliteration at 25.5 ms for 1.3 KB and 503 ms for 7.8 KB. These are selected-path development measurements, not release-build throughput claims or an ordinary-path regression comparison.

Focused correctness evidence records seven linguistic reader tests, six native language-service metadata tests, three linguistic compiler/runtime tests and twelve editor wiring checks. The runtime tests exercise safe delimiter forms, aliases, ordinary functions, LIR reload with the compiler binding disabled, and the working example.

The later notation continuation evidence records the expanded phonetic and transliteration scope; the earlier measurements above remain historical anchors.

The Russian ALA-LC table is implemented natively, independently of ICU data. Unmapped characters are retained; it does not certify other ALA-LC languages. See the versioned contract.

Xigt interlinear corpora

xigt / PARSE_XIGT imports Xigt XML into native Grid values in Standard (Lite), inherited by Pro. This uses the existing native parser and LIR path; there is no Python or extension runtime dependency. Python is used only for independent reference fixtures during development.

The reader implements the core xigt-corpus → igt → tier → item structure and preserves ordered metadata, custom attributes and mixed metadata content. IGT identifiers are corpus-local; tier and item identifiers are checked within each IGT. References must resolve in that IGT, and item references must select the tier declared by the same reference role on their containing tier. Duplicate identities, malformed structure, unresolved references and conflicting segmentation with alignment/content fail with source diagnostics.

alignment records the annotation relationship. content and segmentation select item text; explicit item text takes precedence. Selections support whole items and nonnegative start:end ranges, with multiple ranges and selections. + concatenates; , inserts a space. Offsets count Unicode scalar characters, not UTF-8 bytes or UTF-16 units. Following the reference implementation, ranges past the text end are clamped and reversed ranges yield empty text. Combining marks count separately; the reader does not normalize text. Content cycles are rejected; alignment cycles are preserved. Referenced values resolve once and expansion is bounded by the existing parser budgets and cancellation context.

The result has igts, tiers, items, references, elements, nodes and edges, plus corpus metadata. Each item retains its authored attributes and resolved value (blank when absent). Reference rows retain their role, join operators and span selections. nodes includes all core elements, including unaligned items; edges distinguishes containment and the three reference roles. GRAPH(xigt"""...""") retains unaligned annotations. Pass the literal directly: a named native-data binding is not yet a supported live graph source. Element records preserve XML metadata without fetching or validating external schemas.

The editor uses XML highlighting for all literal forms. Local identity hover, definition, references and reference completion are source-only; item suggestions are restricted to the declared target tier and the current IGT. Completion edits replace identifiers without changing span suffixes. Entity-spelled identifiers, ambiguous duplicates and over-budget input receive no reference assistance. Rename is unavailable because custom attributes can carry references that the core model cannot establish exhaustively. Payload formatting and interlinear layout rendering remain outside this profile.

See the working example. The upstream data model, reference grammar and reference resolver define the adopted semantics. The fixture provenance pins the reference source and every expected item value. Six direct reader tests pass; native integration and live rendered editor verification are tracked separately in the parser ledger. This does not claim validation of arbitrary Xigt schema extensions, morphological analysis, Leipzig category inference, negative-index extensions, media retrieval, or an exact XML export round trip.