This repository stores analyses of abstract shapes for CJKV Unified Ideographs.
The purpose of analyzing abstract shapes is to establish the graphic specification of each character. This is accomplished by analyzing the classes to which the components it contains belong and describing the structure formed by the components.
The graphic specification of components has two sources. The first source is diachronic graphic evolution, which is supported by the systematic changes in Han ideographs. It can be noted that many changes at the writing system level still keep the internal structure of the characters. The second source is language, i.e., the characters containing the component are in the same harmonic scope.
We use prefix expressions composed of components to describe the abstract shape of a character. Each component is either shaped like [A] or shaped like [A(B)]. The operator is IDC, but define them more abstractly, for example, a character containing three identical components, regardless of how the three components are arranged, we always use ⿲.
The visualization of this repository is implemented in another repository, and here is the link.
How to run it — as a developer or as an editor — is in Contribute at the end of this document.
A local editing UI for the abstract-shape corpus. Everything lives in one
ndjson file, so the whole corpus is a single file and git diff stays readable.
./start.bat # picks a free port and opens the server
start.bat first kills any previous instance still listening on
ports 8000–8100, then reuses the lowest free port.
Open 写作. Three columns, all three widths draggable (double-click a divider to reset):
| Column | Contents |
|---|---|
| left | the full character list (97712), grouped by Unicode block, paginated 100/page; each row shows how many notes that character has, and the block chips carry a red count of 核验 problems (基本 3) |
| middle | the note editor: 抽构 / 参考抽构 / 甲骨文 / 小篆 / 札记, with an 编辑·预览 toggle |
| right | 文献 (BibTeX-style reference editor), 资料 (抽构表 / 广韵 / IES / 词表, read-only) and 核验 (what is wrong with the current character) |
Editing a note saves itself: typing pauses 0.9 s, picking a glyph 0.3 s, and any
switch (another note, another character, another tab) flushes what is still
pending, so there is no 保存 button. A note whose 抽构 does not parse is not
written — the editor says 抽构格式错误,未保存 until it is fixed.
Keyboard: Ctrl+S save now (rarely needed), Ctrl+E toggle edit/preview, Ctrl+↑ Ctrl+↓ previous
/ next character, Ctrl+G jump to a character.
The ▶ 未写 button jumps to the first character in the current filter that has
no notes yet.
The 甲骨文 and 金文 views list the glyphs themselves (backend/data/ob.jsonl,
bi.jsonl). Under each glyph, the 汉字字形 it stands for are shown as chips:
+opens a one-line input — type the character and press Enter; spaces,,、、or;split several characters into several chips at once;Esccancels;×deletes that chip;- the toolbar search matches 编号 and the annotated characters too.
Each chip is one entry of annotations (con), so existing records keep working:
{"num": "0002", "glyph": "",
"annotations": [{"con": "匕(妣)", "ref": "", "comm": ""}, {"con": "匕(比)", "ref": "", "comm": ""}]}num is unique in both files, so a record is located by num; the endpoints are
POST /api/ob/annotate, /api/ob/annotate/delete and the same pair for
/api/bi/…. 甲骨文 numbers are the corpus's running numbers; 金文 numbers follow
the source book — see below.
资料 → 抽构表 shows the character's row of
input/abstract_shape.xlsx — every column, in
sheet order: char, ids, cog, unify, auto, his, his (2),
char (2), con, recon, comm, gy (the ExtCI / ExtGH sheets have 9),
with — for an empty cell. One character may own several rows — the
continuation rows leave the first char blank — and each gets its own
「续行 n」 block. Below them 「推导」 carries what the sheet does not: 规范形式
(the con parsed and normalised), 同构字 and 上级抽构, all computed from
input/abstract_*.txt (backend/sheettable.py).
The first read of a sheet takes a few seconds (openpyxl parses the 7.5 MB workbook); the result is cached per sheet and only rebuilt when the file's mtime changes.
抽构 / 参考抽构 are formatted as you type:
- full-width
()[]=become half-width, spaces are dropped; - once the expression parses, the canonical form is written back with every
component in brackets —
⿰木木→⿰[木][木],⿱九乙(ᄅ)→⿱[九][乙(ᄅ)],=下→=[下](the= ~ * ?prefix is kept); - the parser accepts the tolerant spellings (
⿱九乙,[⿱九乙],[乙](ᄅ)), so the formatted text always parses again; - the caret keeps its place measured in "meaningful" characters, and IME composition is never touched mid-word.
Characters that cannot be a component (, . : ; ! ASCII punctuation, digits,
full-width punctuation) raise 非法字符 instead of being silently accepted.
backend/verify.py checks every note and the right column's 核验 tab lists what
it finds for the current character; the left block chips show the whole-corpus
count per Unicode block, and the toolbar shows the current character's count.
Fields with problems are outlined in red.
| Code | Level | Rule |
|---|---|---|
bad-char |
error | a component is punctuation / a digit |
arity |
error | an IDC operator has the wrong number of operands |
bracket / extra |
error | brackets do not pair, or there is trailing junk |
missing-unit |
warning | a component [下] has no record of its own whose 抽构 is [下] |
dup-unit |
error | two or more records use the same shape as their 抽构 — [下], ⿰[虫][下], [與]=⿶[舁][牙] (head included), anything; only one may own it, the rest must be =… |
no-ref |
error | a cited reference id does not exist |
page-range |
error | a cited page falls outside the reference's own pages |
Reference pages use the same notation as the inline citations: 224-225+226-227
(- = range, + = separate spans). The 文献 form converts — — 、 etc. on
input and the backend re-normalizes on save, so both sides always compare
apples to apples; a citation is accepted only if each of its spans fits inside
one registered span.
Reference keys are checked globally too: the same key written twice in
references.ndjson, or a custom record re-using a built-in key, shows up as
「引用键重复」 in the 文献 tab and at the bottom of 核验. Citing the same
reference several times inside one note is fine.
One note is one line of
abstract-shape/abstract-shape.ndjson,
keyed by (character, seq):
{"char": "丂", "seq": 1, "con": "⿱一丂", "ref_con": "*考",
"notes": "…… [ref:C001.244-245+248]"}| Field | Meaning |
|---|---|
char |
the character |
seq |
note number within that character, from 1; referenced as 丂-1 |
con |
the abstract shape — an IDS expression, validated by backend/shape.py |
ref_con |
a comparative shape; =字 / *字 are annotation conventions and skip IDS validation |
ob |
oracle-bone forms attached to this note — nums from backend/data/ob.jsonl, e.g. ["0200"] |
bi |
bronze-inscription forms — nums from backend/data/bi.jsonl, e.g. ["01001", "01001A"] |
xz |
small-seal forms attached to this note — nums from backend/data/seal.ndjson, e.g. ["01998"] |
notes |
the note body, free Markdown; citations are written inline (see below) |
ob / bi / xz are picked in the editor (甲骨文 / 金文 / 小篆 rows, multi-select;
candidates come from /api/glyphs/ob, /api/glyphs/bi and /api/glyphs/seal).
A seal form that the table maps for the character is shown with a 默认 badge but is
never written on its own — one click adopts it (and the auto-save then stores it),
so clearing the row really clears it.
The preview also lists the forms carried by the notes whose 抽构 points at this
character (=[𠀠], [X]=…): this note's own forms come first, then each borrowed
one with its source in an inline code span (- 金文: > |丌| > |其|). The
same rule covers all three kinds, and keeping the marker in code means the |
characters can never be read as table syntax.
num is the key. A number in ob.jsonl / bi.jsonl covers several glyphs, so
backend/scripts/renumber_glyphs.py gave every repeat a letter suffix in file order
(0001, 0001A, 0001B, …); seal.ndjson is 1–11108 and never collides. All
three tables are keyed by a unique num.
The glyphs live in the private use area (fonts ZihuiSongJGW.ttf,
ZihuiSongJW.otf, frontend/sw-ccz-1963-rev1.ttf). Only the seal codepoint can
be computed — 0xF0000 + num - 1 — because its num is still the font index;
the oracle-bone / bronze ids cannot, so the frontend fetches the whole
num → glyph table once from /api/glyphs/index. seal.ndjson carries
{num, glyph, char} (11108 forms, char = the table's zi_strict when it has
one, otherwise zi). In Markdown the forms are written [ob:0001,0200],
[bi:01001A] and [seal:01998].
The bronze corpus is numbered per volume, each volume restarting at 001, so a
number is [卷][三位卷内号] written as five digits: 01001 (卷1 号1) … 01193,
02001 (卷2 号1) … 02452, … 14350 (卷14).
bi.jsonl was stored with a flat 1–4419 running number, which lost the volume
breaks. backend/data/bi_volumes.json records how many numbers each volume has
and which numbers the book skips:
{"volumes": [193, 452, 466, 265, 325, 371, 442, 283, 243, 303, 192, 338, 197, 351],
"holes": ["07086"]}and backend/scripts/number_bi_volumes.py walks the file in order, assigning
卷 × 1000 + 卷内号 and skipping the holes (07085 is followed by 07087). When
several glyphs share one number they keep an A/B/… suffix (01001, 01001A,
01001B). The table then yields 4420 numbers while the file holds 4419, so 卷14
ends at 14350 — one number of the book has no bronze forms in this corpus.
Citations are part of the note body — there is no separate reference list per note:
| Syntax | Meaning |
|---|---|
[ref:C001] |
the reference, no particular page |
[ref:C001.244-245+248] |
C001, pp. 244–245 and 248 |
[ref:C001.12+15-16] |
C001, p. 12 and pp. 15–16 |
- separates the ends of a range, + joins separate spans; , 、 ; ~ —
are normalized to those two when the note is saved. Page numbers here are
the pages cited by this note, distinct from the reference's own page range
(refs.pages): a book may span 100–300 while a particular note cites only
120–135. In the preview each citation becomes a chip that opens the reference
in the 文献 panel.
The API still returns refs on every note
([{"id": "C001", "pages": "244-245+248"}]), but it is parsed from notes on
save and load — the file has no refs field unless an older file already did.
A shape is either a single component or an IDC operator followed by its operands:
<shape> ::= <expr> ( '=' <expr> )*
<expr> ::= <char> | <op> <expr> <expr> | <op> <expr> <expr> <expr>
<char> ::= 汉字 | 汉字(变体)
The optional [與]= head spells out which character is being decomposed, so
[與]=⿶[舁][牙] reads「與 分解为 ⿶舁牙」. The head is not a component: it does
not count as a leaf, it is kept (in brackets) by the formatter, and 核验 treats
[與]=… and … as two different definitions — the same head + body written
twice is still a dup-unit.
= may be chained to spell out several equivalent forms of the same shape:
[氶]=⿴[𠬞][卩]=[丞]. Every segment is parsed on its own (the first one is the
head when it is a single component), normalized keeps the = between them,
and the counts (leaves, leaf_count, operators) cover all segments — but
the head still is not counted.
17 operators are supported (backend/shape.py → IDC.ARITY):
⿰ 左右 |
⿱ 上下 |
⿲ 左中右 |
⿳ 上中下 |
⿸ 左上包围 |
⿺ 左下包围 |
⿹ 右上包围 |
右下包围 |
⿵ 上三包围 |
⿷ 左三包围 |
⿶ 下三包围 |
右三包围 |
⿴ 全包围 |
⿻ 重叠 |
镜像 |
旋转 |
〾 近似形 |
The editor validates as you type and reports the exact position of a mistake:
⿰日 ✕ 「⿰」(左右)需要 2 个部件,只给了 1 个 第 3 字符
⿱一去 ✓ 2 个部件,深 1 ⿱[一][去]
⿰木(木)木 ✓ 2 个部件,深 1 ⚠ 重复部件 木,确认是否应该用 ⿰/⿱ 之外的运算符
backend/refs.py keeps BibTeX-shaped records (18 fields: author, editor, title,
journal, booktitle, publisher, address, year, date, volume, number, pages,
edition, url, doi, note). Two sources, merged by id:
backend/data/papers.json— 94 upstream records, read-only in the UI; saving one writes an override instead of touching the upstream file (editing the file by hand also takes effect — the cache watches its mtime too);abstract-shape/references.ndjson— locally added / edited records (R001…).
The citation line is built from the fields, and date wins over year. date
takes the month/day shorthand, so 3 = 3月 and 3.12 = 3月12日 with the year
coming from year; a full 2007-05-01 (or 2007/5/1, 2007.5.1,
2007年5月1日) carries its own year. Everything is stored as 2007-05-01,
2007-05, 03-12 or 03, and printed as 2007年5月1日 / 2007年3月12日; with
no date at all the citation keeps the plain 2007年.
A work named inside a title is stored with the same 《》 as the title itself —
article_title is 甲骨金文的“夙”與清華簡《五紀》的“丮”, never 〈五紀〉. The
demotion happens only when the title is displayed: the renderer adds the outer
《》 and turns the marks already inside into 〈〉, so the citation reads
《甲骨金文的“夙”與清華簡〈五紀〉的“丮”》 (refs.display_title, and citeTitle
in frontend/writer.js). A citation string that already carries such nesting is
fixed the same way when it is served — refs.demote_nested keeps the outermost
《》 and turns every deeper one into 〈〉, so 《《說文解字》字音注釋研究》 shows as
《〈說文解字〉字音注釋研究》 (upstream raw_title / citation stay as stored).
The list follows the conventional block order, defined once in
backend/blocks.py and served via /api/blocks:
基本 → 兼容 → 部首 → 扩展A → 扩展B → 兼容扩展 → 扩展C → 扩展D → 扩展E → 扩展F → 扩展G → 扩展H → 扩展I → 扩展J → 其他
扩展I sits at the end even though its code points precede 扩展G/H — letter order wins over code-point order. 扩展J (U+323B0–U+3347F, Unicode 17.0) follows 扩展H.
| Syntax | Meaning |
|---|---|
[[丂-1]] |
cross-reference to note 丂-1 (clickable) |
[[文献: L005]] |
reference link rendered as [L005] |
[ob:0001,0200] |
oracle-bone glyphs (numbers from ob.jsonl) |
[bi:01001A] |
bronze-inscription glyphs (num from bi.jsonl) |
[seal:01998] |
small-seal glyphs (num from seal.ndjson) |
The older [[卜:…]] / [[金:…]] / [[篆:…]] spellings still render.
backend/scripts/migrate_to_entries.py converts the earlier single-record
format ({char, con, analysis, references}) into the note model above. It backs
the file up before writing.
Two very different jobs, touching different files: 开发 changes the tool, 编辑 changes the corpus through the tool.
- Clone, then
pip install -r requirements.txt— fastapi + uvicorn for the server, openpyxl + pandas for the exporters underbackend/scripts/. ./start.batexports the derived data if it is missing, frees a port in 8000–8100, runsuvicorn backend.main:app --reloadand opens the browser.--reloadcoversbackend/; the frontend is plain static files.- After editing anything under
frontend/, bump its?v=number infrontend/index.html(writer.js?v=56→57), or the browser keeps serving the old copy.style.css?v=works the same way.
One process serves both sides — FastAPI answers /api/* and hands out
frontend/ — so there is no separate frontend dev server.
| Path | What it owns |
|---|---|
backend/main.py |
the FastAPI app: routes and the static mount |
backend/entrystore.py |
the note store, reading / writing abstract-shape/abstract-shape.ndjson |
backend/refs.py |
references (papers.json + references.ndjson), citation text, 书名号 levels |
backend/glyphs.py |
甲骨文 / 金文 / 小篆 tables and their 汉字 annotations |
backend/shape.py |
the IDS grammar: parser, [與]= head, = chains, normalized |
backend/verify.py |
the 核验 rules (table above) |
backend/blocks.py |
the Unicode block order of the left column |
backend/sheettable.py |
抽构表 / 象形 source data (reads input/abstract_*.txt, input/xiangxing.txt) |
frontend/index.html |
static shell and the ?v= cache numbers |
frontend/writer.js |
写作 view: list, editor, auto-save, preview, glyphs, 核验 |
frontend/refs.js |
the 文献 panel editor |
frontend/app.js |
the other views (广义汉字 / 甲骨文 / 金文 / 参考文献 / 未编码字) |
frontend/markdown.js |
the Markdown renderer ([[丂-1]], [ref:…], [ob:…]) |
frontend/format.js |
shape input formatting — full-width → half-width, normalized write-back |
frontend/typography.js |
Han/Latin spacing and punctuation squeezing |
frontend/images.js |
paste / drop an image → Markdown, upload to /api/images, {w=…} width |
There is no test runner yet, so a change is checked the way the UI checks it
(8001 = whatever port start.bat printed):
curl -s -X POST http://127.0.0.1:8001/api/shape/validate \
-H "Content-Type: application/json" \
-d '{"expr":"[氶]=⿴[𠬞][卩]=[丞]"}'
curl -s http://127.0.0.1:8001/api/verifyCaches differ by file: notes, references and the glyph tables reload when their
mtime changes, but backend/data/papers.json is read once at start-up — restart
the server after editing it.
No code, no virtualenv, no build step.
- Run
./start.bat(double-click on Windows). It prints the URL, opens it, and picks the first free port in 8000–8100. On macOS / Linux runpython -m uvicorn backend.main:app --port 8001instead. - Work in 写作: pick a character on the left (
▶ 未写jumps to the first one without notes), then fill 抽构 / 参考抽构 / 甲骨文 / 金文 / 小篆 / 札记. Every change saves itself — there is no 保存 button, and switching note, character or tab flushes what is still pending. - The right column: 核验 says what is inconsistent for this character (bad 抽构, a component without a note of its own, a citation page out of range), 文献 edits references, 资料 is the read-only 抽构表 / 广韵 / IES / 词表.
- 甲骨文 / 金文 annotate a glyph with the 汉字 it stands for; 未编码字 keeps characters that have no code point yet.
- What you commit is only what you edited:
abstract-shape/abstract-shape.ndjson(one line per note),abstract-shape/references.ndjson, andabstract-shape/images/*.pngfor pasted pictures.git diffshould show just those lines —backend/data/*is upstream data and normally stays untouched.
The original build still works; the web editor does not use it.
-
Updating
- Clone this repo as the
RepoA. - Edit the shape sheet,
RepoA > input/abstract_main.txt(and the otherinput/abstract_*.txtvolumes). - Install the required packages in the
RepoA > requirements.txt. - Run Python file
RepoA > src/build_txt.pyto update all the text files. - Pull your request.
- Clone this repo as the
-
Previewing
- Clone
Kushim-Jiang > kushim-jiang.github.ioin the same root as theRepoB. - Run Python file
RepoA > src/build_json.pyto update the JSON file inRepoB > assets > abstract.json. - Run
jekyll sin theRepoBto preview your changes.
- Clone