Skip to content

feat: bbox of brackets, summations, missing font-mappings etc - #351

Merged
PeterStaar-IBM merged 14 commits into
mainfrom
fix/bbox-of-brackets
Sep 22, 2026
Merged

PeterStaar-IBM merged 14 commits into
mainfrom
fix/bbox-of-brackets

Conversation

@PeterStaar-IBM

@PeterStaar-IBM PeterStaar-IBM commented Sep 18, 2026

Copy link
Copy Markdown
Member

Rewrite cell contraction and fix glyph bounding boxes

Summary

This branch started as a fix for oversized bracket bounding boxes and grew into a
rewrite of how character cells are contracted into words and lines. The previous
contractor merged characters with a set of hand-tuned width factors and a
same-font requirement, which failed on rotated text, mathematical layout, and
documents with unusual spacing. It is replaced by a geometry-driven contractor
that derives word boundaries from the PDF text cursor rather than from painted
glyph ink.

Along the way, several glyph-geometry and encoding bugs that fed bad input into
contraction are fixed.

Cell contraction

page_item_sanitator<PAGE_CELLS> is rewritten around one contraction pass that
produces both word and line views:

  • Cells are partitioned into rotation-aware line runs. Same-line adjacency
    uses the writing-axis projection and transverse overlap of facing edges, so
    the test is invariant under arbitrary rotation, including vertical and
    45-degree text.
  • Word boundaries come from PDF cursor placement, not glyph ink. Each cell
    now records its text origin, advance endpoint, writing axis, and nominal text
    height (transient fields on page_item<PAGE_CELL>, not serialized). Gaps are
    measured between cursor advance and next origin, which is immune to Type 3
    side bearings and oversized mathematical operators.
  • Lines that contain explicit PDF spaces treat them as authoritative
    boundaries. Lines without them infer a line-local threshold by normalizing
    gaps against neighboring advances and separating them with a fixed-iteration
    two-cluster k-means, clamped to conservative bounds.
  • Off-baseline components placed by cursor rewind (fraction parts, limits,
    radical contents, assembled delimiters) are recognized as local attachments
    and kept within their word, gated on both transverse distance and
    math-context heuristics so that figure labels on adjacent rows are not
    absorbed.

Consequences:

  • create_word_cells / create_line_cells are the entry points.
    sanitize_bbox remains as a thin compatibility shim.
  • The quadratic remove_duplicate_cells pass is no longer run during word and
    line creation; the new contractor does not produce the duplicates it existed
    to clean up. The method is kept for callers that still need it.
  • enforce_same_font, horizontal_cell_tolerance, and the three
    *_space_width_factor_for_merge fields are deprecated and ignored. They
    remain in decode_config and on the parse CLI so existing callers and
    saved configurations keep loading. can_reuse_sanitised_cells_for_line_cells
    no longer inspects them.
  • Unmapped glyphs are emitted as U+FFFD instead of a space when
    keep_glyphs is off, and carry a transient flag, so a decoding failure is no
    longer mistaken for a real word boundary.

Glyph geometry

  • Type 3 glyphs now derive their cell rectangle from the CharProc bounds
    (compute_type3_rect) mapped through the same four transformations the
    renderer uses, instead of from the declared advance rectangle. This fixes
    fonts that advertise a loose /FontBBox such as [-10 -10 10 10].
  • A CharProc with metrics but no supported painting operator is classified as
    blank rather than as a failed glyph; operators the lightweight Type 3
    parser cannot rasterize (S, sh, Do, Tj, …) no longer cause a painted
    glyph to be misread as whitespace.
  • Invalid vertical metrics: some producers publish both ascent and descent
    as positive. When descriptor metrics do not straddle the baseline and
    /FontBBox does, the font box is used instead. This is what collapsed tall
    brackets and summation signs into shallow cells above the baseline.
  • freetype_font_cache::get_glyph_bbox resolves exact outline bounds from the
    embedded font (1000 units/em), used when descriptor metrics are known bad. A
    resolved-but-blank glyph is reported distinctly from a resolution failure.
  • Marked-content spans in stream.h now propagate the furthest advance
    endpoint
    of the cells they replace, so composed accents and ligatures do
    not appear artificially separated from the following character.

Encoding and fonts

  • Font-specific base fonts (EncodingScheme FontSpecific) take priority over a
    falsely declared WinAnsi encoding, via the new base_font::is_font_specific.
  • /gNN glyph names on the standard Symbol face resolve through the standard
    Adobe Symbol glyph order (symbol_glyph_indices.h), then through the existing
    AGL path.
  • /G<decimal> names are accepted as Unicode code points only when every
    name in the /Differences vector matches the convention, with G32 as the
    decisive sentinel and a producer-specific fallback for legacy Distiller
    0150 subsets.
  • /C<decimal> now prefers a matching font-specific resource before falling
    back to the code-point heuristic.
  • AFM metrics added for AdvPSSym / AdvPSMP10, loaded from a new
    custom/AdvPSSym directory; caret added to the additional glyph list.
  • ~40 upstream pdf2svg font mappings are vendored under
    custom/_unloaded/pdf2svg-misc/ and are not loaded at runtime. They are
    candidates to promote individually once verified against a real PDF with
    regression coverage; see the directory README for provenance and licensing.

Tooling

  • render gains --draw-glyph-bbox, --draw-char-bbox, --draw-word-bbox
    and --print-bbox; the existing --draw-text-bbox now means line cells.
    Debug quads are drawn in the display-oriented page frame via
    draw_debug_bboxes and pdf_decoder<PAGE>::to_page_frame_point, and
    --print-bbox emits an aligned table on stdout.
  • export_dclx.py numbers a single exported page as page 1, matching the
    standalone one-page document it produces.

Tests

  • New: test_regression_cell_contraction.py, test_unit_rotated_cell_contraction.py,
    test_unit_invalid_font_metrics.py (with an inline synthetic OTF so the
    malformed-font construct stays reviewable).
  • Updated: test_unit_gid_glyph_names.py, test_unit_type3_fonts.py.
  • elsevier-00.pdf and newspaper-00.pdf added to the regression page
    selection.

Breaking changes

  • The contraction tuning parameters listed above are accepted but ignored.
    Callers relying on them to alter word or line grouping will see different
    output.
  • Word and line cell boundaries change for many documents. The regression
    dataset revision is bumped to 3826b7b716a315ae7a3561cfcbb5a26e538ccb27.

Examples

for Elsevier-page-4.pdf, we have

blend2d_renderer_preview

for the newspaper, we have

blend2d_renderer_preview

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@mergify

mergify Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @PeterStaar-IBM, all your commits are properly signed off. 🎉

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
@PeterStaar-IBM PeterStaar-IBM changed the title fix: bbox of brackets, summations etc feat: bbox of brackets, summations, missing font-mappings etc Sep 19, 2026
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants