feat: bbox of brackets, summations, missing font-mappings etc - #351
Merged
Merged
Conversation
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Contributor
|
✅ DCO Check Passed Thanks @PeterStaar-IBM, all your commits are properly signed off. 🎉 |
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
cau-git
approved these changes
Sep 22, 2026
This was referenced Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrite cell contraction and fix glyph bounding boxes
Summary
This branch started as a fix for oversized bracket bounding boxes and grew into a
rewrite of how character cells are contracted into words and lines. The previous
contractor merged characters with a set of hand-tuned width factors and a
same-font requirement, which failed on rotated text, mathematical layout, and
documents with unusual spacing. It is replaced by a geometry-driven contractor
that derives word boundaries from the PDF text cursor rather than from painted
glyph ink.
Along the way, several glyph-geometry and encoding bugs that fed bad input into
contraction are fixed.
Cell contraction
page_item_sanitator<PAGE_CELLS>is rewritten around one contraction pass thatproduces both word and line views:
uses the writing-axis projection and transverse overlap of facing edges, so
the test is invariant under arbitrary rotation, including vertical and
45-degree text.
now records its text origin, advance endpoint, writing axis, and nominal text
height (transient fields on
page_item<PAGE_CELL>, not serialized). Gaps aremeasured between cursor advance and next origin, which is immune to Type 3
side bearings and oversized mathematical operators.
boundaries. Lines without them infer a line-local threshold by normalizing
gaps against neighboring advances and separating them with a fixed-iteration
two-cluster k-means, clamped to conservative bounds.
radical contents, assembled delimiters) are recognized as local attachments
and kept within their word, gated on both transverse distance and
math-context heuristics so that figure labels on adjacent rows are not
absorbed.
Consequences:
create_word_cells/create_line_cellsare the entry points.sanitize_bboxremains as a thin compatibility shim.remove_duplicate_cellspass is no longer run during word andline creation; the new contractor does not produce the duplicates it existed
to clean up. The method is kept for callers that still need it.
enforce_same_font,horizontal_cell_tolerance, and the three*_space_width_factor_for_mergefields are deprecated and ignored. Theyremain in
decode_configand on theparseCLI so existing callers andsaved configurations keep loading.
can_reuse_sanitised_cells_for_line_cellsno longer inspects them.
U+FFFDinstead of a space whenkeep_glyphsis off, and carry a transient flag, so a decoding failure is nolonger mistaken for a real word boundary.
Glyph geometry
(
compute_type3_rect) mapped through the same four transformations therenderer uses, instead of from the declared advance rectangle. This fixes
fonts that advertise a loose
/FontBBoxsuch as[-10 -10 10 10].blank rather than as a failed glyph; operators the lightweight Type 3
parser cannot rasterize (
S,sh,Do,Tj, …) no longer cause a paintedglyph to be misread as whitespace.
as positive. When descriptor metrics do not straddle the baseline and
/FontBBoxdoes, the font box is used instead. This is what collapsed tallbrackets and summation signs into shallow cells above the baseline.
freetype_font_cache::get_glyph_bboxresolves exact outline bounds from theembedded font (1000 units/em), used when descriptor metrics are known bad. A
resolved-but-blank glyph is reported distinctly from a resolution failure.
stream.hnow propagate the furthest advanceendpoint of the cells they replace, so composed accents and ligatures do
not appear artificially separated from the following character.
Encoding and fonts
EncodingScheme FontSpecific) take priority over afalsely declared WinAnsi encoding, via the new
base_font::is_font_specific./gNNglyph names on the standard Symbol face resolve through the standardAdobe Symbol glyph order (
symbol_glyph_indices.h), then through the existingAGL path.
/G<decimal>names are accepted as Unicode code points only when everyname in the
/Differencesvector matches the convention, withG32as thedecisive sentinel and a producer-specific fallback for legacy Distiller
0150subsets./C<decimal>now prefers a matching font-specific resource before fallingback to the code-point heuristic.
AdvPSSym/AdvPSMP10, loaded from a newcustom/AdvPSSymdirectory;caretadded to the additional glyph list.custom/_unloaded/pdf2svg-misc/and are not loaded at runtime. They arecandidates to promote individually once verified against a real PDF with
regression coverage; see the directory README for provenance and licensing.
Tooling
rendergains--draw-glyph-bbox,--draw-char-bbox,--draw-word-bboxand
--print-bbox; the existing--draw-text-bboxnow means line cells.Debug quads are drawn in the display-oriented page frame via
draw_debug_bboxesandpdf_decoder<PAGE>::to_page_frame_point, and--print-bboxemits an aligned table on stdout.export_dclx.pynumbers a single exported page as page 1, matching thestandalone one-page document it produces.
Tests
test_regression_cell_contraction.py,test_unit_rotated_cell_contraction.py,test_unit_invalid_font_metrics.py(with an inline synthetic OTF so themalformed-font construct stays reviewable).
test_unit_gid_glyph_names.py,test_unit_type3_fonts.py.elsevier-00.pdfandnewspaper-00.pdfadded to the regression pageselection.
Breaking changes
Callers relying on them to alter word or line grouping will see different
output.
dataset revision is bumped to
3826b7b716a315ae7a3561cfcbb5a26e538ccb27.Examples
for Elsevier-page-4.pdf, we have
for the newspaper, we have