What happens
Since 7.21.0, some word boundaries that 7.20.0 recovered are lost, most visibly in bold section headings of LaTeX papers. Using 2305.03393v1-pg9.pdf from docling's test data (file):
import sys
from docling_core.types.doc.page import TextCellUnit
from docling_parse.pdf_parser import DoclingPdfParser
doc = DoclingPdfParser(loglevel="fatal").load(path_or_stream=sys.argv[1])
for page_no, page in doc.iterate_pages():
for c in page.iterate_cells(unit_type=TextCellUnit.LINE):
if "Optimization" in c.text:
print(repr(c.text))
| docling-parse |
output |
| 7.20.0 |
'Hyper Parameter Optimization' |
| 7.21.0 |
'5.1 HyperParameterOptimization' |
The same thing shows up across docling's test PDFs (line cells, 7.20.0 -> 7.21.0):
2305.03393v1.pdf: Related Work -> 2 RelatedWork, Problem Statement -> 3 ProblemStatement, Fig. 1. -> Fig.1. Comparison between ..., Fig. 3. -> Fig.3. ..., Language Definition -> LanguageDefinition
2305.03393v1-pg9.pdf: Quantitative Results -> QuantitativeResults
2203.01017v2.pdf: Implementation Details -> ImplementationDetails, custom OCR -> customOCR, problem where -> problemwhere
redp5110_sampled.pdf: dotted TOC leaders get attached to the preceding word (Roles., duties.)
In docling this makes tests/test_interfaces.py::test_convert_path, ::test_convert_stream and tests/test_heading_hierarchy_pdf.py::test_pdf_pipeline_assigns_heading_levels_from_existing_fixture fail against their ground truth. docling's CI doesn't see it because its uv.lock still pins 7.20.0, but pip install docling resolves docling-parse>=7.20.0,<8.0.0 to 7.21.0.
Bisect
Built from source on macOS arm64 (Python 3.12):
So the change comes from the cell-contraction rewrite in #351. Most of the affected lines are short, and the lost gaps are the ones between words of a single bold run. That fits the new line-local threshold (two-cluster k-means over normalized gaps) putting real word gaps in the "intra-word" cluster when a line has few gaps and no explicit space glyphs. That's a guess from reading the PR description, though; I haven't traced it in the code.
Environment
- docling-parse 7.21.0 (PyPI wheel) and a source build of
e64d838
- macOS 26 arm64, Python 3.12
What happens
Since 7.21.0, some word boundaries that 7.20.0 recovered are lost, most visibly in bold section headings of LaTeX papers. Using
2305.03393v1-pg9.pdffrom docling's test data (file):'Hyper Parameter Optimization''5.1 HyperParameterOptimization'The same thing shows up across docling's test PDFs (line cells, 7.20.0 -> 7.21.0):
2305.03393v1.pdf:Related Work->2 RelatedWork,Problem Statement->3 ProblemStatement,Fig. 1.->Fig.1. Comparison between ...,Fig. 3.->Fig.3. ...,Language Definition->LanguageDefinition2305.03393v1-pg9.pdf:Quantitative Results->QuantitativeResults2203.01017v2.pdf:Implementation Details->ImplementationDetails,custom OCR->customOCR,problem where->problemwhereredp5110_sampled.pdf: dotted TOC leaders get attached to the preceding word (Roles.,duties.)In docling this makes
tests/test_interfaces.py::test_convert_path,::test_convert_streamandtests/test_heading_hierarchy_pdf.py::test_pdf_pipeline_assigns_heading_levels_from_existing_fixturefail against their ground truth. docling's CI doesn't see it because itsuv.lockstill pins 7.20.0, butpip install doclingresolvesdocling-parse>=7.20.0,<8.0.0to 7.21.0.Bisect
Built from source on macOS arm64 (Python 3.12):
669616c(fix: fall back to real metrics when a visible glyph declares zero width #328):'Hyper Parameter Optimization'(correct)e64d838(feat: bbox of brackets, summations, missing font-mappings etc #351):'5.1 HyperParameterOptimization'So the change comes from the cell-contraction rewrite in #351. Most of the affected lines are short, and the lost gaps are the ones between words of a single bold run. That fits the new line-local threshold (two-cluster k-means over normalized gaps) putting real word gaps in the "intra-word" cluster when a line has few gaps and no explicit space glyphs. That's a guess from reading the PR description, though; I haven't traced it in the code.
Environment
e64d838