feat: add attachment serializer support and exclusion filtering

[yonikremer]

Summary

This PR adds full attachment serialization support across Doctags, Markdown, and HTML formatters, and integrates AttachmentItem into the DoclingDocument model traversal and index structure.

Closes #712

---

Key Changes & Design Decisions

1. Document Model & Index Integration (DoclingDocument, _DocIndex)

• Traversal & Indexing: Added attachments to DoclingDocument._DocIndex, _iterate_items_with_stack, reference validation, and reference normalization (_normalize_references).
• Serialization Cleanup: Configured _custom_pydantic_serialize to omit the attachments key when empty ([]), maintaining clean JSON output.
• Reference Resolution: Implemented AttachmentItem handling in _append_item generating proper JSON pointers (#/attachments/N).

2. Document Token Mapping (DocumentToken)

• Added ATTACHMENT = "attachment" to DocumentToken and mapped DocItemLabel.ATTACHMENT in create_token_name_from_doc_item_label.
• Exposes <attachment> special token during document token generation.

3. Serializers & Exclusion Filtering (doctags, markdown, html, common)

• Doctags Serializer: Introduced DocTagsAttachmentSerializer rendering <attachment>...</attachment> tags with optional location tokens and conversion status details.
• Exclusion Filtering: Added checks against doc_serializer.get_excluded_refs() in MarkdownAttachmentSerializer, HTMLAttachmentSerializer, and MarkdownDocSerializer to respect label/layer filters.
• Unpositioned Attachments: Filtered out excluded items when rendering the unpositioned ## Attachments section in Markdown serializer output.

---

Testing

• Expanded test/test_attachment_serialization.py with unit tests:
  • Reference normalization (test_attachment_normalize_references)
  • Item appending (test_add_item_attachment)
  • Doctags serialization (test_attachment_export_to_doctags)
  • Label exclusion filtering (test_attachment_exclusion_filtering)

[github-actions]

❌ DCO Check Failed

Hi @yonikremer, your pull request has failed the Developer Certificate of Origin (DCO) check.

This repository supports remediation commits, so you can fix this without rewriting history — but you must follow the required message format.

---

🛠 Quick Fix: Add a remediation commit

Run this command:

git commit --allow-empty -s -m "DCO Remediation Commit for yoni kremer <yoni.kremer@gmail.com>

I, yoni kremer <yoni.kremer@gmail.com>, hereby add my Signed-off-by to this commit: 6378afcb3f9030b56d13f8dbf436fcd81b7f5541
I, yoni kremer <yoni.kremer@gmail.com>, hereby add my Signed-off-by to this commit: b9a54f8a44ca0f5989a5c1d1349092572fa76d3e
I, yoni kremer <yoni.kremer@gmail.com>, hereby add my Signed-off-by to this commit: 4e2bc875cb9da1367e73a3da4191d1d74f596f25
I, yoni kremer <yoni.kremer@gmail.com>, hereby add my Signed-off-by to this commit: afe742577c105ca3199a48de3a285dd8c4955c8f
I, yoni kremer <yoni.kremer@gmail.com>, hereby add my Signed-off-by to this commit: 1f3600570d6e48166b52c902adbf2919c821a1e6"
git push

---

🔧 Advanced: Sign off each commit directly

For the latest commit:

git commit --amend --signoff
git push --force-with-lease

For multiple commits:

git rebase --signoff origin/main
git push --force-with-lease

More info: DCO check report

[mergify]

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

	Protection	Waiting on
🔴	Require two reviewer for test updates	👀 reviews
🟢	Enforce conventional commit	—

🔴 Require two reviewer for test updates

Waiting for

• #approved-reviews-by >= 2

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

---

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[yonikremer]

This PR blocks docling-project/docling-parse#313 and another PR I am writing to docling's main repo. @PeterStaar-IBM can you/another maintainer take a look at it?

[PeterStaar-IBM]

lgtm!

[PeterStaar-IBM]

@yonikremer looks good in general, but you need to fix the CI

[yonikremer]

@PeterStaar-IBM I fixed it, It now passes CI locally

[PeterStaar-IBM]

lgtm!

[codecov]

Codecov Report

❌ Patch coverage is 88.23529% with 14 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling_core/transforms/serializer/html.py	36.36%	7 Missing ⚠️
docling_core/transforms/serializer/doctags.py	80.95%	4 Missing ⚠️
docling_core/transforms/serializer/common.py	80.00%	1 Missing ⚠️
docling_core/transforms/serializer/markdown.py	95.65%	1 Missing ⚠️
docling_core/types/doc/document.py	96.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[PeterStaar-IBM]

@yonikremer Looking at your PR, we discussed with the team internally, and I think the following course of action would be preferred (since there are also other PR's coming into Docling regarding email attachments):

1. make a small faeture PR that only includes FileAttachmentAnnotation, PdfAttachment and that makes those part of the SegmentedPage (similar to what you currently do in DoclingDocument). Since you only use these classes in the PR in docling-parse, we can therefore progress there already.

• small note: please allow for the binary data in PdfAttachment (you have already mimetype and size)

2. make an additional feature PR which will introduce the AttachmentItem (again with binary data in addition to str and Uri) and its storage in the DoclingDocument. I would not yet touch the serializers at this moment
3. We need to look into Docling how we are going to support the recursive parsing (if needed). Currently, we are in favor to store, in addition to the binary data, a serialized DCLG or DCLX as binary. If it is present, we dont need the AttachmentStatus

cc: @dolfim-ibm @vagenas @cau-git