fix: parse Pod below __END__ when __DATA__ section present - #439
Draft
toddr-bot wants to merge 2 commits into
Draft
fix: parse Pod below __END__ when __DATA__ section present#439toddr-bot wants to merge 2 commits into
toddr-bot wants to merge 2 commits into
Conversation
#16) Pod content after __END__ is not recognized when a __DATA__ section is present earlier in the file. The tokenizer's Data zone consumes all remaining lines including __END__ and subsequent Pod. Tests are marked $TODO and will pass once the fix is applied. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…(GH #16) Data.pm's __TOKENIZER__on_line_start unconditionally consumed all remaining lines without checking for __END__. This caused __END__ and any Pod after it to be swallowed into the Data token. Add __END__ detection that mirrors Word.pm's handling: finalize the Data token, emit a Separator, switch to the End zone, and let End.pm handle Pod recognition from there. Remove $TODO markers from the regression tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a
__DATA__section is present,PPI::Token::Data's tokenizer unconditionally consumed all remaining lines — including__END__and any Pod after it. This fix adds__END__detection in the Data zone so the tokenizer transitions to the End zone, allowing Pod after__END__to be properly parsed.Fixes #16
Changes
__END__line detection inPPI::Token::Data::__TOKENIZER__on_line_startthat mirrorsWord.pm's handling: finalize the Data token, emit a Separator, switch zone toPPI::Token::End__DATA__+__END__+ Pod, including the edge case where__END__immediately follows__DATA__Test plan
t/regression_data_end_pod.twith 12 assertions verifying Pod extraction, Separator count, Data token boundaries, and round-trip fidelityGenerated by Kōan /fix
Quality Report
Changes: 2 files changed, 85 insertions(+)
Code scan: clean
Tests: skipped
Branch hygiene: clean
Generated by Kōan post-mission quality pipeline