Repository navigation
Attach exported highlights to their original Reader documents - #41
Merged
Merged
Conversation
Endle
force-pushed
the
fix/reader-request-safety
branch
from
September 28, 2026 03:00
bbc2a8c to
6917f01
Compare
Endle
force-pushed
the
feat/reader-highlights-split
branch
from
October 1, 2026 02:30
ea03443 to
0e0f594
Compare
Endle
force-pushed
the
feat/reader-highlights-split
branch
2 times, most recently
from
October 3, 2026 20:13
0c229ff to
64d9aa8
Compare
Endle
force-pushed
the
fix/stop-sync-on-export-failure
branch
from
October 3, 2026 20:27
43a4276 to
be55205
Compare
Endle
force-pushed
the
feat/reader-highlights-split
branch
from
October 3, 2026 20:29
64d9aa8 to
3b8cf6a
Compare
Highlights in downloaded Reader articles are sent to Reader's POST /api/v3/save/ with the article's parent_id, so they appear on the original document. Other books keep the v2 highlights export. Successful Reader exports are recorded per document and passage and saved after each request. Repeated syncs skip unchanged passages, and note edits or removal update the existing highlight via PATCH. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reader answers 400 when a passage does not match its copy of the article, and may answer 404 for a deleted article. Neither succeeds on retry, and with sync now keeping local files after a failed export, one such passage would hold back cleanup on every sync. Those passages are now sent through the v2 highlights API and recorded, so Reader is not asked again; later note edits are resent through v2. Export continues past a rejected passage, rejected passages are sent even if a later one fails, and highlights without text are skipped instead of failing the book. The transport returns Reader's error body so a rejection is logged with the passage and Reader's explanation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Endle
force-pushed
the
feat/reader-highlights-split
branch
from
October 3, 2026 22:51
3b8cf6a to
c3eaedb
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Route rejected passages back through exportBook as a one-chapter book without a document ID, instead of moving the v2 payload code into a new function. The existing v2 code keeps its original form. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reader answers 400 "This field may not be blank" when notes is an empty string, so every highlight without a note was rejected and sent through the highlights API instead. Found in a KOReader log on Linux. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fallback used to reach the highlights API by calling exportBook again with a metadata table that left out document_id. Passing the full metadata through would have sent the rejected passages back to Reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reader trims notes before validating them, so a note of " " or "\n" got the same 400 as an empty one and the passage was permanently sent to the highlights API instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every 400 from /save/ was recorded as a passage Reader could not match, so a validation error on a field the plugin chooses (like the blank-notes bug) sent the passage to the highlights API permanently. A 400 that names only notes or saved_using is now an export error that is retried on the next sync. Anything else, including unrecognized bodies, still falls back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A note edit on a highlight deleted in Reader got 404 on every sync. That failed the book, which blocked local cleanup and the export of its later passages. The highlight is now recorded as deleted and left alone, rather than recreated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reader history is keyed by document and passage text, so two highlights of the same text shared one record. The second highlight's note overwrote the first, and every later sync PATCHed the note back and forth. Repeats now become one highlight whose note joins the distinct non-blank notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every created or updated highlight rewrote and flushed the whole settings file, which holds the full text of every exported passage. On a large first export that is quadratic I/O on the e-reader's flash. History is now saved once at the end of each book, including when the book fails. The trade-off: a crash in the middle of a book can resend that book's passages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
requestReader logged the whole error body at err level, which can be a multi-KB HTML page from a proxy. It now logs the first 500 characters and still returns the full body. Rejected passage text moves from warn to dbg, because users attach these logs to public issues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Endle
marked this pull request as draft
October 4, 2026 03:03
Notes edited after export were sent with PATCH /update/<id>/, which has not been tried on a real account, and any failure there blocked local cleanup on every sync. A passage that is already recorded is now skipped whatever its note, including passages sent through the highlights API. Records still keep the note that was sent, as a baseline for a future note sync. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The highlights API path logged only the HTTP status line, so a failure when sending a rejected passage there would reach a bug report without the server's explanation. It now logs the first 500 characters of the body, as requestReader does. A passage Reader rejects is never sent to Reader again, so its warning now names the page, which identifies the highlight without logging its text. The highlights API failure test had been exercising the Reader path since Reader articles started going to /save/; it now uses a non-Reader book. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plugin has no record of highlights exported by earlier versions, so the first sync after upgrading sends those in articles still on the device to Reader too. Readwise then holds both the old unlinked copy and the one attached in Reader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Endle
marked this pull request as ready for review
October 12, 2026 19:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Last of the PRs that replace #37. #38 (transport refactor), #39 (201 / POST-retry fixes) and #42 (skip local cleanup when export fails) are merged.
Highlights exported from downloaded Reader articles currently become separate entries through the Readwise v2 API. This change sends them to Reader's
POST /api/v3/save/with the original document'sparent_idand selected passage, so they attach to that document. Other books keep the existing v2 export.Export history: Reader highlight IDs and the notes sent with them are saved in the
reader_highlightssetting once per book, including when the book fails. Each passage is exported once: later syncs skip it, and notes added or edited after export are not sent. Repeated highlights of the same passage become one highlight carrying all their notes.Rejected passages: when Reader answers
400(e.g. the passage is not found in its copy of the article) or404(article deleted), the passage is sent through the v2 highlights API instead and recorded, so Reader is not asked again. The exception is a400that namesnotesorsaved_usingbut notcontentorparent_id: that is a bug in the request, so it is an export error retried on the next sync instead of silently moving passages to v2. Export continues past a rejected passage, and rejected passages are sent even if a later passage fails. Error bodies from both APIs are logged (first 500 characters). The warning for a rejected passage names its page; the passage text is logged only at debug level, since users attach logs to public issues.Highlights without text (e.g. image selections) are skipped and logged instead of failing the book.
Blank notes: a KOReader log on Linux showed Reader's reason for rejecting highlights:
400 {"notes": ["This field may not be blank."]}. The plugin sentnotes: ""for every highlight without a note. Create now leavesnotesout when it is empty or whitespace only. This probably explains most of the rejections in the Attach exported highlights to their original Reader documents #37 device test (3 of 4 passages).Addresses #20, following the API update reported in this comment. API contract: Reader create documentation.
Validation: 31 LuaJIT tests pass with KOReader and HTTP stubs (routing, payloads, saved history across restarts, note edits not resent, repeated passages, partial failures, 400 classification, 400/404 fallback including fallback failure and later failures, rate-limit retries, malformed responses, error logging). Luacheck reports 0 warnings. Against a real account (KOReader on Linux), Reader accepted highlights both with and without a note after the blank-notes fix, and one with a note was checked inline in the article. Saved history with quotes, newlines, backslashes, NUL bytes and long passages round-trips through KOReader's own settings serializer (
dump.lua).Limits (the main ones are in the README): highlights exported by earlier versions are not migrated; the first sync after upgrading sends those still on the device to Reader, so they can appear twice in Readwise. Export tracking is local and keyed by document and exact passage. Notes added or edited after export are not sent, a highlight text change creates a new highlight, and local deletions do not propagate. Passages Reader rejects appear in Readwise but not inline in Reader. Resetting settings, losing a create response, or a crash in the middle of a book can allow a later export to be resent.
🤖 Generated with Claude Code
https://claude.ai/code/session_01W8hfPEJmjGWSdSHeaku17w