Skip to content

Prevent focus on non-cursor thoughts when tapping the edge of the editable#4407

Draft
ethan-james wants to merge 37 commits into
cybersemics:mainfrom
ethan-james:4394-bug-prevent-focus-on-non-cursor-thoughts
Draft

Prevent focus on non-cursor thoughts when tapping the edge of the editable#4407
ethan-james wants to merge 37 commits into
cybersemics:mainfrom
ethan-james:4394-bug-prevent-focus-on-non-cursor-thoughts

Conversation

@ethan-james

@ethan-james ethan-james commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Fixes #4394 and #4291

How the regression test was built

src/e2e/iOS/__tests__/caretFocus.ts reproduces #4394 — the virtual keyboard incorrectly opening when a non-cursor thought is tapped just past its right edge. It passes on this branch (#4407) and fails on the older pre-fix commit. Many approaches were tried this session; these are the ones that turned out to be load-bearing for the final test:

  1. Finger-sized "fat-tap," not a point tap. The edge tap uses a real contact area (width: 40, height: 40, pressure: 0.9). A zero-radius synthetic tap does not trigger Safari's touch-adjustment heuristic; only a finger-sized contact does.
  2. Tap ~4px past the right edge, vertically centered (x = rect.x + rect.width + 4, y = vertical center). This lands the touch on the #thought-annotation overlay — so the editable's onTouchEnd/preventDefault never runs — while staying close enough that Safari retargets the synthesized mousedown into the editable.
  3. Clear the cursor with cursorBack (gesture('r')) so "Hello" becomes a non-cursor thought — the exact path [iOS] Keyboard incorrectly opens when tapping right edge of thought #4394 is about.
  4. Pin to iOS 18 (iPhone 16 Pro Max). The touch-adjustment retargeting fires on iOS 18 but not on the suite's default iOS 17 device.
  5. Prime with a native performActions tap, not a webview element.click(). On real hardware only a native touch focuses the editable and opens the keyboard; the priming tap lets the later synthetic edge-tap win the timing race a real finger triggers on its own.
  6. Isolate the spec onto its own iOS 18 capability. A second BrowserStack capability runs only this spec on iOS 18 while the rest of the suite stays on iOS 17.

Discarded along the way (did not contribute): a sim-only clickThought webview priming (does not open the keyboard on real devices), a releaseActions() call after performActions (unsupported on Safari/XCUITest), and earlier point-tap / x-offset tuning that the fat-tap superseded.

@ethan-james ethan-james self-assigned this Jun 17, 2026
@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

@ethan-james is attempting to deploy a commit to the Cybersemics Institute Team on Vercel.

A member of the Team first needs to authorize it.

@ethan-james ethan-james marked this pull request as draft June 17, 2026 22:58
@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
em Ready Ready Preview, Comment Jun 22, 2026 3:52pm
em-ai Ready Ready Preview, Comment Jun 22, 2026 3:52pm

@ethan-james

Copy link
Copy Markdown
Collaborator Author

@raineorshine This issue is also related to events failing to dispatch when a tap is on the edge of an editable. In this case it's touchend rather than mouseup, so that the focus element is not prevented. I still haven't figured out how to get browserstack tests to behave the same as a real tap under these circumstances, and so I haven't been able to get the test to fail.

@raineorshine

Copy link
Copy Markdown
Contributor

Thanks. A couple of points of clarification:

  • "failing to dispatch" - Is this because preventDefault is being called somewhere?
  • "edge of an editable" - Exactly what tap area results in a missed touchend? Is it related to padding-right?

As for the BrowserStack tests, that's going to be a critical piece for us to get test coverage. Feel free to reach out to Faiz, as he has gained a lot of experience recently getting the BrowserStack tests to work for our agents. It's usually a matter of which event interaction API to use.

@ethan-james

Copy link
Copy Markdown
Collaborator Author

Thanks. A couple of points of clarification:

  • "failing to dispatch" - Is this because preventDefault is being called somewhere?

Not that I know of. If the touchend were being registered in the correct place, then handleTapBehavior would call preventDefault and stop the focus event. I think that touchend is probably being dispatched by a different element than touchstart, or else not at all.

  • "edge of an editable" - Exactly what tap area results in a missed touchend? Is it related to padding-right?

I experimented with extending padding-right, and it seemed to behave normally with a padding of 5em or 10em. It seems like the problem occurs when the touch occurs to the right (on the edge) of whatever padding is set.

I'll try to profile it a bit better to get a better answer. Hopefully that will be helpful in figuring out how to construct a failing test. My hypothesis is that BrowserStack is dispatching all events at the exact same pixel coordinate, and that the real touch events are not quite so precise on a device. It's also possible that it's not dispatching touchstart and touchend in the way that we would expect, as we saw on the ToolbarButton test. I'll let you know what I find, and discuss with Faiz.

As for the BrowserStack tests, that's going to be a critical piece for us to get test coverage. Feel free to reach out to Faiz, as he has gained a lot of experience recently getting the BrowserStack tests to work for our agents. It's usually a matter of which event interaction API to use.

@raineorshine

Copy link
Copy Markdown
Contributor

Thanks. A couple of points of clarification:

  • "failing to dispatch" - Is this because preventDefault is being called somewhere?

Not that I know of. If the touchend were being registered in the correct place, then handleTapBehavior would call preventDefault and stop the focus event. I think that touchend is probably being dispatched by a different element than touchstart, or else not at all.

Okay. And I don't believe we're calling stopPropagation anywhere either.

Keep working on identifying why touchend is not called in this case, as understanding that causal element will be important to designing the right solution.

  • "edge of an editable" - Exactly what tap area results in a missed touchend? Is it related to padding-right?

I experimented with extending padding-right, and it seemed to behave normally with a padding of 5em or 10em. It seems like the problem occurs when the touch occurs to the right (on the edge) of whatever padding is set.

Yeah, I'm curious about that. In my testing it didn't seem like it was confined to a single 1px edge. So I'm wondering what exactly the bounds are and what determines them. Maybe native mobile Safari is somewhat forgiving about how close the touch has to be to the editable to trigger a focus?

I'll try to profile it a bit better to get a better answer. Hopefully that will be helpful in figuring out how to construct a failing test. My hypothesis is that BrowserStack is dispatching all events at the exact same pixel coordinate, and that the real touch events are not quite so precise on a device. It's also possible that it's not dispatching touchstart and touchend in the way that we would expect, as we saw on the ToolbarButton test. I'll let you know what I find, and discuss with Faiz.

Sounds like a plan 👍

@ethan-james

ethan-james commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator Author

#4291 and #4394 are no longer reproducible after #4371 was merged. I haven't merged main into this branch so that I can still figure out how to write this Browserstack test.

@ethan-james

ethan-james commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator Author

As for the BrowserStack tests, that's going to be a critical piece for us to get test coverage. Feel free to reach out to Faiz, as he has gained a lot of experience recently getting the BrowserStack tests to work for our agents. It's usually a matter of which event interaction API to use.

Note: As mentioned above, this bug was fixed by a mostly-unrelated PR that was merged into main. It is still reproducible on this branch because I haven't merged main into it recently.

Hi @fbmcipher, I am trying to write a test that taps slightly past the right edge of an editable in order to reproduce this bug. I was using the tap helper, but have since switched to gesture in hopes that it more faithfully models pointerType: 'touch'. Some relevant context:

  • touchstart and touchend are dispatched on the thought annotation "behind" the editable, at an x position like 148.66666666666666
  • mousedown and mouseup are dispatched on the editable itself at x position 141, i.e. the very right edge of the editable's bounding rect

So I need to figure out how to tap in a position past the edge of the editable while getting iOS Safari to retarget the event onto the editable. Do you know how I could do that?

Here is some more info from Copilot. Sounds pretty reasonable, but I didn't get much info about "compatibility-event behavior" from Google.


What you are likely seeing is a WebKit compatibility-event behavior, not a bug in your math:

  1. iOS generates compatibility mouse events from the touch sequence, but it is allowed to retarget/adjust them for editability and hit-testing.
  2. For contenteditable, Safari often resolves the tap to a caret-valid insertion point first, then emits mouse events from that resolved point, not the original finger centroid.
  3. In “void” regions (padding/gaps/line-end space), that resolved point is commonly at a line edge or nearest glyph boundary, which looks like “always at the edge.”
  4. Fractional touch coordinates plus integer-ish mouse coordinates are also normal: touch uses subpixel CSS coords; synthesized mouse frequently appears quantized.

So the pattern “touch says X, mouse on editable says edge X” is plausible and consistent with Safari’s behavior around editable text.

Practical takeaway: on iOS Safari, trust touch coordinates (touchstart/touchend.changedTouches) for caret placement, and treat compatibility mouse coordinates as potentially normalized/retargeted.

@ethan-james

Copy link
Copy Markdown
Collaborator Author

#4291 and #4394 are no longer reproducible after #4371 was merged. I haven't merged main into this branch so that I can still figure out how to write this Browserstack test.

I got the test to fail, so I merged in main to watch it pass, but it does not pass. I tested document.activeElement === document.body locally to see if that was the right condition to look for, and it definitely seems to be true on my local iPhone. I'll come back to it and try to find the cause, but @fbmcipher please feel free to weigh in if you know why it might be behaving differently in the test.

ethan-james and others added 9 commits June 30, 2026 10:19
The 'Focus is prevented after clearing the cursor' test tapped the home
button with a synthetic tap() (performActions pointer events), which does
not fire the real click event that HomeLink's fastClick onClick handler
relies on. As a result the cursor was never cleared and the test only
passed as an artifact of leftover focus from newThought.

Use a native WebDriver click on the inner anchor to genuinely clear the
cursor, and tap the editable's right edge at x: 6, y: 60.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ethan-james

Copy link
Copy Markdown
Collaborator Author

#4291 and #4394 are no longer reproducible after #4371 was merged. I haven't merged main into this branch so that I can still figure out how to write this Browserstack test.

I got the test to fail, so I merged in main to watch it pass, but it does not pass. I tested document.activeElement === document.body locally to see if that was the right condition to look for, and it definitely seems to be true on my local iPhone. I'll come back to it and try to find the cause, but @fbmcipher please feel free to weigh in if you know why it might be behaving differently in the test.

@fbmcipher I managed to get Appium and @wdio/mcp running and managed to figure out a bug in the test where tapping the home button wasn't clearing the cursor. However, I am trying to get this test to fail in #4482 and it just won't happen. Copilot eventually threw in the towel:

Findings

You were right that the home tap was broken — that was the key discovery. tap(homeNodeHandle) uses synthetic WebDriver pointer actions that don't fire a real click event, so HomeLink's fastClick onClickhandler never ran and the cursor was never cleared. The fix is to use WebDriver's native .click() on the inner (like the existing clickThought helper does).

But fixing the home tap revealed the deeper problem: once the cursor is genuinely cleared, the bug doesn't reproduce locally on either branch at any tap offset. The elementFromPoint probe showed why — locally the contenteditable is shrink-wrapped to the text (65px) and the void area beside it is covered by an absolutely-positioned overlay (pos_absolute us_none). Taps in the void hit that overlay, not the editable, so the fix's guarded code path (onMouseDown/onFocus on the editable) never executes.

The apparent "x:6 fails / x:26 passes" split I saw earlier was an artifact of the broken home tap leaving the editable focused from newThought, not the actual bug.

Bottom line: this focus bug is specific to BrowserStack's rendering/layering and does not reproduce on the local iOS 18.6 simulator.


I'm about ready to recommend that we move on from this, since I believe the issue to be fixed and I also have a reasonable Browserstack test that passes in this branch even though I can't get it to fail in the other branch. I thought I'd solicit your help one more time to see if you have any ideas for how to proceed. Thanks!

@fbmcipher

Copy link
Copy Markdown
Collaborator

Hey @ethan-james! I'll take a quick look at this tonight... will try and update in the next few hours.

@fbmcipher

fbmcipher commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Hey @ethan-james – I dug into this in a background terminal while working on other things tonight. Ran a few tests to try and get a bit more insight. The following is pretty much verbatim output from Claude, but maybe it helps.

I'm a bit skeptical about its claim (2) that the root issue isn't fixed. I almost wanted to hold off on posting this until I could figure out why it was so sure about that, but didn't want to leave you hanging waiting for a response. Figured an imperfect analysis is better than none at all. It's probably worth reading the agent's reasoning/thinking process below, in any case. Maybe it'll help you figure out what's going on.

I'm also skeptical about claim (3) – like your Copilot, it's also pretty confident that "the bug needs a real finger to reproduce", but something tells me that just cannot be true. It wouldn't be the first time AI insists on a conclusion that is vacuous and provably false. And it doesn't make sense – we're not doing anything that exotic in the app...

I have some confidence that I could get to the bottom of it with more time, but we need to make sure claim (2) is addressed first. If, indeed, the bug is not fixed, that would explain the issues.

If the bug is fixed (and the AI is hallucinating, or I am just confused) then I'd support you in just moving on if Raine's cool with it.

The following is agent output from Opus 4.8 + Fable 5 working together:


  1. It doesn't reproduce once the cursor is properly cleared — but that's true everywhere, not just the local simulator. It also stayed correct on a real iPhone (iOS 27 developer beta), on a real iPhone 16 Pro on BrowserStack (iOS 18), and — importantly — with Move setCaretOffset into onMouseDown and preventAutoscrollEnd back into onFocus #4371 reverted.
  2. [iOS] Keyboard incorrectly opens when tapping right edge of thought #4394 does not appear to be fixed. Move setCaretOffset into onMouseDown and preventAutoscrollEnd back into onFocus #4371 provably doesn't change this behavior, and the relevant code hasn't changed since Raine filed the bug.
  3. The most likely reason none of us can get the test to fail is that the bug only triggers for a real fingertip — a synthetic/injected touch reproduces most of the sequence but not the final step. I'll be upfront that this one is a hypothesis, not something I could prove — and it may well be where we're missing something.

Here's what I found.

Where you're spot on ✅

  • The home tap was broken. Confirmed — synthetic pointer actions don't fire fastClick's onClick, so the cursor never actually cleared. Native .click() on the inner <a> (like clickThought) is the right fix. I double-checked that the cursor really goes null by reading window.em.store.getState().cursor after each clear.
  • The "x:6 fails / x:26 passes" difference was a false alarm caused by that same broken home tap. When the cursor hasn't actually been cleared, tapping the thought simply focuses it and opens the keyboard — which is the normal, correct behavior for a thought that already has the cursor. It only looked like the [iOS] Keyboard incorrectly opens when tapping right edge of thought #4394 bug because we believed the cursor was cleared when it wasn't. This turns out to be the key to the whole thing (below).
  • The viewport↔screen offset is real. My runtime calibration measured it at ~50–62px depending on the device (which lines up with your y:60).

One correction on the mechanism

Copilot's conclusion — "taps hit the overlay, not the editable, so onMouseDown/onFocus never execute … specific to BrowserStack's rendering/layering" — isn't quite right. elementFromPoint is a static hit-test: it reports the annotation overlay sitting at the touch point, but iOS then retargets the synthesized mouse events onto the editable. My event logs (capture phase) show mousedown / mouseup / click firing on .editable (contenteditable=true) every single time — on the local simulator and on real devices. So onMouseDown definitely runs; it takes the else branch and calls e.preventDefault(). That's the important part — it isn't a rendering quirk, and it isn't specific to BrowserStack.

What actually happens (with the cursor confirmed cleared), everywhere

A touch lands on the ThoughtAnnotation → iOS retargets mousedown onto the editable → onMouseDown's else branch calls preventDefault()the focus is suppressed, so the keyboard stays down → the cursor moves to the thought. That's the correct behavior, and it was identical in every environment I tried:

Where iOS How the cursor was cleared (confirmed cursor === null) Tap Keyboard opens?
Local simulator 18.4 home .click() + Done button edge +3…+8 No
iPhone 12 mini (real, USB) 27 Cursor Back (swipe →) edge +3…+8 No
iPhone 16 Pro (BrowserStack) 18 Cursor Back (swipe →) edge +3…+5, tap-hold 100–650ms No
Local sim, #4371 reverted 18.4 Cursor Back (swipe →) edge +3…+5 No

In every row the mouse event retargets onto the editable, but no focus event ever fires, so the keyboard stays down. I also varied the tap hold time from 100ms up to 650ms (including the 415–650ms window the onFocus comment specifically calls out) — no difference. The only setup that opened the keyboard was the one where the cursor hadn't actually been cleared — i.e. the same false alarm described above.

Why I don't think it's fixed

Two findings:

1. #4371 doesn't change this behavior. I reverted only useEditMode.ts back to its state just before #4371 (that file hasn't been touched since #4371, so this isolates that PR exactly) and re-ran the identical sequence: same result — the keyboard never opened. Looking at the diff, that makes sense: #4371 changes the branch that runs when the cursor is already on the thought, and adds a guard that keys off a touchstart firing on the editable — but here touchstart fires on the annotation, so that guard never engages. In other words, "no longer reproducible after #4371" was the cleared-cursor false alarm, not #4371 itself. And worth noting: #4371's own PR description says it fixes #4298 ("[iOS] Caret stays on old sibling with children" — a caret bug, now closed), not #4394. So this keyboard bug was never its target; it looks like #4394 got assumed-fixed by association with a neighbouring tap-handling change that landed around the same time (and once the home-tap issue was sorted, it stopped "reproducing" on both branches, which reinforced that).

2. The code behind #4394 hasn't changed since Raine filed it. #4394 was filed June 14. Since then, the only commits touching the relevant files are #4371 (just shown to be neutral here) and #4253 ("Remove useMultiline" — a 2-line change nowhere near the focus/keyboard logic). And the else-branch preventDefault that suppresses the focus in all our automated runs has been in place since well before the report (it traces back through #3410 to #2948). So Raine hit this bug with that preventDefault already in place — meaning that line isn't what fixes it.

So the explanation that fits all the evidence: on a real device the focus fires anyway, despite the preventDefault; under automation the preventDefault holds and the focus never fires. The retarget step reproduces under automation; the "focus-despite-preventDefault" step doesn't. That would explain why none of us can get the test to fail on any platform — and, since the code that produced the bug is unchanged, it points to the bug still being live for real users. (This is the part I'm least sure about — as noted up top. It's the best explanation I have, but exactly the kind of thing worth confirming by hand.)

How I'd proceed

  • Settle whether it's actually fixed, first — by hand. It's Raine's own repro, so it's a ~30-second check on current main / the Vercel preview: create Hello, close the keyboard, Cursor Back until the cursor is null, then tap ~4px past the right edge of the text. If the keyboard opens, it isn't fixed. This one check resolves the rest: if it's still broken by finger yet green in every automated run, that's what would tell us it isn't automatable.
  • Hold off on more device-tap test work until that's settled. If an injected touch genuinely can't trigger it (confirmed so far on local Appium, real-device Appium, and BrowserStack), then the passing BrowserStack test isn't really guarding anything — it would pass whether or not the bug exists.
  • If it's confirmed still live, it needs a real fix — and since the preventDefault doesn't stop the focus for real touches, the fix likely has to handle the focus when it fires, in onFocus: clear the selection and set keyboardOpen(false), the same way the existing showCommandCenter path already does.
  • There's also a way to test the fix that doesn't depend on the touch at all. The thing that actually triggers the bug is the focus event. A component test (Vitest) that fires focus on a non-cursor editable and asserts the keyboard stays down would be a genuine regression guard — no device tap required.

@ethan-james

Copy link
Copy Markdown
Collaborator Author

Thanks @fbmcipher! It wouldn't hurt for me to dig a little to provide a better understanding of why #4371 fixed the bug, since it was targeting a different issue, and I couldn't quickly ascertain what is different about the path that the events take post-#4371. As for claim (3), I wouldn't be too surprised to learn that was true, given the quirkiness of this issue.

In case I haven't provided enough context, the issue is that a tap outside the boundaries of the editable will trigger mouse events on the editable even though touch events are registered on the element that is actually being tapped. Generally, that's the thought annotation. This seems to be an iOS Safari trick to improve the accessibility of input elements, but unfortunately, touch events are key to prevent focus for editables other than the cursor thought.

it looks like #4394 got assumed-fixed by association with a neighbouring tap-handling change that landed around the same time

I wish it said more about what it meant by that. I don't know for sure that #4371 fixed it, just that it appears fixed now.

@ethan-james

ethan-james commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

I did manage to reproduce it on the simulator in commit 68399a5, so hopefully getting the test to fail won't be so difficult after all.

I also bisected and verified that it was reproducible starting from f00598d (#3410) and ending with 1a9f091(#4371).

Replace the weaker home-link/activeElement variant of the "Focus is
prevented after clearing the cursor" test with a faithful reproduction of
cybersemics#4394 in a dedicated caretFocus.ts spec: create a thought, dismiss the
keyboard, clear the cursor via cursorBack, then fat-tap just past the right
edge of the non-cursor thought and assert the keyboard stays hidden.

The bug only reproduces on iOS 18 (Safari touch-adjustment retargets the
synthesized mousedown into the editable), so the BrowserStack config is
split into two capabilities: the suite runs on iOS 17 (excluding this spec)
and caretFocus.ts runs on a dedicated iOS 18 device.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Inlining the two capabilities in the array literal triggered TS2352: the
extra `exclude`/`specs` keys fail the excess-property check when the array
is cast to WebdriverIO.Config['capabilities']. Extracting each capability
to a standalone const widens its type before the cast, so tsc passes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ethan-james ethan-james force-pushed the 4394-bug-prevent-focus-on-non-cursor-thoughts branch 2 times, most recently from b6b861e to 525b4bb Compare July 6, 2026 21:58
ethan-james and others added 4 commits July 7, 2026 11:47
The 'Focus is prevented after clearing the cursor' test tapped the home
button with a synthetic tap() (performActions pointer events), which does
not fire the real click event that HomeLink's fastClick onClick handler
relies on. As a result the cursor was never cleared and the test only
passed as an artifact of leftover focus from newThought.

Use a native WebDriver click on the inner anchor to genuinely clear the
cursor, and tap the editable's right edge at x: 6, y: 60.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…b.com:ethan-james/em into 4394-bug-prevent-focus-on-non-cursor-thoughts
ethan-james added a commit that referenced this pull request Jul 7, 2026
Adopts the touch techniques proven in #4407 to give the #4173 repro a real
chance of failing on hardware:

- doubleTap now dispatches from the NATIVE_APP context using a finger-sized
  contact area (width/height/pressure). A zero-radius webview-context tap does
  not trigger Safari's touch-adjustment / rapid-tap focus handling and cannot
  reproduce the bug. Both element centers are resolved in the webview context
  before switching, so the sub-second interval stays deterministic.
- Moves the #4173 test into its own spec (caretAdjacentTap.ts) pinned to a
  dedicated iOS 18 capability (iPhone 16 Pro Max), mirroring #4407's split. The
  retargeting behavior fires on iOS 18 but not the suite's default iOS 17, which
  is why the earlier iOS 17 run gave a false pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[iOS] Keyboard incorrectly opens when tapping right edge of thought

3 participants