Repository navigation
watcher: add BUN_WATCHER_USE_POLLING stat-polling fallback for --watch/--hot #36293
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
4d8fad8
watcher: add BUN_WATCHER_USE_POLLING stat-polling fallback for --watc…
robobun 728cdb7
fix clippy: use bun_collections::HashMap, drop dead_code cfg_attrs
robobun b0fb36c
trim comments
robobun 6783710
watcher: auto-enable polling on 9p/NFS/SMB mounts (WSL /mnt/*)
robobun 17bea52
address review: don't advance baseline past MAX_COUNT; only ENOENT/EN…
robobun 165d0c1
env_var: drop default on BUN_WATCHER_USE_POLLING so unset reaches aut…
robobun 3ca7894
watcher: make requires_file_descriptors runtime-aware of polling backend
robobun 7ba9f93
docs: note that polling does not detect newly-created files
robobun 2d87a49
ci: retrigger
robobun File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,250 @@ | ||
| //! Stat-polling fallback for filesystems where native change notifications | ||
| //! don't fire (Docker/WSL bind mounts, NFS/SMB, virtiofs/9p). Enabled via | ||
| //! `BUN_WATCHER_USE_POLLING=1`, or automatically on Linux when the project | ||
| //! root is on such a filesystem; interval via `BUN_WATCHER_POLL_INTERVAL` (ms). | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
|
|
||
| use std::time::Duration; | ||
|
|
||
| use bun_collections::{HashMap, IdentityContext}; | ||
| use bun_core::ZStr; | ||
| use bun_sys::{self as sys, stat_mtime}; | ||
|
|
||
| use crate::watcher_impl::{ | ||
| Backend, HashType, MAX_COUNT, Op, WatchEvent, WatchItemColumns, WatchItemIndex, WatchItemKind, | ||
| Watcher, | ||
| }; | ||
|
|
||
| bun_core::declare_scope!(watcher, visible); | ||
|
|
||
| pub const DEFAULT_INTERVAL_MS: u64 = 100; | ||
|
|
||
| /// Returns `true` when `root` is on a Linux filesystem whose inotify watches | ||
| /// are accepted but never emit events for remote changes, so the polling | ||
| /// backend should be selected even without `BUN_WATCHER_USE_POLLING=1`. | ||
| /// Deliberately conservative: overlayfs and FUSE are excluded because the | ||
| /// common Docker/local case works; users on those can opt in explicitly. | ||
| /// Constants from `include/uapi/linux/magic.h`. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| pub(crate) fn should_auto_poll(root: &[u8]) -> bool { | ||
| #[cfg(target_os = "linux")] | ||
| { | ||
| const V9FS_MAGIC: i64 = 0x0102_1997; // WSL2 drvfs (/mnt/*) | ||
| const SMB_MAGIC: i64 = 0x517B; | ||
| const SMB2_MAGIC: i64 = 0xFE53_4D42; | ||
| const CIFS_MAGIC: i64 = 0xFF53_4D42; | ||
| const NFS_MAGIC: i64 = 0x6969; | ||
| const PRL_FS_MAGIC: i64 = 0x7C7C_6673; // Parallels shared folders | ||
|
|
||
| let mut buf = bun_paths::path_buffer_pool::get(); | ||
| let len = root.len().min(buf.len() - 1); | ||
| buf[..len].copy_from_slice(&root[..len]); | ||
| buf[len] = 0; | ||
| let zpath = ZStr::from_buf(&buf[..], len); | ||
| if let Ok(st) = sys::statfs(zpath) { | ||
| let ty = st.f_type as i64; | ||
| return matches!( | ||
| ty, | ||
| V9FS_MAGIC | SMB_MAGIC | SMB2_MAGIC | CIFS_MAGIC | NFS_MAGIC | PRL_FS_MAGIC | ||
| ); | ||
| } | ||
| } | ||
| let _ = root; | ||
| false | ||
| } | ||
|
|
||
| #[derive(Clone, Copy, Default)] | ||
| struct Snapshot { | ||
| mtime_sec: i64, | ||
| mtime_nsec: i64, | ||
| size: u64, | ||
| ino: u64, | ||
| /// last stat succeeded | ||
| exists: bool, | ||
| } | ||
|
|
||
| impl Snapshot { | ||
| fn differs(&self, other: &Snapshot) -> bool { | ||
| self.exists != other.exists | ||
| || self.mtime_sec != other.mtime_sec | ||
| || self.mtime_nsec != other.mtime_nsec | ||
| || self.size != other.size | ||
| || self.ino != other.ino | ||
| } | ||
| } | ||
|
|
||
| pub struct PollingWatcher { | ||
| pub interval: Duration, | ||
| /// Keyed by `WatchItem.hash`; guarded by `Watcher.mutex`. | ||
| snapshots: HashMap<HashType, Snapshot, IdentityContext<HashType>>, | ||
| /// Watcher-thread-only scratch, reused across cycles. | ||
| scratch_idx: Vec<(WatchItemIndex, HashType)>, | ||
| scratch_path: Vec<Vec<u8>>, | ||
| scratch_stat: Vec<Option<Snapshot>>, | ||
| } | ||
|
|
||
| impl PollingWatcher { | ||
| pub fn new(interval_ms: u64) -> Self { | ||
| Self { | ||
| interval: Duration::from_millis(interval_ms.max(1)), | ||
| snapshots: HashMap::default(), | ||
| scratch_idx: Vec::new(), | ||
| scratch_path: Vec::new(), | ||
| scratch_stat: Vec::new(), | ||
| } | ||
| } | ||
|
|
||
| /// Baseline stat at registration time so a write between `add_file` and | ||
| /// the first poll cycle is detected. Caller holds `Watcher.mutex`. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| pub(crate) fn register(&mut self, hash: HashType, path: &[u8]) { | ||
| self.snapshots | ||
| .insert(hash, stat_path(path).unwrap_or_default()); | ||
| } | ||
|
|
||
| /// Caller holds `Watcher.mutex`. | ||
| pub(crate) fn unregister(&mut self, hash: HashType) { | ||
| self.snapshots.remove(&hash); | ||
| } | ||
|
|
||
| pub fn stop(&mut self) {} | ||
| } | ||
|
|
||
| /// `None` = transient errno (ESTALE/EIO/…); caller keeps the previous snapshot. | ||
| fn stat_path(path: &[u8]) -> Option<Snapshot> { | ||
| let mut buf = bun_paths::path_buffer_pool::get(); | ||
| if path.len() >= buf.len() { | ||
| return Some(Snapshot::default()); | ||
| } | ||
| buf[..path.len()].copy_from_slice(path); | ||
| buf[path.len()] = 0; | ||
| let z = ZStr::from_buf(&buf[..], path.len()); | ||
| match sys::stat(z) { | ||
| Ok(st) => { | ||
| let mt = stat_mtime(&st); | ||
| #[cfg(unix)] | ||
| let (size, ino) = (st.st_size as u64, st.st_ino as u64); | ||
| #[cfg(windows)] | ||
| let (size, ino) = (st.st_size, st.st_ino); | ||
| Some(Snapshot { | ||
| mtime_sec: mt.sec, | ||
| mtime_nsec: mt.nsec, | ||
| size, | ||
| ino, | ||
| exists: true, | ||
| }) | ||
| } | ||
| Err(err) => match err.get_errno() { | ||
| sys::E::ENOENT | sys::E::ENOTDIR => Some(Snapshot::default()), | ||
| _ => None, | ||
| }, | ||
| } | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| } | ||
|
|
||
| /// One polling cycle: sleep, snapshot the watchlist under the mutex, stat each | ||
| /// file, diff against the previous snapshot, emit events. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| pub(crate) fn watch_loop_cycle(this: &mut Watcher) -> sys::Result<()> { | ||
| let _flush = bun_core::output::flush_guard(); | ||
|
|
||
| let interval = match &this.platform { | ||
| Backend::Polling(p) => p.interval, | ||
| Backend::Native(_) => unreachable!(), | ||
| }; | ||
|
|
||
| // Sleep in short slices so `shutdown()` (which clears `running`) doesn't | ||
| // have to wait out a long interval. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| let mut remaining = interval; | ||
| let slice = Duration::from_millis(20); | ||
| while remaining > Duration::ZERO { | ||
| if !this.running.load() { | ||
| return Ok(()); | ||
| } | ||
| let step = remaining.min(slice); | ||
| std::thread::sleep(step); | ||
| remaining = remaining.saturating_sub(step); | ||
| } | ||
| if !this.running.load() { | ||
| return Ok(()); | ||
| } | ||
|
|
||
| let Backend::Polling(poll) = &mut this.platform else { | ||
| unreachable!() | ||
| }; | ||
| let mut scratch_idx = core::mem::take(&mut poll.scratch_idx); | ||
| let mut scratch_path = core::mem::take(&mut poll.scratch_path); | ||
| let mut scratch_stat = core::mem::take(&mut poll.scratch_stat); | ||
| scratch_idx.clear(); | ||
| scratch_path.clear(); | ||
| scratch_stat.clear(); | ||
|
|
||
| // Snapshot (index, hash, path) under the mutex. Between unlock and | ||
| // `dispatch_file_updates`' re-lock the JS thread can only append, so | ||
| // indices stay valid. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| { | ||
| let _guard = this.mutex.lock_guard(); | ||
| let paths = this.watchlist.items_file_path(); | ||
| let hashes = this.watchlist.items_hash(); | ||
| let kinds = this.watchlist.items_kind(); | ||
| scratch_idx.reserve(paths.len()); | ||
| scratch_path.reserve(paths.len()); | ||
| for i in 0..paths.len() { | ||
| if kinds[i] != WatchItemKind::File { | ||
| continue; | ||
| } | ||
| scratch_idx.push((i as WatchItemIndex, hashes[i])); | ||
| scratch_path.push(paths[i].to_vec()); | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| } | ||
| } | ||
|
|
||
| for path in &scratch_path { | ||
| scratch_stat.push(stat_path(path)); | ||
| } | ||
|
|
||
| // Diff under the mutex so `register()` from the JS thread can't interleave. | ||
| let mut event_id: usize = 0; | ||
| { | ||
| let _guard = this.mutex.lock_guard(); | ||
| let Backend::Polling(poll) = &mut this.platform else { | ||
| unreachable!() | ||
| }; | ||
| for (k, &(index, hash)) in scratch_idx.iter().enumerate() { | ||
| let Some(now) = scratch_stat[k] else { continue }; | ||
| let op = match poll.snapshots.get(&hash) { | ||
| Some(prev) if now.differs(prev) => { | ||
| if now.exists { | ||
| Op::WRITE | ||
| } else { | ||
| Op::DELETE | ||
| } | ||
| } | ||
| Some(_) => continue, | ||
| None => { | ||
| poll.snapshots.insert(hash, now); | ||
| continue; | ||
| } | ||
| }; | ||
| // Only advance the baseline when the event is actually emitted, so | ||
| // changes past MAX_COUNT in one cycle surface on the next cycle | ||
| // instead of being silently swallowed. | ||
|
robobun marked this conversation as resolved.
Resolved
|
||
| if event_id >= MAX_COUNT { | ||
| break; | ||
| } | ||
| poll.snapshots.insert(hash, now); | ||
| this.watch_events[event_id] = WatchEvent { | ||
| index, | ||
| op, | ||
| name_off: 0, | ||
| name_len: 0, | ||
| }; | ||
| event_id += 1; | ||
| } | ||
| } | ||
|
|
||
| if event_id > 0 { | ||
| this.dispatch_file_updates(event_id, 0); | ||
| } | ||
|
|
||
| if let Backend::Polling(poll) = &mut this.platform { | ||
| poll.scratch_idx = scratch_idx; | ||
| poll.scratch_path = scratch_path; | ||
| poll.scratch_stat = scratch_stat; | ||
| } | ||
| Ok(()) | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.