[pull] develop from baserow:develop#363
Merged
Merged
Conversation
…ace (#5666) * feat(backend): add per-workspace timing telemetry to periodic field updates * feat(backend): add per-workspace periodic field update task with overlap lock * fix(backend): guard periodic field update lock release and cover missing workspace * feat(backend): fan out periodic field updates into one task per workspace * fix(backend): run periodic field update management command synchronously * chore: changelog for periodic field update fan-out * docs(backend): note periodic update eligibility is snapshotted once * refactor(backend): simplify periodic update comments and restore setting-based dispatcher timeout * test(backend): lock dispatch ordering, dispatch metric, and slow-workspace warning * refactor(backend): batch periodic field updates and use singleton lock Address review feedback on PR #5666: - Fan out into BASEROW_PERIODIC_FIELD_UPDATE_BATCH_COUNT tasks (default 1) instead of one task per workspace, so the export queue isn't flooded on large instances. - Replace the manual cache lock with celery-singleton on the batch task (unique_on batch_index) so a slow cycle can't stack on itself. - Record the dispatch metric only for dispatched runs, not the inline management command. * refactor(backend): use itertools.batched for workspace batching * refactor(backend): inline batching and drop otel histograms Inline the batch splitting into the dispatcher (itertools.batched) and remove the per-workspace and dispatch OpenTelemetry histograms. Keep the per-workspace timing and the >60s slow-workspace warning log. * feat(backend): add fenced acquire/extend/clear_if to SingletonAutoRescheduleFlag * feat(backend): add periodic run-lock constants and finish callback * refactor(backend): guard periodic field updates with a fenced per-cycle run lock Dispatch periodic field batches as a chord instead of independent .delay() calls, so the callback reliably fires once all batches finish and releases the per-cycle Redis lock. Batches heartbeat the lock's TTL instead of running as celery_singleton tasks. * docs(backend): changelog for periodic field update run lock * docs(backend): restore flag docstrings and plainer changelog wording * refactor(backend): token-fence periodic run-lock heartbeat; add env vars to .env.example * refactor(backend): warn on periodic field update overlap with tuning hint * fix(backend): isolate periodic field update failures per workspace * refactor(backend): address review feedback on periodic field update - parse BASEROW_PERIODIC_FIELD_UPDATE_BATCH_COUNT with "or 1" so an empty env string falls back to the default - round-robin workspaces across batches so the most overdue run first in parallel instead of being stranded at the tail of one batch - drop the default value from the docstring so it can't rot - make clear_if/extend_if atomic via redis eval to avoid TOCTOU, keyed and encoded through the cache client to match what acquire wrote - escalate the overlap skip to an error naming the settings to tune * fix(backend): harden periodic field update run lock (review round 2) - stop swallowing SoftTimeLimitExceeded: re-raise it in the per-field-type helper and return cleanly in the batch loop, so a soft timeout ends the batch and the chord releases the lock instead of running to the hard-limit SIGKILL and stranding the lock until its TTL - heartbeat the run lock before every workspace, not just once per batch, so a long serialized batch keeps the TTL fresh and the next batch still finds the lock held when it dequeues - log the batch "lock no longer held" stop at warning, not info: it drops the rest of the batch's work for the cycle - acquire the run lock before the eligibility scan so an overlapping run skips cheaply instead of scanning and discarding the result - fetch each field type's due fields once and early-return before refresh_now, dropping the duplicate exists() query per workspace - resolve SingletonAutoRescheduleFlag's default timeout in the body so override_settings(AUTO_INDEX_LOCK_EXPIRY=...) is honored again - note the management command bypasses the run lock (inline path) - document BASEROW_PERIODIC_FIELD_UPDATE_BATCH_COUNT in configuration.md * refactor(backend): pass run-lock TTL in the chord callback for uniformity clear_if ignores the timeout, so this is cosmetic, but every other SingletonAutoRescheduleFlag(RUN_LOCK_KEY, ...) construction passes RUN_LOCK_TTL. Keeping the callback consistent avoids a reader wondering whether the omission is meaningful.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )