Conversation
Add regexp flag for log search.
Add `instance` and `regexp` input fields and the `error` output field for the logs query endpoint.
Add a regexp toggle for log search with dedicated placeholder and helper text, display backend query errors in an inline notification, and show a results summary line with line count and time interval. Rework the start/end date-time layout into a responsive flex group and remove the now-unused collapse/expand filters toggle.
DavidePrincipi
left a comment
DavidePrincipi
left a comment
There was a problem hiding this comment.
The backend part seems ok, I'm proposing just a few little tweaks.
| if s != nil { | ||
| writeSocketResponse(s, "logs-start", gin.H{"id": id, "pid": "", "message": "", "error": message}) | ||
| } else { | ||
| fmt.Fprintln(os.Stderr, message) |
There was a problem hiding this comment.
The message prints twice. It was already printed by line 301.
| fmt.Fprintln(os.Stderr, message) |
There was a problem hiding this comment.
Done in 1890449. I had first moved the print into a shared helper, which kept the duplicate: utils.LogError already writes to stderr. Each path now prints once, the server fault through utils.LogError and the user error through its own fmt.Fprintln.
| args = append(args, streamSelector+logqlPipeline+filter) | ||
| query, errQuery := buildLogqlQuery(logsAction) | ||
| if errQuery != nil { | ||
| writeLogsError(s, logsAction.Id, errQuery.Error(), wg) |
There was a problem hiding this comment.
Please do not log the error here. An invalid user regexp is a typo, not a server fault; it now lands in the api-server stderr/journal.
| writeLogsError(s, logsAction.Id, errQuery.Error(), wg) |
There was a problem hiding this comment.
Done in 4b839d7. The invalid regexp goes through writeLogsQueryError, which does not log: it only sends the message back with an error_code the frontend turns into a translated wording. Checked on a test node, journalctl -u api-server stays empty after a failed pattern.
…string helper text
| class="mg-bottom-lg" | ||
| v-model="internalTimezone" | ||
| > | ||
| <cv-select-option disabled selected hidden>{{ |
There was a problem hiding this comment.
Timezone field should not be displayed in follow mode
There was a problem hiding this comment.
Done in 2d77e1f, the field and its entry in the collapsed filter summary are both hidden while following.
Digging further, hiding it was not enough: the timezone applies to follow output too (logcli -z UTC prints 07:23:02Z, -z Europe/Rome prints 10:01:01+02:00 on the same lines), so a UTC picked for an earlier dump search kept driving live timestamps with no visible control. Follow mode now always uses the browser timezone, 975657f. Verified: dump in UTC shows 2026-08-26T07:11:48Z, switching to follow shows 2026-08-26T10:05:36+02:00.
| : $t("system_logs.expand_filters") | ||
| }} | ||
| </cv-link> | ||
| <NsInlineNotification |
There was a problem hiding this comment.
Let's move this error notification below the Search button
There was a problem hiding this comment.
Done in 7e2af90, the notification now renders after the toolbar row. It is also hidden when the search returned no lines, where it read as a failure report on an empty result.
| this.isFollowing = false; | ||
| this.pid = ""; | ||
| this.noLogsFound = false; | ||
| this.logsError = message; |
There was a problem hiding this comment.
The regex error is not internationalized 🤔
There was a problem hiding this comment.
Done. The error banner is translated (title and sentence), and the engine detail is appended as it comes: "The search pattern is not valid RE2 syntax: error parsing regexp: missing closing ): (". That tail is produced by Go's regexp package and names the offending token, so it stays in English rather than being dropped.
| "context_module": "Application", | ||
| "search_query": "Search query", | ||
| "regexp": "Regular expression", | ||
| "regexp_helper": "RE2 syntax is accepted. Hint: (?i) ignores case, | matches any of the words", |
There was a problem hiding this comment.
| "regexp_helper": "RE2 syntax is accepted. Hint: (?i) ignores case, | matches any of the words", | |
| "regexp_helper": "RE2 syntax is accepted. Hint: '(?i)' ignores case, '|' matches any of the words", |
There was a problem hiding this comment.
Applied. The string then had to change once more: regexp mode no longer highlights matches, so it now reads "RE2 syntax is accepted, matches are not highlighted. Hint: '(?i)' ignores case, '|' matches any of the words". Tell me if you prefer that second sentence elsewhere.
There was a problem hiding this comment.
back to your proposal
|
It looks like some patterns (e.g. |
… thread The search pattern comes from the user and runs through the JavaScript engine, which backtracks. Loki validates with Go RE2, which simulates the automaton instead and therefore accepts patterns no backtracking engine can run: deciding whether a line matches is linear, but finding where the match sits inside it is 2^n. Measured on a real log line, 5.5 ms at 18 characters and 340 ms at 24, a factor of 1.98 per added character, on lines averaging 245. The regex API offers no timeout and no interruption point, so a match on the main thread freezes the tab beyond recovery. Running it here lets the caller enforce a deadline by terminating the worker, which is the only reliable way to stop a regex already backtracking. The matching logic is the one that previously lived inline in LogHighlightMark, moved unchanged: the zero-length-match guard, the adjacent-match merge, the MAX_PARTS ceiling, and matching each line segment separately so a match cannot span the timestamp/tag boundary. Offsets are returned rather than substrings, to keep the reply from carrying a second copy of the log text. Built from a Blob because the toolchain is webpack 4, which has no support for `new Worker(new URL(...))`, and worker-loader is not a dependency.
Splitting a line used to mean executing the user pattern here, once per line segment, during render. That is what froze the tab on a catastrophic pattern, and it also looped forever on a zero-length match. The pattern now runs in a worker under a deadline, so this component only slices the text at the offsets it receives, and renders nothing highlighted while they are missing or when the pattern turned out to be too expensive. Verified identical to the previous splitter across 7600 line/pattern comparisons on real log lines, covering "()", "(.)", "\w*", "\b" and substring searches. Level colorization stays here, computed from four fixed internal patterns with no user input, so it carries no risk and its behaviour is unchanged.
Every line was joined into a single string and handed to vue-text-highlight along with the user pattern. Its indicesOf() has no zero-length-match guard, so "()", ".*" or "\w*" never advanced lastIndex and looped forever filling an array, hanging the tab; its mergeRange() also keeps contiguous ranges separate, so "(.)" produced one chunk per character over the whole output. Render one LogHighlightMark per line instead, which is what that component was written for. Wrapping each line in a <mark> also applies the UA rule that sets the mark text color, so lines with no level class came out black on the black log background: reset the color as well, and let the level rules that follow override it. The removed LOG_LEVEL_QUERIES and PROCESS_TAG_QUERY only told the library where to cut. Colorization comes from LOG_LEVEL_CLASSIFIERS and PROCESS_TAG_PATTERN in LogHighlightMark, and was verified unchanged: same tally over 500 real lines. Own the worker here, since outputLines and highlight already arrive as props. outputLines is trimmed from the front in follow mode, so index bookkeeping across updates would silently misalign: recompute the whole buffer, debounced, which costs 1-30 ms for every realistic pattern. Each request carries an id so a reply for a superseded search is dropped instead of applied to different lines. Past a 400 ms deadline the worker is terminated and the lines stay on screen unmarked behind an inline notification, since they are still the ones the backend matched. Never fall back to matching on the main thread, which is the freeze.
Worded around what actually happened: locating the matches inside each line is what is too slow, not the search, so the unmarked output is not read as "the pattern matches nothing".
Loki runs RE2, so the pattern is validated server-side and the failure came back as a raw Go message. Tag it with a code so the UI can word it in the user language, and keep the engine detail as the only English part left.
…e up In follow mode the highlight is recomputed on every batch of new lines, so a pattern past the deadline started a fresh doomed worker every 120 ms and kept a core busy. Remember the pattern that gave up until it changes or a new search clears the buffer.
A substring is escaped before it is compiled, so matching it is linear and can never be the expensive case the notice describes: seeing it there only suggests the search is broken. Two changes make the notice regexp-only. Past the mark budget the worker now stops marking instead of dropping every mark, so a common letter searched over a full buffer keeps its highlighting on the lines already processed. The budget goes down to 20000 marks: 50000 cost 3.8s of rendering, 20000 cost 1.3s. The deadline is left as the only way to lose the highlighting entirely, and it belongs to a backtracking pattern, which only regexp mode can produce.
Loki matches with RE2 and the browser with its own engine, which knows a different dialect: "(?P<lvl>error|warn)" and "err(?i)OR" are valid queries the browser rejects. The fallback treated the pattern as literal text, so the search worked and nothing was ever highlighted, with no explanation. Report it instead, with its own wording: the lines are the ones that matched, only the marking is missing.
A search matching every other character turns each line into dozens of fragments, and the template compiler was spending ten times longer diffing them than the browser spends parsing the equivalent markup: 2000 lines and 50000 marks cost 3.8s, against 0.35s for the same DOM built directly. Two costs, both paid per fragment. The line is now assembled as one escaped string rendered through v-html, so Vue sees one node per line instead of fifty. And the offsets coming back from the worker are frozen, which stops Vue from installing a reactive accessor on each of the hundreds of thousands of numbers they contain -- on its own that was three quarters of the time. The same search now costs 1.0s and highlights all 2000 lines instead of the first 895, so the mark budget goes back up to 100000.
The viewer scrolls to the bottom in both modes, so spending the budget on the oldest lines left the visible ones plain: the highlighting looked broken while it was in fact complete on lines nobody was looking at.
melody defaults to a 512 byte read limit and closes the connection past it. A logs-start payload is 280 bytes with an empty query, so a search of about 240 characters was enough to drop the socket: measured, 507 bytes go through and 522 kill it. Regexp queries reach that length easily. The frontend was then waiting for a reply that could never come, spinner running until the page was reloaded, so it now ends the pending search when the connection drops.
Cut the restatements and keep one line per non-obvious constraint: the RE2 dialect gap, the backtracking deadline, the v-html escaping, the Vue 2 reactivity walk, the melody message limit.
Loki matches with RE2 and the browser with its own engine, and making the
second one locate the matches of a pattern written for the first cost more
than the feature is worth: a pattern that is linear in RE2 can be
exponential in JavaScript, which needed a worker and a deadline to stay
recoverable; the two dialects disagree without erroring, so "\Q...\E" or
"\p{L}+" returned the right lines and marked nothing; and syntax RE2
accepts but JavaScript rejects needed a notice of its own.
Regexp mode now marks nothing, and the substring highlighter goes back to
what it was before this branch. No regex is built from user input in the
browser any more, so the worker, its deadline, the mark budget, the
abandoned-pattern cache and both notices have nothing left to protect.
utils.LogError already writes the message to stderr, so the CLI path printed it a second time.
|
The freeze is gone, and not by containing it: regexp mode no longer highlights matches at all, so no pattern written by the user is ever compiled by the browser engine. Substring mode keeps the highlighter it had before this branch, unchanged. Measured on a test node, 500 lines, worst main-thread stall per search:
|
The timezone field is hidden while following, but the value was still sent: a UTC picked for an earlier dump search kept driving the timestamps of live lines, with no visible control to change it back.
The error and error_code attributes of logs-start need a matching case on every consumer, and let the frontend pick a translated wording from a backend constant the API document does not mention. A query fails the way logcli and Loki word it, and that wording is log output: sending it as message prints it in the output area, with no attribute, no second error writer and no notification above the logs. Loki compiles the regexp of a |~ filter itself, so the query builder no longer duplicates that check to raise an invalid_regexp code, and the tail goroutine sends logs-stop whenever it returns, which is how the frontend already leaves the follow state. Assisted-by: Claude Code:claude-opus-5
DavidePrincipi
left a comment
DavidePrincipi
left a comment
There was a problem hiding this comment.
The last commit removes the inline notification that displays backend error details. Since this part required a change of the websocket protocol, I preferred to make a step back and report backend errors in the text area itself.
I know it is less effective for UX, but the changes introduced by the feature (which should be rarely used) should be kept small.
I'm requesting Ande's review again.
In follow mode the command stderr was buffered and reported only after the process ended, so a message about a query that keeps running was never shown, and a clean exit dropped it altogether. Both streams now share one pipe: what logcli says about a failure reaches the frontend as log text, in order with the lines around it. Reading that pipe to EOF before Wait also closes a race, as Wait used to close the pipe while the scanner was still reading from it. Assisted-by: Claude Code:claude-opus-5
f9c33f0 to
0977d99
Compare
There was a problem hiding this comment.
Filed upstream bug grafana/loki#24249 about tail mode hangs in some cases.
NS8 bug NethServer/dev#8140 for core 3.22+
3 participants
Adds a "Regular expression" toggle to the System Logs search bar. When enabled, the filter is applied with the LogQL
|~operator instead of the substring|=.Backslashes are now escaped along with quotes for both operators, so a filter carrying either no longer produces an unparsable query and a misleading "No log found".
Invalid patterns are rejected client-side, and again server-side with Go regexp (RE2, the engine Loki runs). logcli failures now reach the UI as an error notification: until now a broken query was indistinguishable from an empty result.
Also from the Figma mockup:
Refs NethServer/dev#8032
Design: NethServer/dev#7974
Testing
redis: results unchangedredis|traefikand^2026-05[unclosed: error notification, no empty stateapi-server-logs logs -s 'err(or)?' -r -m dump -l 50