Skip to content

Add semantic tokens support - #136

Open
msujew wants to merge 2 commits into
mainfrom
msujew/semantic-tokens
Open

Add semantic tokens support#136
msujew wants to merge 2 commits into
mainfrom
msujew/semantic-tokens

Conversation

@msujew

@msujew msujew commented Aug 6, 2026

Copy link
Copy Markdown
Member

Closes #95

Provides the infrastructure to implement semantic tokens support for any language. Comes with:

  • Testing infrastructure using AssertSemanticTokens.
  • Options to easily provide custom token types/modifier via ExtendableSemanticTokensLegendProvider.
  • Composable semantic tokens data via SemanticTokensBuilder.
  • Simple semantic token highlighting strategies via TokenHighlightingStrategy and TokenBasedSemanticTokensProvider.
  • A semantic token implementation for the grammar language.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite Current: d971cab Previous: d803417 Ratio
BenchmarkWorkspaceCycle (typefox.dev/fastbelt/examples/statemachine) - MB/s 13 MB/s 5.21 MB/s 2.50

This comment was automatically generated by workflow using github-action-benchmark.

@Lotes Lotes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found 2 things.

Comment thread test/doc_fixture_lsp.go
Comment thread test/doc_fixture_lsp.go
@msujew
msujew force-pushed the msujew/semantic-tokens branch from d971cab to 0e72789 Compare August 19, 2026 13:44
@msujew
msujew requested a review from Lotes August 24, 2026 11:27

@Lotes Lotes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good. Just some little questions about the legend provider.

Comment thread internal/grammar/semantic_tokens_test.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LSP] Semantic token provider

2 participants