Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

[feat] syntactic semantic tokens #4672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
MangoIV wants to merge 7 commits into haskell:master
base: master
Choose a base branch
Loading
from MangoIV:mangoiv/syntactic-tokens

Conversation

Copy link
Contributor

@MangoIV MangoIV commented Jul 22, 2025 •
edited
Loading

Use the GHC AST and lsp semantic tokens to convince the language server to give highlighting even without any editor highlighting plugins.

Fixes #4501

  • check that this works on all ghc versions
  • check which AST nodes I might have overlooked that are important. (add a tests that checks that all source ranges are covered (except space)

soulomoon reacted with rocket emoji
@MangoIV MangoIV force-pushed the mangoiv/syntactic-tokens branch from e6621bc to 1bde91a Compare July 22, 2025 18:14
@MangoIV MangoIV changed the title (削除) [feat] syntactic syntactic tokens (削除ここまで) (追記) [feat] syntactic semantic tokens (追記ここまで) Jul 22, 2025
Copy link
Collaborator

soulomoon commented Jul 22, 2025 •
edited
Loading

Some keypoints beyond the tokens retrieval:

  1. UTF8 vs UTF16 differences for the token range between haskell and LSP spec. see focusTokenAt.
  2. Consider position mappings so we have correct highlighting even the file could not even parse, see toCurrentRange.
  3. we need to consider delta between each result, see semanticTokensFullDelta. This is to minimize the bandwith between LSP server and clients.

Post it here as a reminder.

Copy link
Contributor Author

MangoIV commented Jul 22, 2025

Thank you. I will go through all of these.
soulomoon reacted with thumbs up emoji 

Use the GHC AST and lsp semantic tokens to convince the language server
to give highlighting even without any editor highlighting plugins.
@MangoIV MangoIV force-pushed the mangoiv/syntactic-tokens branch from 1bde91a to bc4ed9a Compare July 22, 2025 18:47
Copy link
Contributor Author

MangoIV commented Jul 26, 2025

@soulomoon I'm currently struggling a bit with the tests - the ASTs are different enough that making the syntax highlighting consistent between ghc versions is probably a big pain. Should I introduce more goldens for all supported ghc versions?

Copy link
Contributor Author

MangoIV commented Jul 26, 2025

also stylish-haskell's parser seems to be confused about the CPP :/

Copy link
Collaborator

@soulomoon I'm currently struggling a bit with the tests - the ASTs are different enough that making the syntax highlighting consistent between ghc versions is probably a big pain. Should I introduce more goldens for all supported ghc versions?

The only different should be that module name in the head is included since 9.10?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@soulomoon soulomoon soulomoon left review comments

@berberman berberman Awaiting requested review from berberman berberman will be requested when the pull request is marked ready for review berberman is a code owner

@michaelpj michaelpj Awaiting requested review from michaelpj michaelpj will be requested when the pull request is marked ready for review michaelpj is a code owner

@guibou guibou Awaiting requested review from guibou guibou will be requested when the pull request is marked ready for review guibou is a code owner

@fendor fendor Awaiting requested review from fendor fendor will be requested when the pull request is marked ready for review fendor is a code owner

@wz1000 wz1000 Awaiting requested review from wz1000 wz1000 will be requested when the pull request is marked ready for review wz1000 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

the syntax highlighting of a field name used as a function and a field used with -XOverloadedRecordDot should look the same

AltStyle によって変換されたページ (->オリジナル) /