gopls: 'Token length and text length do not match' error with semantic tokens enabled

[cpheps]

Type: Bug

With the Go extension active once I type a . for the auto complete menu comes up and I can no longer type. If I select an item in the auto complete menu I get a Token length and text length do not match! error. I noticed it works if I restart vscode until the window loses focus. Once I come back it no longer works. Disabling the Go extension also fixes the issue.

Extension version: 0.46.1
VS Code version: Code - Insiders 1.99.0-insider (ea9b53fbcbcfd76f57d3f9414fb50ed2d05d3215, 2025-03-18T05:04:34.769Z)
OS version: Darwin arm64 24.3.0
Modes:

System Info

Item	Value
CPUs	Apple M3 Pro (12 x 2400)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	4, 3, 3
Memory (System)	36.00GB (3.87GB free)
Process Argv	. --crash-reporter-id 700b5b9d-e567-428a-9b02-27b2e964a403
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256197
pythonvspyt551cf:31249598
vscod805:30301674
py29gd2263:31024238
c4g48928:30535728
2i9eh265:30646982
962ge761:30841072
dsvsc014:30777825
dsvsc015:30821418
h48ei257:31000450
pythontbext0:30879054
cppperfnew:30980852
pythonait:30973460
dwnewjupyter:31046869
nativerepl1:31134653
pythonrstrctxt:31093868
nativeloc1:31118317
e80f6927:31120813
iacca1:31150324
notype1:31143044
dwcopilot:31158714
5b1c1929:31184661
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
9064b325:31222308
copilot_t_ci:31222730
850i5325:31233431
g012b348:31231168
jda6j935:31233686
copilothoveron:31262361
pythoneinst12:31251391
bgtreat:31252123
fh1c7952:31247580
6h6a5794:31261449
31787653:31262186
pythonpulldiagcf:31258011
6gi0g917:31259950

[sebastian-altamirano]

I am experiencing the same issue with version 0.46.1. Additionally, the file view becomes broken when scrolling or editing the file.

I was able to fix this by disabling the ui.semanticTokens setting in VSCode. Since the setting is labeled as experimental, issues like this might be expected.

This is how I am reproducing it:

package schema

import (
	"github.com/go-playground/validator/v10"
)

func RegisterValidators(validate *validator.Validate) {
	validate.RegisterValidation("filepath_if", validateFilepathIf)
}

func validateFilepathIf(fl validator.FieldLevel) bool {
	fl.
}

[findleyr]

CC @h9jiang @pjweinb

This indicates some sort of bug in semantic tokens.
VS Code logic is here:
https://github.com/microsoft/vscode/blob/79abfa3cef98763d4782da39e15b03f2930cedff/src/vs/editor/common/tokens/lineTokens.ts#L107

It would be helpful if someone experiencing this error could capture LSP logs related to a small hello world go project (since the tokens will describe the full file).

https://github.com/golang/vscode-go/blob/master/docs/troubleshooting.md#collect-gopls-information

[findleyr]

FWIW, I cannot reproduce this in my VS Code session, so I must ask more questions.

If someone can reproduce this, I would be curious to know if it reproces with gopls@v0.17.1

go install golang.org/x/tools/gopls@v0.17.1

[h9jiang]

I can reproduce this in vscode-insider version with gopls v0.17.1 and v0.18.1.

Version: 1.99.0-insider (Universal)
Commit: e2eba0acf61d58e502cdbb4b8a4cdec4cb1ab065
Date: 2025-03-19T16:25:54.570Z (3 hrs ago)
Electron: 34.3.2
ElectronBuildId: 11161073
Chromium: 132.0.6834.210
Node.js: 20.18.3
V8: 13.2.152.41-electron.0
OS: Darwin arm64 23.6.0

But I can not reproduce this with vscode regular version

Version: 1.98.2 (Universal)

This error message is recently added in vscode commit.

[h9jiang]

I tried scrolling in vscode but the code start disapprearing, this symptom is similar to an ongoing issue reported very recently here against vscode.

Try to move the cursor, or scroll. The cursor doesn't move and scrolling will probably make the text in the editor disappear in chunks.

Screenshot in my editor

@cpheps could you give it a try using vscode release version and let me know if the same symptom persist?

[findleyr]

Indeed, thanks for investigating @h9jiang.

It looks like that is an upstream bug. Feel free to close as a dupe if you agree.

[sebastian-altamirano]

I was also using insiders yesterday when I commented, I just tried the release version and can't reproduce it.

[h9jiang]

Thank you @sebastian-altamirano for confirmation.

I will mark this as closed as duplication of microsoft/vscode#243923.