Skip to content

Commit 5a26f77

Browse files
slevithanslevithan
authored
feat: update js engine and grammars (#869)
1 parent c489859 commit 5a26f77

File tree

8 files changed

+2613
-521
lines changed

8 files changed

+2613
-521
lines changed

‎docs/references/engine-js-compat.md

+30-30
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Compatibility reference of all built-in grammars with the [JavaScript RegExp engine](/guide/regex-engines#javascript-regexp-engine-experimental).
44

5-
> Generated on Monday, December 16, 2024
5+
> Generated on Thursday, December 19, 2024
66
>
77
> Version `1.24.2`
88
>
@@ -13,8 +13,8 @@ Compatibility reference of all built-in grammars with the [JavaScript RegExp eng
1313
| | Count |
1414
| :-------------- | --------------------------: |
1515
| Total Languages | 218 |
16-
| Supported | [199](#supported-languages) |
17-
| Mismatched | [13](#mismatched-languages) |
16+
| Supported | [206](#supported-languages) |
17+
| Mismatched | [6](#mismatched-languages) |
1818
| Unsupported | [6](#unsupported-languages) |
1919

2020
## Supported Languages
@@ -39,6 +39,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
3939
| ballerina | ✅ OK | 230 | - | |
4040
| bash | ✅ OK | 148 | - | |
4141
| bat | ✅ OK | 58 | - | |
42+
| beancount | ✅ OK | 39 | - | |
4243
| berry | ✅ OK | 18 | - | |
4344
| bibtex | ✅ OK | 19 | - | |
4445
| bicep | ✅ OK | 27 | - | |
@@ -74,7 +75,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
7475
| elixir | ✅ OK | 708 | - | |
7576
| elm | ✅ OK | 244 | - | |
7677
| emacs-lisp | ✅ OK | 153 | - | 22 |
77-
| erb | ✅ OK | 1312 | - | |
78+
| erb | ✅ OK | 1786 | - | |
7879
| erlang | ✅ OK | 147 | - | |
7980
| fennel | ✅ OK | 31 | - | |
8081
| fish | ✅ OK | 25 | - | |
@@ -83,8 +84,8 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
8384
| fortran-free-form | ✅ OK | 328 | - | |
8485
| fsharp | ✅ OK | 239 | - | |
8586
| fsl | ✅ OK | 30 | - | |
86-
| gdresource | ✅ OK | 159 | - | |
87-
| gdscript | ✅ OK | 95 | - | |
87+
| gdresource | ✅ OK | 158 | - | |
88+
| gdscript | ✅ OK | 94 | - | |
8889
| gdshader | ✅ OK | 39 | - | |
8990
| genie | ✅ OK | 20 | - | |
9091
| gherkin | ✅ OK | 19 | - | |
@@ -98,6 +99,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
9899
| groovy | ✅ OK | 133 | - | |
99100
| haml | ✅ OK | 562 | - | |
100101
| handlebars | ✅ OK | 699 | - | |
102+
| haskell | ✅ OK | 157 | - | |
101103
| haxe | ✅ OK | 174 | - | |
102104
| hcl | ✅ OK | 67 | - | |
103105
| hjson | ✅ OK | 57 | - | |
@@ -120,6 +122,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
120122
| jssm | ✅ OK | 30 | - | |
121123
| jsx | ✅ OK | 376 | - | |
122124
| julia | ✅ OK | 1168 | - | |
125+
| kotlin | ✅ OK | 58 | - | |
123126
| latex | ✅ OK | 283 | - | |
124127
| lean | ✅ OK | 32 | - | |
125128
| less | ✅ OK | 280 | - | |
@@ -131,6 +134,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
131134
| make | ✅ OK | 51 | - | |
132135
| marko | ✅ OK | 926 | - | |
133136
| matlab | ✅ OK | 88 | - | |
137+
| mdc | ✅ OK | 783 | - | |
134138
| mdx | ✅ OK | 197 | - | |
135139
| mermaid | ✅ OK | 134 | - | |
136140
| mipsasm | ✅ OK | 17 | - | |
@@ -146,26 +150,29 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
146150
| ocaml | ✅ OK | 178 | - | |
147151
| pascal | ✅ OK | 23 | - | |
148152
| perl | ✅ OK | 941 | - | |
153+
| php | ✅ OK | 1131 | - | |
149154
| plsql | ✅ OK | 45 | - | |
150155
| postcss | ✅ OK | 49 | - | |
151156
| powerquery | ✅ OK | 30 | - | |
152157
| powershell | ✅ OK | 88 | - | |
153-
| prisma | ✅ OK | 26 | - | |
158+
| prisma | ✅ OK | 28 | - | |
154159
| prolog | ✅ OK | 26 | - | |
155160
| proto | ✅ OK | 33 | - | |
161+
| pug | ✅ OK | 686 | - | |
156162
| puppet | ✅ OK | 61 | - | |
157163
| python | ✅ OK | 218 | - | |
158164
| qml | ✅ OK | 408 | - | |
159165
| qmldir | ✅ OK | 7 | - | |
160166
| qss | ✅ OK | 31 | - | |
161167
| r | ✅ OK | 71 | - | |
162-
| racket | ✅ OK | 69 | - | 8 |
168+
| racket | ✅ OK | 69 | - | |
163169
| raku | ✅ OK | 52 | - | |
164170
| reg | ✅ OK | 9 | - | |
165171
| regexp | ✅ OK | 34 | - | |
166172
| rel | ✅ OK | 17 | - | |
167173
| riscv | ✅ OK | 36 | - | |
168-
| ruby | ✅ OK | 1307 | - | |
174+
| rst | ✅ OK | 2031 | - | |
175+
| ruby | ✅ OK | 1781 | - | |
169176
| rust | ✅ OK | 89 | - | |
170177
| sas | ✅ OK | 101 | - | |
171178
| sass | ✅ OK | 69 | - | |
@@ -199,7 +206,7 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
199206
| tsv | ✅ OK | 1 | - | |
200207
| tsx | ✅ OK | 376 | - | |
201208
| turtle | ✅ OK | 15 | - | |
202-
| twig | ✅ OK | 1984 | - | |
209+
| twig | ✅ OK | 2426 | - | |
203210
| typescript | ✅ OK | 363 | - | |
204211
| typespec | ✅ OK | 80 | - | |
205212
| typst | ✅ OK | 78 | - | |
@@ -235,31 +242,24 @@ In some edge cases, it's not guaranteed that the highlighting will be 100% the s
235242

236243
Languages that do not throw with the JavaScript RegExp engine, but will produce different results than the WASM engine. Please use with caution.
237244

238-
| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
239-
| --------- | :----------------------------------------------------------------------------- | ----------------: | --------------: | ---: |
240-
| apex | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=apex) | 187 | - | 236 |
241-
| beancount | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=beancount) | 39 | - | 171 |
242-
| cpp | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=cpp) | 512 | - | 8 |
243-
| haskell | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=haskell) | 157 | - | 39 |
244-
| kotlin | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kotlin) | 58 | - | 1953 |
245-
| kusto | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kusto) | 60 | - | 40 |
246-
| markdown | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=markdown) | 118 | - | 915 |
247-
| mdc | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=mdc) | 783 | - | 702 |
248-
| nginx | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=nginx) | 378 | - | 4 |
249-
| php | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=php) | 1131 | - | 605 |
250-
| po | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=po) | 23 | - | 423 |
251-
| pug | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=pug) | 686 | - | 164 |
252-
| rst | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=rst) | 1857 | - | 62 |
245+
| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
246+
| -------- | :---------------------------------------------------------------------------- | ----------------: | --------------: | ---: |
247+
| apex | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=apex) | 187 | - | 33 |
248+
| cpp | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=cpp) | 512 | - | 8 |
249+
| kusto | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=kusto) | 60 | - | 40 |
250+
| markdown | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=markdown) | 118 | - | 100 |
251+
| nginx | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=nginx) | 378 | - | 122 |
252+
| po | [🚧 Mismatch](https://textmate-grammars-themes.netlify.app/?grammar=po) | 23 | - | 114 |
253253

254254
## Unsupported Languages
255255

256-
Languages that throw with the JavaScript RegExp engine, either because they contain syntax we can't polyfill yet or because the grammar contains an invalid Oniguruma regex (that would also fail when using the WASM engine, but silently). You can try these languages with the `forgiving` option to skip errors, but usually it's best to use the Oniguruma engine instead.
256+
Languages that throw with the JavaScript RegExp engine, either because they contain syntax we can't polyfill yet or because the grammar contains an invalid Oniguruma regex (that would also fail when using the WASM engine, but silently). You can try these languages with the `forgiving` option to skip errors.
257257

258258
| Language | Highlight Match | Patterns Parsable | Patterns Failed | Diff |
259259
| ---------- | :-------------- | ----------------: | --------------: | ---: |
260260
| ada | ✅ OK | 201 | 1 | |
261261
| hack | ❌ Error | 947 | 1 | 114 |
262-
| purescript | ❌ Error | 72 | 1 | 42 |
263-
| csharp | ❌ Error | 306 | 3 | 204 |
264-
| swift | ❌ Error | 326 | 3 | 40 |
265-
| razor | ❌ Error | 952 | 5 | 22 |
262+
| purescript | ❌ Error | 72 | 1 | |
263+
| csharp | ❌ Error | 306 | 3 | 148 |
264+
| swift | ❌ Error | 326 | 3 | 8 |
265+
| razor | ❌ Error | 952 | 5 | 14 |

‎packages/engine-javascript/src/index.ts

+5-3
Original file line numberDiff line numberDiff line change
@@ -57,13 +57,15 @@ export function defaultJavaScriptRegexConstructor(pattern: string, options?: Oni
5757
global: true,
5858
hasIndices: true,
5959
rules: {
60+
// Needed since TextMate grammars merge backrefs across patterns
6061
allowOrphanBackrefs: true,
61-
// Dropping unhandled `\G` anchors without erroring allows more grammars to pass but also
62-
// allows some false positive and negative matches. It's mostly because this is true that
63-
// the JS engine is marked as experimental
62+
// Removing `\G` anchors in cases when they're not supported for emulation allows
63+
// supporting more grammars, but also allows some mismatches
6464
allowUnhandledGAnchors: true,
6565
// Improves search performance for generated regexes
6666
asciiWordBoundaries: true,
67+
// Follow `vscode-oniguruma` which enables this Oniguruma option by default
68+
captureGroup: true,
6769
},
6870
...options,
6971
},

0 commit comments

Comments
 (0)