visual-studio-codevscode-extensionstmlanguage
Weird identifier problem in TMLanguage syntax highlighting for AS3
I've finished a basic TMLanguage grammar for AS3 only (testing it in Visual Studio Code), but I'm getting a weird highlighting with the snippet:
reviver = JSBridge.toJSFunction("");
For some reason, it does not highlight the string literal inside the call. But, if I do
reviver = Aaa.aaaaaaaaaaaaaaaaa("");
It works. Comparison:
It does not happen only with string literal; it happens with function expression and anything else.
Related parts of my grammar:
# repository: directive (merges directives + statements + expressions)
# Parenthesized
- begin: \(
end: \)
name: expression.group
patterns:
- include: '#directive'
# String literal
- begin: \@"""
end: \"""
name: string.quoted.triple
- begin: \@'''
end: \'''
name: string.quoted.triple
- begin: \"""
end: \"""
name: string.quoted.triple
patterns:
- include: '#stringLiteral'
- begin: \'''
end: \'''
name: string.quoted.triple
patterns:
- include: '#stringLiteral'
- begin: \@"
end: \"
name: string.quoted.double
- begin: \@'
end: \'
name: string.quoted.single
- begin: \"
end: \"
name: string.quoted.double
patterns:
- include: '#stringLiteral'
- begin: \'
end: \'
name: string.quoted.single
patterns:
- include: '#stringLiteral'
# Dot handling
- match: (\.)\s*({{id}})
captures:
1: { name: punctuation }
- begin: (\.)\s*(\<)
end: \>
patterns:
- include: '#directive'
- match: \.\.
name: keyword.operator
- match: \.
name: punctuation
Full grammar:
---
$schema: https://raw.githubusercontent.com/martinring/tmlanguage/master/tmlanguage.json
name: ActionScript 3
scopeName: source.as3
variables:
unicodeEscape: >-
(?:(?x) \\u((\{[A-Fa-f0-9]+\})|[A-Fa-f0-9]{4}))
idStart: >-
(?:(?x) [\p{L}$_\p{Nl}] | {{unicodeEscape}})
idPart: >-
(?:(?x) [\p{L}$_\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}] | {{unicodeEscape}})
id: >-
(?:(?x) {{idStart}} {{idPart}}* )
xmlNameStart: >-
(?:(?x) [\p{L}:_\p{Nl}])
xmlNamePart: >-
(?:(?x) [\p{L}:.\-_\p{Nl}\p{Nd}])
xmlName: >-
(?:(?x) {{xmlNameStart}} {{xmlNamePart}}*)
# Based on https://github.com/microsoft/TypeScript-TmLanguage
decimalNumber: |-
(?<!\$)(?:(?x)(?:
(?:\b[0-9][0-9_]*(\.)[0-9][0-9_]*[eE][+-]?[0-9][0-9_]*([f|F])?\b)| # 1.1E+3
(?:\b[0-9][0-9_]*(\.)[eE][+-]?[0-9][0-9_]*([f|F])?\b)| # 1.E+3
(?:\B(\.)[0-9][0-9_]*[eE][+-]?[0-9][0-9_]*([f|F])?\b)| # .1E+3
(?:\b[0-9][0-9_]*[eE][+-]?[0-9][0-9_]*([f|F])?\b)| # 1E+3
(?:\b[0-9][0-9_]*(\.)[0-9][0-9_]*([f|F])?\b)| # 1.1
(?:\b[0-9][0-9_]*(\.)([f|F])?\B)| # 1.
(?:\B(\.)[0-9][0-9_]*([f|F])?\b)| # .1
(?:\b[0-9][0-9_]*([f|F])?\b(?!\.)) # 1
))(?!\$)
hexLiteral: |-
\b0[Xx][A-Fa-f0-9_]+
binLiteral: |-
\b0[Bb][01_]+
keywordAttribute: >-
(?:(?x) public | private | protected | internal | final | static | dynamic | override | abstract)
definitionKeyword: >-
class|enum|interface|type|namespace|var|const|function
definitionKeywordStop: >-
class|enum|interface|type|namespace|var|const|function|extends|implements|default\s+xml\s+namespace|new|delete|void|typeof|await|yield|instanceof|not\s+in|in|is(\s+not)?|as|if|else|for\s+each|for|return|throw|switch\s+type|switch|case|try|catch|finally
regexOrXMLStartCondition: >-
(?<=^\s*|[:=,;(\[{+\-*/%~&\^|<>]\s*)
patterns:
- include: '#directive'
repository:
directive:
patterns:
- include: '#comment'
# Package definition start
- match: (?x) \b(package) \s* ({{idStart}}{{idPart}}* \s* (\s*\.\s*{{idStart}}{{idPart}}*)*)\b
captures:
1: { name: keyword.other }
2: { name: entity.name.other }
# Package definition start
- match: (?x) \b(package)\b
captures:
1: { name: keyword.other }
# Labeled statement starting as in `labelname:`
- match: >-
(?x) \b({{id}})\s*:(?!:)
captures:
1: { name: entity.name.label }
# Block
- begin: \{
end: \}
beginCaptures:
0: { name: punctuation.definition.block }
endCaptures:
0: { name: punctuation.definition.block }
patterns:
- include: '#directive'
# Annotatable definition attributes
- match: >-
(?x) \b ( (?:\s*(?!{{definitionKeywordStop}}\b){{id}}(?:\s*\.\s*{{id}})*)+ ) \s* (?={{definitionKeyword}})\b
captures:
1: { name: keyword.other }
# Getter or setter
- begin: >-
(?x) \b(function) \s+ (get|set) \s+ ({{id}}) \s* \(
end: \)
captures:
1: { name: keyword.other }
2: { name: keyword.other }
3: { name: entity.name.function }
patterns:
- include: '#parameterList'
# Class
- match: >-
(?x) \b(class) \s+ ({{id}})
captures:
1: { name: keyword.other }
2: { name: entity.name.type.class }
- match: >-
(?x) \b(class)\b
captures:
1: { name: keyword.other }
# Enum
- match: >-
(?x) \b(enum) \s+ ({{id}})
captures:
1: { name: keyword.other }
2: { name: entity.name.type.enum }
# Interface
- match: >-
(?x) \b(interface) \s+ ({{id}})
captures:
1: { name: keyword.other }
2: { name: entity.name.type.interface }
- match: >-
(?x) \b(interface)\b
captures:
1: { name: keyword.other }
# Type
- match: >-
(?x) \b(type) \s+ ({{id}})
captures:
1: { name: keyword.other }
2: { name: entity.name.type }
# Namespace
- match: >-
(?x) \b(namespace) \s+ ({{id}})
captures:
1: { name: keyword.other }
2: { name: entity.name.other }
# `extends` and `implements` keywords
- match: \b(extends|implements)\b
name: keyword.other
# `return` followed by regex
- begin: \b(return)\s*(/)(?![/*])
end: >-
(?x) /([A-Za-z]+\b)?
name: string.regexp
beginCaptures:
1: { name: keyword.control }
patterns:
- include: '#regex'
# `return` followed by XML
- begin: \b(return)\s*(<)({{xmlName}})
end: >-
(?x) (/>) | ( (</) \s* ({{xmlName}}) \s* (>) )
beginCaptures:
1: { name: keyword.control }
2: { name: punctuation.definition.tag.xml }
3: { name: entity.name.tag }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
4: { name: entity.name.tag }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# `return` followed by XML containing interpolated tag names
- begin: \b(return)\s*(<)(\{[^}]*\})
end: >-
(?x) (/>) | ( (</) \s* (\{[^}]*\}) \s* (>) )
beginCaptures:
1: { name: keyword.control }
2: { name: punctuation.definition.tag.xml }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# `break` and `continue`
- match: \b(break|continue)(\s+{{id}})?
captures:
1: { name: keyword.control }
2: { name: entity.name.label }
# Various control keywords
- match: \b(if|else|for\s+each|for|return|throw|switch\s+type|switch|case|try|catch|finally)\b
name: keyword.control
# with
- match: \b(with)\b
name: keyword.other
# default xml namespace =
- match: \b(default\s+xml\s+namespace)\s*(=)
captures:
1: { name: keyword.other }
# import or include
- match: \b(import|include)\b
name: keyword.other
# Variable definition (object destructuring)
- begin: >-
(?x) \b (var|const) \s* \{
end: \}(\s*!)?
captures:
1: { name: keyword.other }
patterns:
- include: '#object'
# Variable definition (array destructuring)
- begin: >-
(?x) \b (var|const) \s* \[
end: \](\s*!)?
captures:
1: { name: keyword.other }
patterns:
- include: '#array'
# Variable definition (non destructuring)
- match: >-
(?x) \b (var|const) \s+ ({{id}})\b
captures:
1: { name: keyword.other }
2: { name: variable }
- match: >-
(?x) \b(var|const)\b
captures:
1: { name: keyword.other }
# Function
- begin: >-
(?x) \b(function) \s+ ({{id}}) \s* \(
end: \)
captures:
1: { name: keyword.other }
2: { name: entity.name.function }
patterns:
- include: '#parameterList'
- begin: >-
(?x) \b(function) \s* \(
end: \)
captures:
1: { name: keyword.other }
patterns:
- include: '#parameterList'
- match: >-
(?x) \b(function)\b
name: keyword.other
# Reserved namespaces
- match: \b(public|private|protected|internal)\b
name: keyword.other
- match: \b(null|false|true)\b
name: constant.language
- match: \b(this|super)\b
name: variable.language
- match: \b(import\s*\.\s*meta)\b
name: variable.language
- begin: \b(new)\s*\<
end: \>
beginCaptures:
1: { name: keyword.other }
patterns:
- include: '#directive'
- match: \b(new|delete|void|typeof|await|yield|instanceof|not\s+in|in|is(\s+not)?|as)\b
name: keyword.other
- match: \b(public|private|protected|internal)\b
name: constant.language
- match: (::)\s*({{id}})
captures:
1: { name: keyword.operator }
- match: |-
::
name: keyword.operator
- match: ({{id}})
- match: |-
{{decimalNumber}}
name: constant.numeric
- match: |-
{{hexLiteral}}
name: constant.numeric
- match: |-
{{binLiteral}}
name: constant.numeric
- begin: \(
end: \)
name: expression.group
patterns:
- include: '#directive'
# String literal
- begin: \@"""
end: \"""
name: string.quoted.triple
- begin: \@'''
end: \'''
name: string.quoted.triple
- begin: \"""
end: \"""
name: string.quoted.triple
patterns:
- include: '#stringLiteral'
- begin: \'''
end: \'''
name: string.quoted.triple
patterns:
- include: '#stringLiteral'
- begin: \@"
end: \"
name: string.quoted.double
- begin: \@'
end: \'
name: string.quoted.single
- begin: \"
end: \"
name: string.quoted.double
patterns:
- include: '#stringLiteral'
- begin: \'
end: \'
name: string.quoted.single
patterns:
- include: '#stringLiteral'
# RegExp
- begin: >-
(?x) {{regexOrXMLStartCondition}} (/)(?![/*])
end: >-
(?x) /([A-Za-z]+\b)?
name: string.regexp
patterns:
- include: '#regex'
# XML tag
- begin: >-
(?x) {{regexOrXMLStartCondition}} (<)({{xmlName}})
end: >-
(?x) (/>) | ( (</) \s* ({{xmlName}}) \s* (>) )
beginCaptures:
1: { name: punctuation.definition.tag.xml }
2: { name: entity.name.tag }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
4: { name: entity.name.tag }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# XML tag containing interpolated tag names
- begin: >-
(?x) {{regexOrXMLStartCondition}} (<)(\{[^}]*\})
end: >-
(?x) (/>) | ( (</) \s* (\{[^}]*\}) \s* (>) )
beginCaptures:
1: { name: punctuation.definition.tag.xml }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# XMLCDATA
- begin: \<\!\[CDATA\[
end: \]\]>
name: string.other
# XMLComment
- begin: \<\!\-\-
end: \-\->
name: comment.block
# XMLPI
- begin: \<\?
end: \?>
name: comment.block
# XMLList
- begin: >-
(?x) (<>)
end: >-
(?x) (</>)
beginCaptures:
1: { name: punctuation.definition.tag.xml }
endCaptures:
1: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlContent'
- match: ;
name: punctuation.terminator.statement
- begin: \[
end: \]
patterns:
- include: '#directive'
- begin: \{
end: \}
patterns:
- include: '#object'
- match: \.\.\.
name: punctuation
- match: >-
(?x) \, | \:
name: punctuation
- match: (\.)\s*({{id}})
captures:
1: { name: punctuation }
- begin: (\.)\s*(\<)
end: \>
patterns:
- include: '#directive'
- match: \.\.
name: keyword.operator
- match: \.
name: punctuation
- match: \@
name: keyword.operator
- match: \.|\?
name: punctuation
- match: (\+\+?|\-\-?|!(==?)?|~|\*\*?|/|%|<=|>=|\<\<?|\>(\>\>?)?|=(==?)?|&&?|\^\^?|\|\|?)
name: keyword.operator
parameterList:
patterns:
- match: >-
(?x) (?<=[,(] \s*) ({{id}})
captures:
1: { name: variable.parameter }
- include: '#directive'
object:
patterns:
- begin: >-
(?x) {{id}} \s* :
end: >-
(?x) [,}]
patterns:
- include: '#directive'
# Shorthand
- match: >-
(?x) ({{id}})
captures:
1: { name: variable }
- include: '#directive'
array:
patterns:
- include: '#directive'
regex:
patterns:
- match: >-
(?x) \\.
xmlTag:
patterns:
- begin: >-
(?x) (?<! </ \s* {{xmlName}} \s*) (>)
beginCaptures:
1: { name: punctuation.definition.tag.xml }
end: (?=\</)
patterns:
- include: '#xmlContent'
- begin: \"
end: \"
name: string.quoted.double
- begin: \'
end: \'
name: string.quoted.single
- match: \{\{xmlName}}
name: entity.name.other.attribute-name
# Interpolation
- begin: \{
end: \}
patterns:
- include: '#directive'
xmlContent:
patterns:
- begin: \<\!\[CDATA\[
end: \]\]>
name: string.other
- begin: \<\!\-\-
end: \-\->
name: comment.block
- begin: \<\?
end: \?>
name: comment.block
- begin: (<)({{xmlName}})
end: >-
(?x) (/>) | ( (</) \s* ({{xmlName}}) \s* (>) )
beginCaptures:
1: { name: punctuation.definition.tag.xml }
2: { name: entity.name.tag }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
4: { name: entity.name.tag }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# XML tag containing interpolated tag names
- begin: (<)(\{[^}]*\})
end: >-
(?x) (/>) | ( (</) \s* (\{[^}]*\}) \s* (>) )
beginCaptures:
1: { name: punctuation.definition.tag.xml }
endCaptures:
1: { name: punctuation.definition.tag.xml }
3: { name: punctuation.definition.tag.xml }
5: { name: punctuation.definition.tag.xml }
patterns:
- include: '#xmlTag'
# Interpolation
- begin: \{
end: \}
patterns:
- include: '#directive'
stringLiteral:
patterns:
- match: \\x[A-Fa-f0-9]{2}
name: constant.character.escape
- match: \\u\{[A-Fa-f0-9]*\}
name: constant.character.escape
- match: \\u[A-Fa-f0-9]{4}
name: constant.character.escape
- match: \\['"\\bfnrtv]
name: constant.character.escape
- match: \\[^0-9]
name: constant.character.escape
- match: \\$
name: constant.character.escape
comment:
patterns:
- match: >-
(?x) (//)(.*)
name: comment.line
captures:
1: { name: punctuation.definition.comment }
- begin: (/\*)
beginCaptures:
1: { name: punctuation.definition.comment }
end: (\*/)
endCaptures:
1: { name: punctuation.definition.comment }
name: comment.block
Solution
Your 6th item in directive causes Catastrophic backtracking
# Annotatable definition attributes
resulting in TextMate giving up on the rest of the line
\b((?:\s*(?!class|enum|interface|type|namespace|var|const|function|extends|implements|default\s+xml\s+namespace|new|delete|void|typeof|await|yield|instanceof|not\s+in|in|is(\s+not)?|as|if|else|for\s+each|for|return|throw|switch\s+type|switch|case|try|catch|finally\b)(?:[\p{L}$_\p{Nl}]|\\u((\{[A-Fa-f0-9]+\})|[A-Fa-f0-9]{4}))(?:[\p{L}$_\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}]|\\u((\{[A-Fa-f0-9]+\})|[A-Fa-f0-9]{4}))*(?:\s*\.\s*(?:[\p{L}$_\p{Nl}]|\\u((\{[A-Fa-f0-9]+\})|[A-Fa-f0-9]{4}))(?:[\p{L}$_\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}]|\\u((\{[A-Fa-f0-9]+\})|[A-Fa-f0-9]{4}))*)*)+)\s*(?=class|enum|interface|type|namespace|var|const|function)\b
I would recommend using atomic groups (?>...) and possessive quantifiers ?+, *+ & ++ where possible
This will disable backtracking
Just remember it can be a double edge sword if you've set up your regex in a way that requires backtracking for it to work
- How to install VS Code in Alpine Linux
- Why is font size in VS Code and VS IDE different, despite having the same values?
- Error "end of central directory record signature not found" while installing ionide-fsharp in vscode
- How do I change VS Code Settings to use JetBrains Mono Font
- How do I use Qt in Visual Studio Code?
- @node_modules keeps showing in Visual Studio Code auto import and suggesions in Nextjs 15
- VS Code removes trailing commas when saving dart files for new projects
- VSCode does not do auto formatting on save - Go
- GitHub Copilot Extension Chat function not appearing in VSCode
- Visual Studio Code cannot detect installed Git
- VS Code Extension: How do you set an order/priority to the extension's settings/configurations?
- #include errors detected in vscode
- How can I use the .NET MAUI extension for VS Code to debug on my iOS/Android devices or simulators?
- How to install dependencies using pip install in VS Code on macOS Catalina
- C++ Segmentation fault on Hello World, cin and cout
- How do I turn off the "Evaluating: plt.show() did not finish after 3.00s seconds." warning in the VsCode debugger?
- VSCode debugger blank page and loading on launch with VS Code 1.76.1 and Chrome 111
- Problem to use Copilot in Vs Code with ArchLinux
- Weird identifier problem in TMLanguage syntax highlighting for AS3
- Visual Studio Code source control not showing changes
- Visual Studio -- Python Preview Error "Failed to launch the Python Process, please validate the path 'python'"
- Run Code vs Run Python File in Terminal for VSCODE
- vs code not working properly when pressing enter
- How to fix "No .NET SDKs were found." error--VSCode
- How to clear the entire terminal (PowerShell)
- Visual Studio Code: Moving the terminal back to the bottom
- VSCode Prettier not formatting PHP
- Make VSCode apply syntax highlighting to .tsx files
- In a VS Code task, how can I use my global node modules?
- Jupyter server crashed. Unable to connect. Error code from jupyter: 1"