Is there a way to use a tmLanguage grammar to extend vscode integrated markdown extensions syntax highlighting?

Question

I am working on a language extension for vscode. I defined a tmLanguage file and everything works nicely. In the hover feature, using vscode.MarkdownString.appendCodeblock() the markdown is being interpreted correctly and syntax highlighting for my custom language just works out of the box, by doing something like:

const content = new MarkdownString("", true)
content.appendMarkdown("## Examples: \n")
content.appendCodeblock(examples, "lmps")

where examples contains some example code in my custom language and "lmps" is my language identifier. (Example Image Hover)

I wonder if there is a way to achieve the same thing in a webview. I succeeded displaying the content in a webview, compiling the markdownString into html:

async function set_doc_panel_content(panel: DocPanel | undefined, md_content: vscode.MarkdownString) {
        const html: string = await vscode.commands.executeCommand('markdown.api.render', md_content.value) as string;
        panel!.webview.html = html;
    }

So far so good, but in this way, markdown doesn't know my custom language and doesn't do any syntax highlighting. (Example image Webview) Now, I understand that adding language-support to the Markdown extension can be achieved by contributing a markdown-it plugin by returning an object in the activation function.

export function activate(context: vscode.ExtensionContext) {

...

    return {
        extendMarkdownIt(md: any) {
          return md.use(require('markdown-it-emoji'));
        }
}

However, this requires actually developing a dedicated markdown-it plugin if I see that right. I wonder if there is an easier way to integrate my language into the markdown api. Since vscode can obviously do it natively in hover and completion suggestions already. Can I somehow use this functionality in a webview maybe? Alternatively, is there a way to generate a markdown-it plugin from a tmLanguage-file without having to learn markdown-it plugin development in depth? Perhaps someone has pointers to projects where something like this was done?

Answer 1

For anyone having the same problem: There doesn't seem to be a super-easy way. The easiest-to-use package I've found for the job is highlights. However, this package (and others like first-mate) depend on the native module Oniguruma. That package needs to be compiled against the specific version of Electron. That makes it very difficult to publish a vscode extension to the marketplace, because vscode doesn't allow to run this compilation in the package installation.

A solution I've found is to provide a highlight function to markdown-it. The grammar as .plist or .tmLanguage can be read by vscode-textmate for example. This package can be used with vscode-oniguruma. The trick here is to load WASM in order for it to work:

const oniguruma = require('vscode-oniguruma')
const oniguruma_root: string = path.join(env.appRoot, 'node_modules.asar', 'vscode-oniguruma')
const wasm = readFileSync(path.join(oniguruma_root, 'release', 'onig.wasm')).buffer;
const on_wasm = oniguruma.loadWASM(wasm);

Then one can do:

const registry = new vsctm.Registry({
    onigLib: Promise.resolve({
        createOnigScanner: (sources) => new oniguruma.OnigScanner(sources),
                createOnigString: (str) => new oniguruma.OnigString(str)
            }),
            loadGrammar: () => {
                return readJSON2plist(path.join(context.extensionPath, 'syntaxes', 'lmps.tmLanguage.json'))
            .then(data => {
                return vsctm.parseRawGrammar(data)
            }).catch(null)
    }
});
    
const grammar = await registry.loadGrammar('source.lmps')
    
const md = require('markdown-it')(
    {
        html: true,
        linkify: true,
        typographer: true,
        langPrefix: '',
        highlight: function (str: string, lang: string) {
            if (grammar && lang && lang == 'lmps') {
                return tokenize_lmps(str, grammar)
            }
        }
    });
return md

md can then be used to render markdown content:

let html = md.render(md_string)

A working implementation can be found here