prerender.io via prerender-node middleware, blacklist hashbang

Question

I'm successfully using prerender.io service via the prerender.io expresss middleware package prerender-node.

I've not noticed a lot of pollution in my cached results from bots/probes/redirects.

I'm trying to implement a blacklist.

prerender-node state I can do this with something similar to:

if (env === 'production') {
  console.log('production environment, prerender.io enabled')
  const prerender = require('prerender-node').set('prerenderToken', '******')
  prerender.blacklisted(
    [
      '#!',
      '/#!/',
      '^/#!/.*'
    ]
  )
  app.use(prerender)
}

In this example I'm just trying to filter out all hahsbang URLs from being cached.

i.e. https://example.com/#!/some/url

But it doesn't seem to be working. After deploying this code, and clearing the cached results. The cache is getting refilled up with results containing '#!'.

Prerender-node README (https://github.com/prerender/prerender-node) states this should be simple regex.
And regex pattern validates here: https://regex101.com/

Answer 1

after getting in touch with prerender authors, they've pointed out that #! is actually being redirected as ?_escaped_fragment_=

therefore:

const prerender = require('prerender-node').set('prerenderToken', '******')
prerender.blacklisted('_escaped_fragment_');
app.use(prerender)

...seems to be working as desired.