How to call a fetch request and wait for it's answer inside a onBeforeRequest in a web extension

Question

I'm trying to write a web extension that stops the requests from a url list provided locally, fetches the URL's response, analyzes it in a certain way and based on the analysis results, blocks or doesn't block the request. Is that even possible? The browser doesn't matter. If it's possible, could you provide some examples?

I tried doing it with Chrome extensions, but it seems like it's not possible. I heard it's possible on mozilla though

Answer 1

I think that this is only possible using the old webRequestBlocking API which Chrome is removing as a part of Manifest v3. Fortunately, Firefox is planning to continue supporting blocking web requests even as they transition to manifest v3 (read more here).

In terms of implementation, I would highly recommend referring to the MDN documentation for webRequest, in particular their section on modifying responses and their documentation for the filterResponseData method.

Mozilla have also provided a great example project that demonstrates how to achieve something very close to what I think you want to do.

Below I've modified their background.js code slightly so it is a little closer to what you want to do:

function listener(details) {
  if (mySpecialUrls.indexOf(details.url) === -1) {
    // Ignore this url, it's not on our list.
    return {};
  }

  let filter = browser.webRequest.filterResponseData(details.requestId);
  let decoder = new TextDecoder("utf-8");
  let encoder = new TextEncoder();

  filter.ondata = event => {
    let str = decoder.decode(event.data, {stream: true});
    // Just change any instance of Example in the HTTP response
    // to WebExtension Example.
    str = str.replace(/Example/g, 'WebExtension Example');
    filter.write(encoder.encode(str));
    filter.disconnect();
  }

  // This is a BlockingResponse object, you can set parameters here to e.g. cancel the request if you want to.
  // See: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/BlockingResponse#type
  return {};
}

browser.webRequest.onBeforeRequest.addListener(
  listener,
  // 'main_frame' means this will only affect requests for the main frame of the browser (e.g. the HTML for a page rather than the images, CSS, etc. that are loaded afterwards). You might want to look into whether you want to expand this.
  {urls: ["*://*/*"], types: ["main_frame"]},
  ["blocking"]
);

Correction: The above example only works properly if the response data fits in one chunk. If it is larger (and you still want to inspect the entirety of the response data), you would need to put all of the data into a buffer, and then work on it once all data has been received. See the document here for more information: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/StreamFilter/ondata#webextension_examples (the code section titled "This example combines all buffers into a single buffer" would be of most interest to you I think).

In terms of using this API to block responses, data is only returned from this URL if you call filter.write(), so if you don't like the response, you can simply not call it (just call filter.close()) and an empty response will be returned. You can also only return part of the full response body by filter.write()ing only the bits that you want to return.