How can I use ISAPI-Rewrite to rewrite a URL only when rewritten target URL exists?

Or the opposite, how can I rewrite a URL only if it does not exist? eg, intercept 404s.

Here's the context: We are slowly migrating our site from ASP classic to ASP.NET. By slowly, I mean converting 9000+ .asp pages one page at a time to .aspx.

While doing this we want to prevent any broken links as well as avoid losing any rankings in the search engines. That means all our .asp URLs must continue to exist even though they would be served by .aspx pages.

What I would like to be able to do is have ISAPI-Rewrite rule(s) that will rewrite .asp -> .aspx unless the target URL does not actually exist in which case it would just execute and return the .asp.

So if we haven't converted somepage.asp to .aspx yet, the URL would not get rewritten and somepage.asp would be displayed. Once somepage.asp is gone the URL would be rewritten to somepage.aspx which would be executed and returned, but the web browser/search engine would think that it received somepage.asp.

At some point in the future we would set the canonical url in the .aspx pages and let the search engines catch up at their convenience.

Note: I have experimented with using a custom 404 on IIS to check the file system and do a Server.Transfer if the target .aspx page exists, however you cannot transfer from asp classic to asp.net. ( You get "The specified 'Page Language="c#" ' option is unknown or invalid. error '8000d001'" ) The only way this would work is if we did a 301/302 redirect instead of the Server.Transfer and that is undesirable for us at this time. (Management is paranoid and says not an option.)

Solution

I figured it out. As Chris Haas suggested, if I make my 404 handler an .aspx page, then I am able to do a Server.Transfer when the target page exists and if not, just fall through and let the regular 404 status be returned. So something like this:

<%@ Page Language="c#" %>

<%@ Import Namespace="System.IO" %>
<script RunAt="server">

    protected void Page_Load(object sender, EventArgs e)
    {
        String requestedUrl = Request.ServerVariables["HTTP_X_REWRITE_URL"];

        if (requestedUrl.EndsWith(".asp", StringComparison.CurrentCultureIgnoreCase)) {
            String targetUrl = requestedUrl + "x";
            String targetPath = Server.MapPath(targetUrl);

            if (File.Exists(targetPath)) {
                Server.Transfer(targetUrl);
            }
        }
        Response.Status = "404 Not Found";
    }

</script>

I still need to test it with querystrings but so far it handles each case perfectly. When the time comes and management is okay with it, we'll simply stop doing the Server.Transfer and change it to a 301 Moved Permanently. With IIS able to set error pages per directory, we can even be somewhat selective about when we do it.

Incidentally, it looks like ISAPI_Rewrite 3 might have a file checking option that could also have worked but we only have v2 on our server.