Launching an external application from within a NPAPI Plugin

Question

I am trying to work out why an NPAPI plugin I have written, which works fine in terms of performing operations triggered via Javascipt calls, cannot use CreateProcess() or ShellExecute() to launch an application from a path specified via the Javascript call.

I can seemingly use either of these methods and they return success, i.e. no error code. But the application just does not launch. I have tried modifying the parameters used when calling them, to create new process group etc. But seemingly with no effect.

I know this may seem like a bit of a security risk, but for the very specific purpose we wish to use it for it shouldn't be a problem.

Using Windows XP Pro SP3, Firefox 3.5 and the following code:

ZeroMemory( &si, sizeof(si) ); 
si.cb = sizeof(si);
ZeroMemory( &pi, sizeof(pi) ); 
if( CreateProcess( NULL, wFileName, NULL, NULL, FALSE, 
                  CREATE_NEW_CONSOLE | CREATE_NEW_PROCESS_GROUP, 
                  NULL, NULL, &si, &pi ) )
{ 
    bSuccess = true; // Close process and thread handles.      
    WaitForSingleObject(pi.hProcess,INFINITE); 
    CloseHandle( pi.hProcess ); 
    CloseHandle( pi.hThread ); 
}

Answer 1

Without any code (snippet) to chew on it is really hard to give valuable hints:

I don't see why this should not work on XP. Are you sure that the CreateProcess()/ShellExecute() calls succeed?

Things to try:

• Does the ShellExecute() call also succeed if you pass an invalid path?

• What happens if you use a hard-coded path (e.g. the path to notepad.exe, a path without spaces, etc.)?

• Check with ProcessMonitor (former filemon) if the executable file of the application you are trying to launch is acceessed.

Maybe the executable doesn't launch because a depending DLL is not found.
Keep in mind that the working/current directory is the directory of the browser executable when launching the application from within a plugin.
Therefore DLLs that are located in the application folder might not be found.