I have been struggling with this problem for almost a week now. I need a vb6 application which is running as a service to open a file. I don't need to do anything with the file, I just need it to open. I have tried using ShellExecute and ShellExecuteEx as well as using CreateProcess to attempt to launch the file from the command line. When none of these implementations worked, I tried instead launching another application (using CreateProcess) with the sole task of opening the file and then closing itself.
These solutions all work when the application is run normally, but Not when it is run as a service. It is extremely important that the application be able to open the file while running as a service, either directly or indirectly, it just needs to be able to trigger it.
I understand that Windows has locked down the ability of services to interact with the desktop since Windows Vista, but I'm sure there must be a way to trigger a file open command from a service. The app I've developed is able to run pg_dump.exe (a backup executable for postgres databases) from the command line with CreateProcess to backup database files, while running as a service. That is why I though launching an exe from the service to indirectly open the file might work. However, for some reason the application runs pg_dump.exe fine but will not run the executable I created. I'm wondering if the exe I created is expecting to have some sort of presence on the desktop and that is why the service doesn't want to start it. I changed the properties of the main form in the secondary exe so that the form would not be visible and wouldn't show up on the taskbar, but something tells me that's not enough.
Here is my CreateProcess code (I didn't write most of this so please excuse my ignorance):
Private Declare Function WaitForSingleObject Lib "KERNEL32" (ByVal _
hHandle As Long, ByVal dwMilliseconds As Long) As Long
Private Declare Function CloseHandle Lib "KERNEL32" _
(ByVal hObject As Long) As Long
Private Declare Function GetExitCodeProcess Lib "KERNEL32" _
(ByVal hProcess As Long, lpExitCode As Long) As Long
'create a new win process.
Private Declare Function CreateProcessA Lib "KERNEL32" (ByVal _
lpApplicationName As String, ByVal lpCommandLine As String, ByVal _
lpProcessAttributes As Long, ByVal lpThreadAttributes As Long, _
ByVal bInheritHandles As Long, ByVal dwCreationFlags As Long, _
ByVal lpEnvironment As Long, ByVal lpCurrentDirectory As String, _
lpStartupInfo As STARTUPINFO, lpProcessInformation As _
PROCESS_INFORMATION) As Long
'used by CreateProcess
Private Type STARTUPINFO
cb As Long
lpReserved As String
lpDesktop As String
lpTitle As String
dwX As Long
dwY As Long
dwXSize As Long
dwYSize As Long
dwXCountChars As Long
dwYCountChars As Long
dwFillAttribute As Long
dwFlags As Long
wShowWindow As Integer
cbReserved2 As Integer
lpReserved2 As Long
hStdInput As Long
hStdOutput As Long
hStdError As Long
End Type
Private Type PROCESS_INFORMATION
hProcess As Long
hThread As Long
dwProcessID As Long
dwThreadID As Long
End Type
Const NORMAL_PRIORITY_CLASS = &H20&
Const INFINITE = -1&
Public Function ExecSynchronousCmd(cmdline As String) As Long
' - Used to force a shelled command to run synchronously (code will
' suspend where this function is called until shelled process
' returns a return value)
' - There is no time out - it will wait forever!!
' - Function returns exit value for shelled process
Dim proc As PROCESS_INFORMATION
Dim start As STARTUPINFO
Dim ret As Long
'Initialize the STARTUPINFO structure:
start.cb = Len(start)
'Start the shelled application:
ret = CreateProcessA(vbNullString, cmdline$, 0&, 0&, 1&, _
NORMAL_PRIORITY_CLASS, 0&, vbNullString, start, proc)
'Wait for the shelled application to finish:
ret = WaitForSingleObject(proc.hProcess, INFINITE)
Call GetExitCodeProcess(proc.hProcess, ret&)
Call CloseHandle(proc.hThread)
Call CloseHandle(proc.hProcess)
ExecSynchronousCmd = ret
End Function
Here is the implementation for running pg_dump.exe which is SUCCESSFUL at running the exe From the service and creating database backup files:
i = ExecSynchronousCmd(Chr$(34) & "C:\Program Files (x86)\PostgreSQL\9.3\bin\pg_dump.exe" & Chr$(34) & _
" -Ft " & _
" -f " & Chr$(34) & tempName & Chr$(34) & _
" -U " & s1 & _
" -h " & s3 & _
" -p " & s4 & _
" " & sDB(0, x))
Here is a similar implementation which tries to run the secondary exe that will attempt to open the file in question:
i = ExecSynchronousCmd(Chr$(34) & "C:\Program Files (x86)\GranDocsNP\GDNPOpener.exe" & Chr$(34))
The above code does not work when the app is run as a service. Why is pg_dump.exe successful in running but my own GDNPOpener.exe is not?
As I stated above, I also tried using ShellExecute and ShellExecuteEx to open the file directly from the service, which didn't work. (I am using ShellExecuteEx within the secondary exe (GDNPOpener.exe) to open the file)
If anyone knows how to fix my exe so that my service will run it, I would greatly appreciate the help! If anyone knows any alternative ways to open a file from a service, that would be appreciated as well, thanks!
You cannot simply call Shellexecute() in the context of a process running as a service. Even if it succeeds (never really tested it), the launched app/document still wouldn't show on the desktop of the logged-on user because those are two different sessions, isolated from each other (unless you're on Windows XP or Windows Server 2003; for those, services & console-session apps run within session 0 but a service still needs to be marked as interactive to interact with the desktop).
See http://blogs.technet.com/b/askperf/archive/2007/04/27/application-compatibility-session-0-isolation.aspx or search "Session 0 Isolation" for more details on this concept.
To address this, you have mainly two options: (**if I missed one, feel free to correct me!)
Use the client/server route;
e.g. You could write a small utility that will run in user space (for example, launched on session start) and communicate with your service over some choice of inter-process communication (IPC; named pipes, mapped memory, etc.). Your service could then ask the utility to open files in the context of the logged-on user (so it would be the utility that calls ShellExecute(), for example).
To know where to start, look for examples of inter-process communications written for VB6.
IMPORTANT: Since the server (your service) and the client (the user-space utility) will communicate across sessions and under different user tokens, pay attention to notes on security descriptors used and access rights required when creating and accessing communication channels (named pipe, etc.), so as not to have your client get "access denied" errors.
WARNING: The usual security warnings regarding the risk of privilege escalation apply here. Since you basically "open a door" into your service, take extra care not to allow any rogue application posing as your client/utility to take control of your service (buffer overrun, etc.) or make it do something it shouldn't do.
Make use of dedicated APIs for starting processes across different sessions;
Depending on (1) whether your service runs under the LocalSystem account or another, administrative account; (2) on whether you want to launch the document/app in the context of the logged-on user or you don't care creating a new session, and (3) whether you have the user's credentials (user+password) or not, a few APIs exist to allow a service to communicate with the desktop or launch an app in the context of another user.
Have a look at the APIs CreateProcessAsUser(), CreateProcessWithLogonW(), WTSQueryUserToken() and related ones, or search for examples using them. Also, the following article could be a good read: Launching an interactive process from Windows Service in Windows Vista and later
Hope this answer your question! At least, it should give you pointers on what to do next.