How to use System::Call and MessageBox when calling the C++ dll method from the NSIS script

My requirement is I need to test "If the device is present before we try to disable the Native Power" from the system.

For that I need to call the below function that is there in testutil.dll

BOOL IsTherePower()

Below is the NSIS script to call this function:

Name "PowerTest"

OutFile "PowerTest.exe"

InstallDir $PROGRAMFILES\PowerTest

Section "PowerTest(required)"

  SectionIn RO
  
  DetailPrint "PowerTest"

  ; Set output path to the installation directory. Here is the path C:\Program Files\PowerTest
  SetOutPath $INSTDIR

  ; Give the dll path
  File E:\Code\Source\Validatepower.exe
  File E:\Code\Source\testutil.dll
  File E:\Code\Source\ntutil.dll
  File E:\Code\Source\dlgdll.dll
  
  System::Call "$INSTDIR\testutil.dll::IsTherePower() i.r0"
  Pop $0
  MessageBox MB_OK "Return value = $R0, lasterr = $0"
  IntCmp $R0 1 OkToInstall CancelInstall
  
  CancelInstall:
  Abort "Not allowed to install"
  OkToInstall:
   Do the install

With the above code when i run the application i am getting "Return value=, lasterr = error". I am not sure why i am getting the "Return value" blank (null). Did I miss anything here?

I have written "System::Call" and "MessageBox" but not sure what they are doing. Here I want to know what is "i.r0" from System::Call And also what is "Pop $0"?

Solution

You are using the wrong register. r0 in System syntax is $0, not $R0 (R0 and r10 is $R0). System::Call "$INSTDIR\drvutil.dll::IsUPSPresent() i.r0" puts the INT32 return value in $0 and then you overwrite $0 with the Pop and your stack happened to be empty.

If you need to call GetLastError() then you must append the ?e option:

System::Call "$INSTDIR\drvutil.dll::IsUPSPresent() i.r0 ?e" ; Pushes error code on top of the stack
Pop $1 ; Get error code
DetailPrint "Return=$0 LastError=$1"

?e pushes the last error on the stack and Pop extracts the top item on the stack.

I can confirm that my code works, I tested in on a dummy .DLL. If it does not work for you then System::Call is unable to load the .DLL or find the exported function. The most likely issue is that you have not exported the function correctly in your .DLL.

Inspect your .DLL with Dependency Walker, it is supposed to look like this:

not

You can also try do verify it manually in NSIS:

!include LogicLib.nsh

Section
SetOutPath $InstDir
File drvutil.dll
System::Call 'KERNEL32::LoadLibrary(t "$InstDir\drvutil.dll")p.r8 ?e'
Pop $7
${If} $8 P<> 0
    MessageBox MB_OK 'Successfully loaded "$InstDir\drvutil.dll" @ $8'
    System::Call 'KERNEL32::GetProcAddress(pr8, m "IsUPSPresent")p.r9 ?e'
    Pop $7
    ${If} $9 P<> 0
        MessageBox MB_OK 'Successfully found "IsUPSPresent" @ $9'
    ${Else}
        MessageBox MB_ICONSTOP 'Unable to find "IsUPSPresent", error $7'
    ${EndIf}
    System::Call 'KERNEL32::FreeLibrary(pr8)'
${Else}
    MessageBox MB_ICONSTOP 'Unable to load "$InstDir\drvutil.dll", error $7'
${EndIf}