`mypy` returning with exit status of 0 even though code is not properly type hinted in gitlab CI/CD pipeline

I have the following .gitlab-ci.yml file:

stages:
  - check-code

before_script:
  - C:\Users\9279\Documents\WindowsPowerShell\profile.ps1
  - conda activate temp

run_tests:
  stage: check-code
  script:
    - pytest test.py

type_checker:
  stage: check-code
  script:
    - (ls -recurse *.py).fullname | foreach-object {echo "`n$_`n";mypy --strict $_}

The check-code stage runs as expected. However the type_checker stage passes in GitLab pipelines even though many of the python scripts in my current directory actually fail the type check.

I suspect this might be because the (ls -recurse *.py).fullname | foreach-object {echo "n$_n";mypy --strict $_} somehow returns a zero exit code. Is this correct? How can I make sure my pipeline fails when the code is improperly typed?

Edit

I think the exit code might not be zero after all. I created a temporary script with the following and ran it under the same conditions:

(ls -recurse *.py).fullname | foreach-object {echo "`n$_`n";mypy --strict $_}

exit $LASTEXITCODE

When I echoed the $LASTEXITCODE it contained a value of 1. So now I am quite confused....

Solution

Presumably, the problem arises because the GitLab CI/CD pipeline executes your PowerShell code via the -Command (-c) parameter of the PowerShell CLI (powershell.exe for Windows PowerShell, pwsh for PowerShell (Core) 7+):

It is the last statement executed that determines the CLI invocation's exit code.
In your case, that translates to the calls to Get-ChildItem and ForEach-Object (%): Only if one or both calls as a whole are considered to have encountered errors - as reflected in the automatic $? variable containing $false - is 1 reported as the exit code; otherwise, it is 0.

In other words: the calls to external programs such as mypy inside the script block passed to ForEach-Object have no impact on the overall exit code.

To change that, keep track of failing external-program calls and use an explicit exit statement as the last statement (code spread across multiple lines for readability - the ; are only needed if you reformat to a single line):

$overallExitCode = 0;
(ls -recurse *.py).fullname | foreach-object { 
  echo "`n$_`n"; mypy --strict $_;
  if ($LASTEXITCODE -ne 0) { $overallExitCode = $LASTEXITCODE }
};
exit $overallExitCode

If you only care about the first failure:

(ls -recurse *.py).fullname | foreach-object { 
  echo "`n$_`n"; mypy --strict $_;
  if ($LASTEXITCODE -ne 0) { break }
};
exit $LASTEXITCODE

If you don't care about relaying the specific non-zero exit code reported by mypy and want to signal any failure with overall exit code 1, you can use the following streamlined formulation (which you've come up with yourself), which relies on the fact that a non-zero exit code reported by calls to external programs also results in $? reflecting $false:

(ls -recurse *.py).fullname | foreach-object {mypy --strict $_; if (-not $?) {exit 1}}

For more information about exit codes in PowerShell, see this answer.