coveralls fails on AppVeyor in some builds but not in others

Question

I'm trying to make pytest publish coverage reports from AppVeyor. I've added the repository's encrypted token as explained here. Here's the appveyor.yml file:

environment:
  COVERALLS_REPO_TOKEN:
    secure: 2NJ5Ct55cHJ9WEg3xbSqCuv0rdgzzb6pnzOIG5OkMbTndw3wOBrXntWFoQrXiMFi
    # this is pytest's token in coveralls.io, encrypted
    # using pytestbot account as detailed here:
    # https://www.appveyor.com/docs/build-configuration#secure-variables

install:
  - echo Installed Pythons
  - dir c:\Python*

  # install pypy using choco (redirect to a file and write to console in case
  # choco install returns non-zero, because choco install python.pypy is too
  # noisy)
  - choco install python.pypy > pypy-inst.log 2>&1 || (type pypy-inst.log & exit /b 1)
  - set PATH=C:\tools\pypy\pypy;%PATH% # so tox can find pypy
  - echo PyPy installed
  - pypy --version

  - C:\Python35\python -m pip install tox

build: false  # Not a C# project, build stuff at the test step instead.

test_script:
  # - C:\Python35\python -m tox  
  - C:\Python35\python -m tox -e coveralls  # coveralls is not in tox's envlist

Now the weird thing is, some builds pass others do not. The GH panel for continuous integration on the PR shows that the build for the continuous-integration/appveyor/branch passes and publishes coverage as expected, but fails for the continuous-integration/appveyor/pr build with this message:

You have to provide either repo_token in .coveralls.yml, or launch via Travis or CircleCI
ERROR: InvocationError: 'C:\\projects\\pytest\\.tox\\coveralls\\Scripts\\coveralls.EXE' 

Does anyone have any idea what's happening?

Answer 1

Secure variables are not set during PR builds for security reasons.