Search code examples
pythonvisual-studio-2012numpynoseptvs

Python tests pass on cmd and fail on VS2012


I'm setting up Python (CPython 3.4, 64-bit) on a new machine (Windows 10). I installed numpy & nose, and ran numpy.test() through the interpreter prompt to make sure everything's working as expected:

Running unit tests for numpy
NumPy version 1.9.0
NumPy is installed in C:\Python34\lib\site-packages\numpy
Python version 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)]
nose version 1.3.4

----------------------------------------------------------------------
Ran 5162 tests in 36.783s

OK (KNOWNFAIL=10, SKIP=20)

So far so good, but when I do the same thing through PTVS on VS2012 (my team uses TFS for source control), there are errors and test failures (below):

Running unit tests for numpy
NumPy version 1.9.0
NumPy is installed in C:\Python34\lib\site-packages\numpy
Python version 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)]
nose version 1.3.4

======================================================================
ERROR: test_basic (test_multiarray.TestResize)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2850, in test_basic
    x.resize((5, 5))
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
ERROR: test_freeform_shape (test_multiarray.TestResize)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2880, in test_freeform_shape
    x.resize(3, 2, 1)
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
ERROR: test_int_shape (test_multiarray.TestResize)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2862, in test_int_shape
    x.resize(3)
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
ERROR: test_obj_obj (test_multiarray.TestResize)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2892, in test_obj_obj
    a.resize(15,)
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
ERROR: test_zeros_appended (test_multiarray.TestResize)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2885, in test_zeros_appended
    x.resize(2, 3, 3)
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
ERROR: Ticket #950
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_regression.py", line 1272, in test_blasdot_uninitialized_memory
    x.resize((m, 0))
ValueError: cannot resize an array that references or is referenced
by another array in this way.  Use the resize function

======================================================================
FAIL: test_blasdot.test_dot_3args
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_blasdot.py", line 54, in test_dot_3args
    assert_equal(sys.getrefcount(r), 2)
  File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: 3
 DESIRED: 2

======================================================================
FAIL: test_1d (test_indexing.TestMultiIndexingAutomated)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 940, in test_1d
    self._check_single_index(a, index)
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 859, in _check_single_index
    self._compare_index_result(arr, index, mimic_get, no_copy)
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 875, in _compare_index_result
    assert_equal(sys.getrefcount(arr), 3)
  File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: 4
 DESIRED: 3

======================================================================
FAIL: test_multidim (test_indexing.TestMultiIndexingAutomated)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 922, in test_multidim
    self._check_multi_index(self.a, index)
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 836, in _check_multi_index
    self._compare_index_result(arr, index, mimic_get, no_copy)
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 875, in _compare_index_result
    assert_equal(sys.getrefcount(arr), 3)
  File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: 4
 DESIRED: 3

======================================================================
FAIL: test_dot_3args (test_multiarray.TestDot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 3285, in test_dot_3args
    assert_equal(sys.getrefcount(r), 2)
  File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: 3
 DESIRED: 2

----------------------------------------------------------------------
Ran 5162 tests in 181.506s

FAILED (KNOWNFAIL=10, SKIP=20, errors=6, failures=4)
Press any key to continue . . .

I ran the script that I wrote in VS through the command line, and the results were the same as running the test through the interpreter directly, so I'm confident there's something amiss in my VS/Python setup rather than the script itself. What could the problem be?


Solution

  • There's nothing amiss in your setup. It's just that numpy tests seem to be fragile in a sense that they don't tolerate debuggers. I'm not sure what's going on there, but it seems that tests involving sys.getrefcount, and semantics that depend on it being something specifically (usually just having a single reference to an array) are what's failing.

    You can repro this when running the interpreter directly by registering your own trace function with sys.settrace, e.g.:

    import sys
    def trace_func(f, e, a):
        return trace_func
    sys.settrace(trace_func)
    
    import numpy
    numpy.test()
    

    Note that you can run the script from VS without debugging it via Debug -> Start Without Debugging. This will give you results identical to running the interpreter directly.