I have a function in Numba
that uses a List(float64[:,::1])
type, this is a dummy function to try out the type, I will do a lot of operations under the for loop. It has an strange behavior, while the to arr lists have the same excatly numba.typeof() signature, one works and the other does not and tells me it is not matching type.
This is definitely a type of object error, but I can figure it out.
this is the file test.npy to load:
https://drive.google.com/file/d/1guAe1C2sKZyy2U2_qXAhMA1v46PfeKnN/view?usp=sharing
Error
raise TypeError(msg)
TypeError: No matching definition for argument type(s) ListType[array(float64, 2d, C)]
Code
import numpy as np
import numba
from numba.typed import List
branches = np.load('test.npy', allow_pickle=True).item()
@numba.njit('List(int64)(List(float64[:, ::1]))')
def not_test(branches):
a_list = []
for branch in branches:
for i in range(len(branch)):
a_list.append(i)
return a_list
# this does not work
arr = []
for branch in branches.features[:2]:
arr.append(np.asarray(branch.geometry.coordinates).copy())
arr = List(arr.copy())
print(numba.typeof(arr))
no_test(arr)
# this works
arr = List([np.array([np.array([7.0,7.3]), np.array([7.4,8.6])])])
print(numba.typeof(arr))
no_test(arr)
For anybody stuck in something trivial like this, turns out the correct signature type is:
@numba.njit('List(int64)(ListType(float64[:, ::1]))')
I do not understand the differences between List
and ListType
and I could not find it on the Numba
official website. I could be really helpfull to have a cheatsheet for types in decorators as is not easy to infer how they should be written just from the available data type found in the arguments of the function. Also, it could really be of great help to have a parser function based on numba.typeof() return, and been able to create the string of the decorator just of off this.
Plus, the conversion to List() is pretty slow, I found a post on Numba GitHub that talks about this this problem. This is the original post improve performance of numba.typed.List constructor with Python list as arg
def convert2(x, dtype=np.float64):
try:
# Try and convert x to a Numpy array. If this succeeds
# then we have reached the end of the nesting-depth.
y = np.asarray(x, dtype=dtype)
except:
# If the conversion to a Numpy array fails, then it can
# be because not all elements of x can be converted to
# the given dtype. There is currently no way to distinguish
# if this is because x is a nested list, or just a list
# of simple elements with incompatible types.
# Recursively call this function on all elements of x.
y = [convert2(x_, dtype=dtype) for x_ in x]
# Convert Python list to Numba list.
y = numba.typed.List(y)
return y
EDIT
I found a super helpfull line to get the type signatures off of a numba function
print(not_test.inspect_types())
#not_test (ListType[array(float64, 2d, C)], array(float64, 1d, A))