Elements in preexisting Python list are disappearing and changing into None

Question

I am having difficulty with a python project involving iterating over several lists at the same time. I am analyzing data collected from a serial device and iterating over it to make derivative calculations, find peak values, write raw data and results to a csv file, and more. I am not brand new to python or programming in general, but new enough that I may not see easy solutions immediately, so please bear with me.

Here is a portion of my code for context:

def processData():
 
    x = elapTime
    y = adcData
    dx = []
    dy = []
    peakTime = []
    peakData = []
    
    cutOff = len(adcData) - 6
    
    for i, (n, m) in enumerate(zip(x, y)):
        dx.append(x[i+5] - x[i])
        dy.append(y[i+5] - y[i])        
    
        if i == cutOff:
            break
    
    dx = np.asarray(dx)
    dy = np.asarray(dy)        
    dydx = dy/dx
    der = dydx.tolist()
    
    oneZero = dydx
    oneZero = np.where(oneZero >= 0, 1, oneZero)
    oneZero = np.where(oneZero < 0, 0, oneZero)
    oneZero = oneZero.tolist()
    
    definedPeak = [1,1,1,1,1,1,1,1,1,1,0]
    
    for j, (a, b, c, d) in enumerate(itertools.zip_longest(x, y, der, oneZero)):
        if c > 3:
            chunk = oneZero[j:j+11]
            if chunk == definedPeak:
                peakTime.append(x[j+9])
                peakData.append(y[j+9])
             
    
 with open(fileName, 'w',) as testfile:
        wr = csv.writer(testfile, quoting=csv.QUOTE_ALL)
        wr.writerows(itertools.zip_longest(elapTime,
                                           adcData,
                                           der,
                                           oneZero,
                                           peakTime,
                                           peakData))

And here is the part giving me trouble:

for j, (a, b, c, d) in enumerate(itertools.zip_longest(x, y, der, oneZero)):
    if c > 3:
        chunk = oneZero[j:j+11]
        if chunk == definedPeak:
            peakTime.append(x[j+9])
            peakData.append(y[j+9])

When I run this code on my Raspberry Pi 4 I get this error on the if statement asking if 'c' is greater than 3:

TypeError: '>' not supported between instances of 'NoneType' and 'int'

My expectations for this entire block of code is that it returns to me the raw time and data I collected from my serial device, the derivative values I calculated, the 'oneZero' list where each of the derivative values were changed to a one or zero, and the specific locations of peaks in the data. I have defined what a peak should look like and as long the rolling window I created of the oneZero list matches my predefined peak list, then I will save the time and data value in respective lists.

This code was all stable and functional until I added this line:

if c > 3:

I used a print statement within the corresponding for loop to see what could be causing the error and saw that the elapTime and adcData lists (which have existed before I called the processData() funcion) had at some arbitrary time lost the recorded values they had and were all replaced by 'None.'

Even though removing the if statement returns my program to a stable and functional state, I need a condition like this since it will filter out any noise in the derivatives I am calculating. I appreciate your time and patience with me if I have missed something elementary.

EDIT:

The size of the lists I am using are up to 2000+ entries, and the shortest list I iterate through is only about 5 entries less than the longest one. Until I added the if statement above, all the data I collected was present, but following that line I will notice that I will only have about a few hundred for the same amount of time I run my serial device.

Answer 1

As juanpa.arrivillaga mentioned in the comments, at least one of the iterables x, y, oneZero is longer than der, and zip_longest will fill the shorter iterables with None values to match the length of the longest iterable.

For example

list(zip_longest([1, 2, 3, 4], ['a', 'b']))

will return

[(1, 'a'), (2, 'b'), (3, None), (4, None)]

Bulit-in zip on the other hand cuts off the longer iterables:

list(zip([1, 2, 3, 4], ['a', 'b']))

will return

[(1, 'a'), (2, 'b')]

zip will also work for multiple iterables, just like zip_longest.

---

If you want to skip the c > 3 check for None values you can simply extend it like this:

if c is not None and c > 3:

But as it is, your program will not do anything once the end of der is reached, you will need to define a behaviour for the case c is None. If you want to handle c is None the same as c > 3 you should use

if c is None or c > 3:

If you want the opposite you should just switch zip_longest to zip.