Is it possible to speed up this code using Parallel.For loop?
I have List of few millions Integer arrays and need to remove all arrays that doesn't meet compare criteria. How can I use multi-threading to speed-up the loop through the List if I need to remove some items from it?
(This is simple code example, real code has more complex criteria checks inside the loop)
Function NewList(iList As List(Of Integer())) As List(Of Integer())
Dim i As Integer
Dim j As Integer
Dim compareArray As Integer() = {0, 1, 2, 3, 4}
Dim item As Integer()
For i = iList.Count - 1 To 0 Step -1
item = iList.Item(i)
For j = 0 To UBound(compareArray )
If compareArray(j) > 0 AndAlso Not item.Contains(compareArray(j)) Then
iList.RemoveAt(i)
GoTo nextIteration
End If
Next j
nextIteration:
Next i
Return iList
End Function
I would try something like this (untested):
Public Shared Function GetFilteredList(valueList As List(Of Int32()), mustIncludeList As Int32()) As List(Of Int32())
'Check args
If (valueList Is Nothing) Then Throw New ArgumentNullException(NameOf(valueList))
If (mustIncludeList Is Nothing) OrElse (Not mustIncludeList.Any()) Then Return New List(Of Int32())(valueList) 'A new instance to avoid side effects. Could be optimized if not needed
'Normalize args
mustIncludeList = (From e In mustIncludeList.Distinct() Where e > 0 Order By e).ToArray() 'Normalize it, remove duplicates and values < 1 (don't do it 1 million times inside the loop)
'Initialize the filter
Dim myFilter As Func(Of Int32(), Boolean) = Function(valueArray As Int32())
'As all of the values must be included we can skip all arrays that are shorter
If (valueArray Is Nothing) OrElse (valueArray.Length < mustIncludeList.Length) Then Return False
'Try out if the next line gives a better performance (depends a bit the size of the arrays)
Dim myValues As New HashSet(Of Int32)(valueArray)
For Each myMustInclude As Int32 In mustIncludeList
If (Not myValues.Contains(myMustInclude)) Then Return False
Next
Return True
End Function
'Apply filter and return new list
Return valueList.AsParallel().Where(myFilter).ToList()
End Function