I am curious what would be an efficient way of uniquifying such data objects:
testdata = [ ['9034968', 'ETH'], ['14160113', 'ETH'], ['9034968', 'ETH'], ['11111', 'NOT'], ['9555269', 'NOT'], ['15724032', 'ETH'], ['15481740', 'ETH'], ['15481757', 'ETH'], ['15481724', 'ETH'], ['10307528', 'ETH'], ['15481757', 'ETH'], ['15481724', 'ETH'], ['15481740', 'ETH'], ['15379365', 'ETH'], ['11111', 'NOT'], ['9555269', 'NOT'], ['15379365', 'ETH']
]
For each data pair, left numeric string PLUS the type at the right tells the uniqueness of a data element. The returned value should be a list of lists, the same as testdata
, but only unique values should be kept.
You can use a set:
unique_data = [list(x) for x in set(tuple(x) for x in testdata)]
You can also see this page which benchmarks a variety of methods that either preserve or don't preserve order.