Search code examples
python-3.xduplicatesdatalist

Removing duplicate elements in a list with sets of arrays in python


I have a list[(position,id)] from the user based on the ids it chooses which I collect in data

data[(position,id)] = [(1,0),(2,0),(7,3),(8,6),(3,11),(3,11),(4,0),(5,1),(5,1),(6,2),(9,5),(10,7),(15,0),(16,10),(11,0),(11,1),(12,15),(13,8),(13,8),(13,9),(14,9)]

There are some duplicate elements in the list which I sorted out using *set(list) command

listdata = data
res=[]
res = [*set(listdata)]
print(res)

I get the res list as follows:

output: res = [(11, 1), (13, 8), (6, 2), (4, 0), (16, 10), (11, 0), (2, 0), (5, 1), (10, 7), (7, 3), (9, 5), (15, 0), (13, 9), (14, 9), (8, 6), (12, 15), (1, 0), (3, 11)]

But what I want is only 16 elements in the list on first come basis with positions 1 to 16, if you see here I have got 2 entries with position 11 [(11,1),(11,0)] and position 13[(13,8),(13,9)]

Required output: res=[(11, 1), (13, 8), (6, 2), (4, 0), (16, 10), (2, 0), (5, 1), (10, 7), (7, 3), (9, 5), (15, 0), (14, 9), (8, 6), (12, 15), (1, 0), (3, 11)]

Can anyone suggest alternate solution?


Solution

  • you can use a map to record every position only once

    data = [(1,0),(2,0),(7,3),(8,6),(3,11),(3,11),(4,0),(5,1),(5,1),(6,2),(9,5),(10,7),(15,0),(16,10),(11,0),(11,1),(12,15),(13,8),(13,8),(13,9),(14,9)]
    bucket = {}
    for d in data:
        bucket[d[0]] = d[1]
    print(list(bucket.items()))
    

    output is:

    [(1, 0), (2, 0), (7, 3), (8, 6), (3, 11), (4, 0), (5, 1), (6, 2), (9, 5), (10, 7), (15, 0), (16, 10), (11, 1), (12, 15), (13, 9), (14, 9)]
    

    update: I am sorry I missed the description: "But what I want is only 16 elements in the list on first come basis", my answer above keeps the later came value, if you want to keep the first came one, do something like this:

    data = [(1,0),(2,0),(7,3),(8,6),(3,11),(3,11),(4,0),(5,1),(5,1),(6,2),(9,5),(10,7),(15,0),(16,10),(11,0),(11,1),(12,15),(13,8),(13,8),(13,9),(14,9)]
    bucket = {}
    for d in data:
        if d[0] not in bucket:
            # if you never seen it, keep it, else ingore it. 
            bucket[d[0]] = d[1]
    print(list(bucket.items()))