I have this dataset:
import pandas as pd
import itertools
A = ['A','B','C']
M = ['1','2','3']
F = ['plus','minus','square']
df = pd.DataFrame(list(itertools.product(A,M,F)), columns=['A','M','F'])
print(df)
The example output is like this:
A M F
0 A 1 plus
1 A 1 minus
2 A 1 square
3 A 2 plus
4 A 2 minus
5 A 2 square
I want to pairwise comparison (jaccard similarity) of each row from this data frame, for example, comparing
A 1 plus and A 2 square and get the similarity value between those both set.
I have wrote a jaccard function:
def jaccard(a, b):
c = a.intersection(b)
return float(len(c)) / (len(a) + len(b) - len(c))
Which is only work on set because I used intersection
I want the output like this (this expected result value is just random number):
0 1 2 3 45
0 1.00 0.43 0.61 0.55 0.46
1 0.43 1.00 0.52 0.56 0.49
2 0.61 0.52 1.00 0.48 0.53
3 0.55 0.56 0.48 1.00 0.49
45 0.46 0.49 0.53 0.49 1.00
What is the best way to get the result of pairwise metrics?
Thank you,
A full implementation of what you want can be found here:
series_set = df.apply(frozenset, axis=1)
new_df = series_set.apply(lambda a: series_set.apply(lambda b: jaccard(a,b)))