How to make all combination of features?

Question

I am trying to improve my machine learning system by combining my features. I have written some code to generate all of the combinations of my features. However, it seems very inefficient. Is there a way to do this faster?

allcomb=[]
for i in range(pow(2,len(features))):
    com=[]
    for j in (range(len(features))):
        if((i&(1<<j))==1):
            com.append(features[j])
    allcomb.append(com)

Answer 1

First, check out Python's itertools package; that will help make your code more efficient and readable. You can do most of the combinatoric things you want with that package.

One great advantage is that you get a generator for the sequence of combinations, so you don't have to store them all at once in a list.

Also, do note that you have an inherently slow process. If you want all possible combinations of 16 features, that's 2**16 times through any processing loop. You're already at 64K iterations, and that will double for every added feature.

Can you consider doing a PCA (Principal Component Analysis) and feature reduction before you get into any heavy processing?