Search code examples
pythonkerasdeep-learninglstmrecommendation-engine

Comparing LSTM structure

I'm trying to build an LSTM model according to that picture. I'm a beginner in deep learning particulary WITH RNN structure, so i require your advice to lead me

LSTM Structure

so, for that i'm dealing with a dataframe of 70k users and 12k animes, my dataframe contains :

  • user id

  • user rating

  • anime id

  • genre : a list of tags associated with anime like : action, comedy, school ...etc.

  • users_tags : a list of 15 unique tags for unique user that i built thanks to tfifd method and some text data related to users

My dataframe looks like : 

    anime_id    user_id user_rating name    tags    genre
0   1   234 9.0 Cowboy Bebop    drama , fi , mal action , military , sci fi , ...   Action, Adventure, Comedy, Drama, Sci-Fi, Space
1   1   382 10.0    Cowboy Bebop    life , shiki , tv , thriller , movie short , c...   Action, Adventure, Comedy, Drama, Sci-Fi, Space
2   1   160 9.0 Cowboy Bebop    fantasy , action , supernatural , tv , mystery...   Action, Adventure, Comedy, Drama, Sci-Fi, Space
3   1   341 8.0 Cowboy Bebop    action , school , romance , new , short , mal ...   Action, Adventure, Comedy, Drama, Sci-Fi, Space
4   1   490 9.0 Cowboy Bebop    mal adventure , movie short , school , strange...   Action, Adventure, Comedy, Drama, Sci-Fi,

Here parameters i use for my model : 

#parameters
users = interactions_full_df.user_id.unique()
animes = interactions_full_df.anime_id.unique()
animes_tags = " ".join(interactions_full_df["genre"].unique()).split(",")
n_animes_tags = len(animes_tags)
n_users = len(users)
n_animes =  len(animes)
n_users_tags = 15

I put 100for my "latent dim" for embedding layer.

Here, my attempt to build this model. Can you say if i'm in the right way or not ?

    """ The lstm cell is the concatenation of 3 things : 
--> 1.0  Anime Embedding Vector
--> 2.0 Average of : 
        --> 2.1 Tags embedding vectors associated with the current anime
        --> 2.2 Tags embedding vectors associated with the next anime in a sequence
"""
# 1.0
animes_input = Input(shape=[1],name='Anime')
animes_embedding = Embedding(n_animes + 1, 
                                 latent_dim, 
                                 name='Animes-Embedding')(animes_input)

""" I suppose we need Users embedding to find what's anime chosen by users ??"""
Users_input = Input(shape=[1],name='Users')
Users_embedding = Embedding(n_users + 1, 
                                 latent_dim, 
                                 name='Users-Embeddings')(Users_input)
#2.0
    # 2.1
""" Anime Tags """
animes_tags_input = Input(shape=[1],name='anime_tags')
tags_embedding = Embedding(n_animes_tags + 1, 
                                 latent_dim, 
                                 name='Animes-Tags-embedding')(animes_tags_input)

#2.2 : tags of future anime in a sequence ???
#my input will be a padded sequence of tags used as a string object <<<<<---- 
inp_shape = max_sequence_len - 1
input_len = Input(shape=[inp_shape], name = "future_tags")
sequence_tags_embeddings = Embedding(tags_total_words, latent_dim)(input_len)
sequence_lstm_cells = LSTM(30)(sequence_tags_embeddings)
future_tags_embedding = Dense(latent_dim, activation='softmax')(sequence_lstm_cells) #???????????? i'm not sure at all 
# then average them
averaged_tags = average([tags_embedding, future_tags_embedding])
#then we need to concatenate all of them
merged_cell =  merge([averaged_tags, animes_embedding, Users_embedding])
# My lstm cells is ready : the structure seems to be an Many to One (may be i'm wrong ?)
n_neurons = 100
lstm_cell = LSTM(30, input_shape=(10, 1))(merged_cell)
result = Dense(1, activation='softmax', name = "Recommendation")(lstm_cell)

LSTM_MODEL = Model([animes_input, animes_tags_input, Users_input, input_len], result)
LSTM_MODEL.compile(loss='categorical_crossentropy',
              optimizer='rmsprop')
LSTM_MODEL.summary()

for the part of "future tags" i use a padded sequence of tags like that :

def get_sequence_of_tokens(corpus):
    ## tokenization
    tokenizer.fit_on_texts(corpus)
    total_words = len(tokenizer.word_index) + 1

    ## convert data to sequence of tokens 
    input_sequences = []
    for line in corpus:
        token_list = tokenizer.texts_to_sequences([line])[0]
        for i in range(1, len(token_list)):
            n_gram_sequence = token_list[:i+1]
            input_sequences.append(n_gram_sequence)
    return input_sequences, total_words

def generate_padded_sequences(input_sequences, input_total_words):
    max_sequence_len = max([len(x) for x in tqdm(input_sequences)])
    input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))
    predictors, label = input_sequences[:,:-1],input_sequences[:,-1]
    label = ku.to_categorical(label, num_classes=input_total_words)
    return predictors, label, max_sequence_len

print("create list ..")
train_tags_anime_list = [get_tags_anime(anime_id) for anime_id in tqdm(train["anime_id"])]
test_tags_anime_list = [get_tags_anime(anime_id) for anime_id in tqdm(valid["anime_id"])]
print("cleaning ...")
train_tags_corpus = [clean_text(x) for x in tqdm(train_tags_anime_list)]
valid_tags_corpus = [clean_text(x) for x in tqdm(test_tags_anime_list)]
print("tokenization ..")
train_tags_inp_sequences, train_tags_total_words = get_sequence_of_tokens(train_tags_corpus)
valid_tags_inp_sequences, valid_tags_total_words = get_sequence_of_tokens(valid_tags_corpus)
print("padd sequence")
train_tags_predictors, train_tags_label, train_max_sequence_len = generate_padded_sequences(train_tags_inp_sequences, train_tags_total_words)
valid_tags_predictors, valid_tags_label, valid_max_sequence_len = generate_padded_sequences(valid_tags_inp_sequences, valid_tags_total_words)
Solution

  • You want to build a Stacked LSTM network with multiple features ( what you name parameters is often called features ), this is described in https://machinelearningmastery.com/stacked-long-short-term-memory-networks/ and https://machinelearningmastery.com/use-features-lstm-networks-time-series-forecasting/ and https://datascience.stackexchange.com/questions/17024/rnns-with-multiple-features

    RNNs and so LSTMs are only able to handle sequential data, however this can be expanded by a feature vector with more than one dimensions ( your ensemble of parameters as described in the answer in https://datascience.stackexchange.com/questions/17024/rnns-with-multiple-features )

    The displayed structure of the 6 LSTM cells in 2 layers is a Stacked LSTM network with 2 layers feature_dim = data_dim=6 (or 7) ( number of your parameters / features ) and timesteps=3 ( 2 layers with 3 unit in each layer ) cf section Stacked LSTM for sequence classification in https://keras.io/getting-started/sequential-model-guide/ and How to stack multiple lstm in keras? for keras code.

    Setting the accurate input shape is vital cf Understanding Keras LSTMs, your network is many-to-many case. The shape of the input passed to the LSTM should be in the form (num_samples,timesteps,data_dim) where data_dim is the feature vector or vector of your parameters

    Embedding Layers are for One-Hot encoding cf https://towardsdatascience.com/deep-learning-4-embedding-layers-f9a02d55ac12 for keras code see https://towardsdatascience.com/deep-learning-4-embedding-layers-f9a02d55ac12 and https://keras.io/layers/embeddings/ , possibly you could also use simple label encoding ( http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html , http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html#sklearn.preprocessing.OneHotEncoder )