Search code examples
pythonmachine-learningkerastext-classificationtweets

Keras: tweets classification

Hello dear forum members,

I have a data set of 20 Million randomly collected individual tweets (no two tweets come from the same account). Let me refer to this data set as "general" data set. Also, I have another "specific" data set that includes 100,000 tweets collected from drug (opioid) abusers. Each tweet has at least one tag associated with it, e.g., opioids, addiction, overdose, hydrocodone, etc. (max 25 tags).

My goal is to use the "specific" data set to train the model using Keras and then use it to tag tweets in the "general" data set to identify tweets that might have been written by drug abusers.

Following examples in source1 and source2, I managed to build a simple working version of such model:

from tensorflow.python import keras
import pandas as pd
import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn.preprocessing import LabelBinarizer, LabelEncoder
from sklearn.metrics import confusion_matrix
from tensorflow import keras
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout
from keras.preprocessing import text, sequence
from keras import utils

# load opioid-specific data set, where post is a tweet and tags is a single tag associated with a tweet
# how would I include multiple tags to be used in training?
data = pd.read_csv("filename.csv")
train_size = int(len(data) * .8)
train_posts = data['post'][:train_size]
train_tags = data['tags'][:train_size]
test_posts = data['post'][train_size:]
test_tags = data['tags'][train_size:]

# tokenize tweets
vocab_size = 100000 # what does vocabulary size really mean?
tokenize = text.Tokenizer(num_words=vocab_size)
tokenize.fit_on_texts(train_posts)
x_train = tokenize.texts_to_matrix(train_posts)
x_test = tokenize.texts_to_matrix(test_posts)

# make sure columns are strings
data['post'] = data['post'].astype(str)
data['tags'] = data['tags'].astype(str)

# labeling
# is this where I add more columns with tags for training?
encoder = LabelBinarizer()
encoder.fit(train_tags)
y_train = encoder.transform(train_tags)
y_test = encoder.transform(test_tags)

# model building
batch_size = 32
model = Sequential()
model.add(Dense(512, input_shape=(vocab_size,)))
model.add(Activation('relu'))
num_labels = np.max(y_train) + 1 #what does this +1 really mean?
model.add(Dense(1865))
model.add(Activation('softmax'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(x_train, y_train, batch_size = batch_size, epochs = 5, verbose = 1, validation_split = 0.1)

# test prediction accuracy
score = model.evaluate(x_test, y_test, 
                       batch_size=batch_size, verbose=1)
print('Test score:', score[0])
print('Test accuracy:', score[1])

# make predictions using a test set
for i in range(1000):    
    prediction = model.predict(np.array([x_test[i]]))
text_labels = encoder.classes_ 
predicted_label = text_labels[np.argmax(prediction[0])]
print(test_posts.iloc[i][:50], "...")
print('Actual label:' + test_tags.iloc[i])
print("Predicted label: " + predicted_label)

In order to move forward, I would like to clarify a few things:

  1. Let's say all my training tweets have a single tag -- opioids. Then if I pass the non-tagged tweets through it, isn't it likely that the model simply tags all of them as opioids as it doesn't know anything else? Should I be using a variety of different tweets/tags then for the learning purpose? Perhaps, there are any general guidelines for the selection of the tweets/tags for the training purposes?
  2. How can I add more columns with tags for training (not a single one like is used in the code)?
  3. Once I train the model and achieve appropriate accuracy, how do I pass non-tagged tweets through it to make predictions?
  4. How do I add a confusion matrix?

Any other relevant feedback is also greatly appreciated.

Thanks!

Examples of "general" tweets:

everybody messages me when im in class but never communicates on the weekends like this when im free. feels like that anyway lol.
i woke up late, and now i look like shit. im the type of person who will still be early to whatever, ill just look like i just woke up.

Examples of "specific" tweets:

$2 million grant to educate clinicians who prescribe opioids
early and regular marijuana use is associated with use of other illicit drugs, including opioids
Solution

  • My shot to this is:

    1. Create a new dataset with tweets from general + specific data. Let's say 200k-250K where 100K is you specific data set, rest is general

    2. Take your 25 keywords/tags and write a rule if any one or more than one exists in a tweet it is DA (Drug Abuser) or NDA(Non Drug Abuser). This will be your dependent variable.

    3. Your new dataset will be one column with all the tweets and another column with the dependent variable telling it is DA or NDA

    4. Now divide into train/test and use keras or any other algo. so that it can learn.

    5. Then test the model by plotting Confusion Matrix

    6. Passs you other remaining data set from General to this model and check,

    If their are new words other than 25 which are not in the specific dataset, from the model you built it will still try to intelligently guess the right category by the group of words that come together, tone etc.