How do I use TextClassifier to load a previously generated model?

Question

I have used arcgis learn.text to import TextClassifier in order for creating a Machine learning module. Now I want to use the same model in Streamlitfor creating an interface for re-use and displaying the predictions.

Code for streamlit-app:

import streamlit as st
import os
from arcgis.learn.text import TextClassifier, SequenceToSequence
import pickle

with st.sidebar:
    st.image('https://www.attomdata.com/wp-content/uploads/2021/05/ATTOM-main-full-1000.jpg')
    st.title("AutoAttom")
    st.info("This project application will help in text classification and sequence to sequence labelling")

# Text Classifier Section
st.title("Text Classifier")
user_input = st.text_input(""" """)
if user_input:
    model_folder = "models/text-classifier"
    print(os.listdir(model_folder))
    model_path = os.path.join(model_folder, 'text-classifier.pth')
    model = TextClassifier.load(model_path, name_or_path=model_path)
    st.write(model.predict(user_input))

Now, whenever I am running this code I am getting the following error:

NotADirectoryError: [WinError 267] The directory name is invalid: 'models\\text-classifier\\text-classifier.pth'
Traceback:
File "d:\python projects\attom\text2seq\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
File "D:\Python Projects\ATTOM\app.py", line 18, in <module>
    model = TextClassifier.load(model_path, name_or_path=model_path)
File "d:\python projects\attom\text2seq\lib\site-packages\arcgis\learn\text\_text_classifier.py", line 294, in load
    name_or_path = str(_get_emd_path(name_or_path))
File "d:\python projects\attom\text2seq\lib\site-packages\arcgis\learn\_utils\common.py", line 460, in _get_emd_path
    list_files = get_files(emd_path, extensions=['.emd'])
File "d:\python projects\attom\text2seq\lib\site-packages\fastai\data_block.py", line 44, in get_files
    f = [o.name for o in os.scandir(path) if o.is_file()]

Now, I have checked my folder structure numerous times. And it is correct as follows:

D:\PYTHON PROJECTS\ATTOM\MODELS\TEXT-CLASSIFIER
│   model_metrics.html
│   text-classifier.dlpk
│   text-classifier.emd
│   text-classifier.pth
│
└───ModelCharacteristics
        loss_graph.png
        sample_results.html

How do i reuse the model that i have generated?

Answer 1

Solved it!

Please use path to the dlpk/emd file inside the text-classifier folder while using the TextClassifier.load() function.

In my case, to make prediction from an already existing pretrained model I had to use from_model() like:

st.title("Text Classifier")
user_input = st.text_input(""" """)
if user_input:
    emd_path = os.path.join('models', 'text-classifier', "text-classifier.emd")
    model = TextClassifier.from_model(emd_path)
    st.write(model.predict(user_input))