How to create meaningful column value pair lists from a string?

Question

I am trying to categorize columns and values (column=value) meaningfully from an input string using Python dictionaries.

input_string = "the status is processing and product subtypes are HL year 30 ARM and applicant name is Ryan"

I have created dictionaries of key value pairs. In the first scenario, the key is the column name. The value represents the lowest index of key found in input_string.

Here is the dictionary of column names:

 dict_columns = {'status': 4, 'product subtypes': 29, 'applicant name': 69}

In the above dictionary, 'status' has the lowest index of 4 in the input_string.

Similarly, here is the dictionary of values:

dict_values = {'processing': 14, 'hl': 50, 'year': 53, '30': 58, 'arm': 61, 'ryan': 87}

The question is:
How to get the expected ouput as:

list_parsed_values = ['processing', 'hl year 30 arm', 'ryan']

and the (optional) corresponding list of columns as:

list_parsed_columns = ['status', 'product subtypes', 'applicant name']

How to clearly distinguish the values in a list?

Answer 1

Check the following approach:

• Build the regex to remove irrelevant words from the results based on the English nltk stopword list
• Build the regex to split the text with using the dict_columns keys
• After splitting, zip the resulting list into a tuple list
• Remove the irrelevant words from the values and strip the whitespace

Here is the code I have come so far:

import nltk, re
s = "the status is processing and product subtypes are HL year 30 ARM and applicant name is Ryan"
dict_columns = {'status': 4, 'product subtypes': 29, 'applicant name': 69}
dict_values = {'processing': 14, 'hl': 50, 'year': 53, '30': 58, 'arm': 61, 'ryan': 87}
# Build the regex to remove irrelevant words from the results
rx_stopwords = r"\b(?:{})\b".format("|".join([x for x in nltk.corpus.stopwords.words("English")]))
# Build the regex to split the text with using the dict_columns keys
rx_split = r"\b({})\b".format("|".join([x for x in dict_columns]))
chunks = re.split(rx_split, s)
# After splitting, zip the resulting list into a tuple list
it = iter(chunks[1:])
lst = list(zip(it, it))
# Remove the irrelevant words from the values and trim them (this can be further enhanced
res = [(x, re.sub(rx_stopwords, "", y).strip()) for x, y in lst]
# =>
#   [('status', 'processing'), ('product subtypes', 'HL year 30 ARM'), ('applicant name', 'Ryan')]
# It can be cast to a dictionary
dict(res)
# => 
#   {'product subtypes': 'HL year 30 ARM', 'status': 'processing', 'applicant name': 'Ryan'}