Search code examples
c#regexlinqlistreplace

Remove substring from a list of strings


I have a list of strings that contain banned words. What's an efficient way of checking if a string contains any of the banned words and removing it from the string? At the moment, I have this:

cleaned = String.Join(" ", str.Split().Where(b => !bannedWords.Contains(b,
                            StringComparer.OrdinalIgnoreCase)).ToArray());

This works fine for single banned words, but not for phrases (e.g. more than one word). Any instance of more than one word should also be removed. An alternative I thought of trying is to use the List's Contains method, but that only returns a bool and not an index of the matching word. If I could get an index of the matching word, I could just use String.Replace(bannedWords[i],"");


Solution

  • A simple String.Replace will not work as it will remove word parts. If "sex" is a banned word and you have the word "sextet", which is not banned, you should keep it as is.

    Using Regex you can find whole words and phrases in a text with

    string text = "A sextet is a musical composition for six instruments or voices.".
    string word = "sex";
    var matches = Regex.Matches(text, @"\b" + word + @"\b");
    

    The matches collection will be empty in this case.

    You can use the Regex.Replace method

    foreach (string word in bannedWords) {
        text = Regex.Replace(text, @"\b" + word + @"\b", "")
    }
    

    \b stands for word beginnings and ends. I first had a solution using regex lookaheads and lookbehinds; however, this is not necessary for \b as it is of zero length.

    If your banned words or phrases can contain special characters, it would be safer to escape them with Regex.Escape(word).


    Using @zmbq's idea you could create a Regex pattern once with

    string pattern =
        @"\b(" +
        String.Join(
            "|",
            bannedWords
                .Select(w => Regex.Escape(w))
                .ToArray()) +
         @")\b";
    var regex = new Regex(pattern); // Is compiled by default
    

    and then apply it repeatedly to different texts with

    string result = regex.Replace(text, "");