Search code examples
regexnotepad++editpad

How to remove duplicate strings in Notepad++/EditPad?

I looked around and only managed to get this: \b(\w+)\b([\w\W]*)\b\1\b, substitute with: $1$2.

However, it only woks by removing words, like if you have:

word1, word2, word1, word2, word3
*you get:*
word1, word2, word3

What I want is if you have:

"i love you","i love you too", "i love you", "i love you so much"

I should get:

"i love you","i love you too", "i love you so much"
Solution

  • You have a regex that matches a whole word, then any 0+ chars up to the last occurrence of the whole word captured in Group 1.

    You now need a regex where a word boundary should be replaced with ", and the \w pattern must be replaced with [^"] (not "). Additionally, an optional comma and whitespaces can be matched.

    Find what: ("(?!\s*,\s*")[^"]+")(.*)\1,?\s*
    Replace with: $1$2
    . matches newline option must be ON if your dupes may appear across multiple lines.

    The (?!\s*,\s*") negative lookahead will fail all ", " like matches, so as not to remove the field delimiters.

    You will need to click Replace All several times to remove all dupes.

    See an example screen where "he loves you", and "i love you", are removed.

    enter image description here