Search code examples
sphinxthinking-sphinx

Tell Sphinx (or thinking Sphinx) to ignore periods when indexing


I have a strange issue with Sphinx, I am trying to be able to match things like:

L.A. Confidential

So people can search for "LA Confidential" and still get that title. Similarly for "P.M." to be able to match "PM", etc.

I tried putting the period (full stop character U+002E) in the ignore_char list. This didn't make any difference.

So then I tried implementing index_sp =1. This did not solve the issue either.

According to what I understand of the documentation, either of these should have solved this issue correct?

I wonder if it has somthing to do with our math mode which is set to extended2, using Sphinx 2.0.3.

Any help would be greatly appreciated.

Edit, here is my thinking_sphinx.yml config:

Note that the period character (U+002E) is not used anywhere else in my config except in the ignore_chars line.

production:
  mem_limit: 512M
  morphology: stem_en
  wordforms: "db/sphinx/wordforms.txt"
  stopwords: "db/sphinx/stopwords.txt"
  ngram_chars: "U+4E00..U+9FBB, U+3400..U+4DB5, U+20000..U+2A6D6, U+FA0E, U+FA0F, U+FA11, \
  U+FA13, U+FA14, U+FA1F, U+FA21, U+FA23, U+FA24, U+FA27, U+FA28, U+FA29, U+3105..U+312C, \
  U+31A0..U+31B7, U+3041, U+3043, U+3045, U+3047, U+3049, U+304B, U+304D, U+304F, U+3051, \
  U+3053, U+3055, U+3057, U+3059, U+305B, U+305D, U+305F, U+3061, U+3063, U+3066, U+3068, \
  U+306A..U+306F, U+3072, U+3075, U+3078, U+307B, U+307E..U+3083, U+3085, U+3087, \
  U+3089..U+308E, U+3090..U+3093, U+30A1, U+30A3, U+30A5, U+30A7, U+30A9, U+30AD, \
  U+30AF, U+30B3, U+30B5, U+30BB, U+30BD, U+30BF, U+30C1, U+30C3, U+30C4, U+30C6, \
  U+30CA, U+30CB, U+30CD, U+30CE, U+30DE, U+30DF, U+30E1, U+30E2, U+30E3, U+30E5, \/
  U+30E7, U+30EE, U+30F0..U+30F3, U+30F5, U+30F6, U+31F0, U+31F1, U+31F2, U+31F3, \
  U+31F4, U+31F5, U+31F6, U+31F7, U+31F8, U+31F9, U+31FA, U+31FB, U+31FC, U+31FD, \
  U+31FE, U+31FF, U+AC00..U+D7A3, U+1100..U+1159, U+1161..U+11A2, U+11A8..U+11F9, \
  U+A000..U+A48C, U+A492..U+A4C6"
  ngram_len:  1
  ignore_chars: "U+0027, U+2013, U+2014, U+0026, U+002E, ., &"
  (huge char_set entry here for different languages, ommited.)

Solution

  • I ran a test locally with the following in my thinking_sphinx.yml for Thinking Sphinx v3.0.4 and it worked:

    development:
      ignore_chars: U+002E
    

    The same in sphinx.yml for Thinking Sphinx v2.0.14 worked too. I am using Sphinx 2.0.8, but I'll be a little surprised if that's the problem. It's certainly unrelated to your match mode.