How to register porter tokenizer in python

Question

How is the default tokenizer 'porter' in fts3 module registered.

one way to register user defined tokenizers is fts.register_tokenizer() but what are its arguments?

since porter being a in built tokenizer does it even need to be registered?

Answer 1

If your Python has the Porter tokenizer compiled in, you do not need to register it.

To register a user-defined tokenizer, you have to call the fts3_tokenizer() SQL function with a pointer to a C structure containing C function pointers; this cannot be easily done from Python.