AnsweredAssumed Answered

Tokenization issues

Question asked by lista on Jun 3, 2015
Hi,

I'm not sure I've got this right, maybe some of you will be able to help.

Anyways, I have a string "aa_aa-aa/aa\aa" which im using to figure out tokenization.
Now, if the property is set up with tokenization to "false", I get the whole string in index, as expected. If the property is set up with tokenization to "true", I get all tokens in the index, as expected (aa aa aa aa aa).
However, if the property is set up with "both", I'd expect to see both values in the index - but that's not the case, the whole string from the "false" case is missing.

Any help will be appreciated.

Outcomes