Create N-gram Tokens
You can access it from 'Add' (Plus) button. "Text Mining..." -> "Create N-Gram Tokens".
- Select a column as sentence id - A column considered as sentence id in a document. If you run do_tokenize beforehand, this can be sentence_id.
- Max # of Tokens to be Concatenated (Optional) - The default is 2. Maximum number of tokens to be connected.
- Text to Concatenate Words (Optional) - The default is "_". Character to be used to connect ngrams.)