# Package weka.core.tokenizers

- Class Summary
  Class | Description
  --- | ---
  AlphabeticTokenizer | Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.
  CharacterDelimitedTokenizer | Abstract superclass for tokenizers that take characters as delimiters.
  CharacterNGramTokenizer | Splits a string into an n-gram with min and max grams.
  NGramTokenizer | Splits a string into an n-gram with min and max grams.
  Tokenizer | A superclass for all tokenizer algorithms.
  WordTokenizer | A simple tokenizer that is using the java.util.StringTokenizer class to tokenize the strings.