Package io.github.qishr.cascara.common.lang.processor

Interface Tokenizer

java.lang.Object
        io.github.qishr.cascara.common.lang.processor.Tokenizer

All Implemented Interfaces:
    Processor

All Known Implementing Classes:
    JsonTokenizer, XmlTokenizer, YamlTokenizer

---

public interface Tokenizer

Method Summary

Modifier and Type	Method	Description
public abstract List	tokenize(String text)
public default Set<? extends TokenType>	getTokenTypes()

Method Details

tokenize

public abstract List tokenize(String text)

---

getTokenTypes

public default Set<? extends TokenType> getTokenTypes()

---