Package io.github.qishr.cascara.lang.yaml.processor
Class YamlTokenizer
java.lang.Object
io.github.qishr.cascara.lang.yaml.processor.AbstractYamlProcessor<io.github.qishr.cascara.lang.yaml.processor.YamlTokenizer>
io.github.qishr.cascara.lang.yaml.processor.YamlTokenizer
All Implemented Interfaces:
Tokenizer<YamlToken>
public class YamlTokenizer
extends AbstractYamlProcessor<YamlTokenizer>
Processes raw YAML source text into a stream of YamlToken objects.This tokenizer implements a stateful scan that tracks indentation levels
to produce structural tokens (INDENT, DEDENT) alongside standard
YAML indicators and scalars.
Indentation Rules
- Only spaces are permitted for indentation.
- Tabs encountered in a whitespace context will trigger a RuntimeException.
- A
DEDENTmust return exactly to a previously established indentation column.
Scalar Constraints
- Plain scalars (unquoted) cannot contain a colon (
:) unless it is a valid value indicator followed by whitespace.
Constructor Summary
| Constructor | Description |
|---|---|
| YamlTokenizer() | Default constructor for SPI |
Method Summary
| Modifier and Type | Method | Description |
|---|---|---|
| protected YamlTokenizer | self() | |
| public Set<YamlTokenType> | getTokenTypes() | |
| public List<YamlToken> | tokenize(String source) | Entry point for the tokenization process. |
| public char | peek() | |
| public char | peekNext() | |
| public boolean | isAtEnd() |
Methods inherited from AbstractYamlProcessor
setReporter, getContentType, setOptions, getServiceProperties
Method Details
self
protected YamlTokenizer self()
getTokenTypes
public Set<YamlTokenType> getTokenTypes()
Specified By:
tokenize
public List<YamlToken> tokenize(String source)
Entry point for the tokenization process.
Parameters:
source - The YAML text to process.
Returns:
A list of tokens including structural start/end markers.
Specified By:
peek
public char peek()
peekNext
public char peekNext()
isAtEnd
public boolean isAtEnd()