Package io.github.qishr.cascara.lang.yaml.processor

Class YamlTokenizer

java.lang.Object
        io.github.qishr.cascara.lang.yaml.processor.AbstractYamlProcessor<io.github.qishr.cascara.lang.yaml.processor.YamlTokenizer>
                io.github.qishr.cascara.lang.yaml.processor.YamlTokenizer

All Implemented Interfaces:
    Tokenizer<YamlToken>

---

public class YamlTokenizer
extends AbstractYamlProcessor<YamlTokenizer>

Processes raw YAML source text into a stream of YamlToken objects.This tokenizer implements a stateful scan that tracks indentation levels to produce structural tokens (INDENT, DEDENT) alongside standard YAML indicators and scalars.

Indentation Rules

• Only spaces are permitted for indentation.
• Tabs encountered in a whitespace context will trigger a RuntimeException.
• A DEDENT must return exactly to a previously established indentation column.

Scalar Constraints

• Plain scalars (unquoted) cannot contain a colon (:) unless it is a valid value indicator followed by whitespace.

Constructor Summary

Constructor	Description
YamlTokenizer()	Default constructor for SPI

Method Summary

Modifier and Type	Method	Description
protected YamlTokenizer	self()
public int	getOffset()
public int	getLine()
public int	getColumn()
public void	open(String text)
public void	open(Reader reader)
public void	open(InputStream is)
public List<YamlToken>	tokenize(byte[] bytes)
public List<YamlToken>	tokenize(String source)
public Set<YamlTokenType>	getTokenTypes()
public YamlToken	nextToken()
public YamlToken	peekToken()
public void	scanBlockScalar(char headerChar)	Scans a folded or literal block scalar

Methods inherited from AbstractYamlProcessor

setReporter, getOptions, getContentType, getServiceProperties, getVersion, getReporter, setOptions

Method Details

self

protected YamlTokenizer self()

Overrides:

AbstractYamlProcessor

---

getOffset

public int getOffset()

Specified By:

Tokenizer

---

getLine

public int getLine()

---

getColumn

public int getColumn()

---

open

public void open(String text)

Specified By:

Tokenizer

---

open

public void open(Reader reader)

Specified By:

Tokenizer

---

open

public void open(InputStream is)

Specified By:

Tokenizer

---

tokenize

public List<YamlToken> tokenize(byte[] bytes)

---

tokenize

public List<YamlToken> tokenize(String source)

Specified By:

Tokenizer

---

getTokenTypes

public Set<YamlTokenType> getTokenTypes()

Specified By:

Tokenizer

---

nextToken

public YamlToken nextToken()

Specified By:

Tokenizer

---

peekToken

public YamlToken peekToken()

Throws:

ParserException

---

scanBlockScalar

public void scanBlockScalar(char headerChar)

Scans a folded or literal block scalar

---