Simple tokenization into words and separators. Can ignore end-of-line comments and recognize double/single quoted string literals. It is not a goal to be very customizable, or have very high quality. Higher-quality lexers should be implemented with a lexer generator.
In PMD 7, this replaces AbstractTokenizer, which provided nearly no more functionality.
Note: This class has been called AnyTokenizer in PMD 6.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4