Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/lexer/struct.Lexer.html below:

Lexer in rustc_parse::lexer - Rust

struct Lexer<'psess, 'src> {
    psess: &'psess ParseSess,
    start_pos: BytePos,
    pos: BytePos,
    src: &'src str,
    cursor: Cursor<'src>,
    override_span: Option<Span>,
    nbsp_is_whitespace: bool,
    last_lifetime: Option<Span>,
    token: Token,
    diag_info: TokenTreeDiagInfo,
}

Initial position, read-only.

The absolute offset within the source_map of the current character.

Source text to tokenize.

Cursor for getting lexer tokens.

When a “unknown start of token: \u{a0}” has already been emitted earlier in this file, it’s safe to treat further occurrences of the non-breaking space character as whitespace.

Track the Span for the leading ' of the last lifetime. Used for diagnostics to detect possible typo where " was meant.

The current token.

Returns the next token, paired with a bool indicating if the token was preceded by whitespace.

Detect usages of Unicode codepoints changing the direction of the text on screen and loudly complain about it.

Slice of the source text from start up to but excluding self.pos, meaning the slice does not include the character self.ch.

As symbol_from, with an explicit endpoint.

Slice of the source text spanning from start up to but excluding end.

Slice of the source text spanning from start until the end

Detect guarded string literal syntax

RFC 3593 reserved this syntax for future use. As of Rust 2024, using this syntax produces an error. In earlier editions, however, it only results in an (allowed by default) lint, and is treated as separate tokens.

Note: Most layout information is completely unstable and may even differ between compilations. The only exception is types with certain repr(...) attributes. Please see the Rust Reference's “Type Layout” chapter for details on type layout guarantees.

Size: 232 bytes

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4