When a port is provided to a character-based operation, such as read-char or read, the port’s bytes are read and interpreted as a UTF-8 encoding of characters. Thus, reading a single character may require reading multiple bytes, and a procedure like char-ready? may need to peek several bytes into the stream to determine whether a character is available. In the case of a byte stream that does not correspond to a valid UTF-8 encoding, functions such as read-char may need to peek one byte ahead in the stream to discover that the stream is not a valid encoding.
When an input port produces a sequence of bytes that is not a valid UTF-8 encoding in a character-reading context, then bytes that constitute an invalid sequence are converted to the character #\uFFFD. Specifically, bytes 255 and 254 are always converted to #\uFFFD, bytes in the range 192 to 253 produce #\uFFFD when they are not followed by bytes that form a valid UTF-8 encoding, and bytes in the range 128 to 191 are converted to #\uFFFD when they are not part of a valid encoding that was started by a preceding byte in the range 192 to 253. To put it another way, when reading a sequence of bytes as characters, a minimal set of bytes are changed to the encoding of #\uFFFD so that the entire sequence of bytes is a valid UTF-8 encoding.
See Byte Strings for procedures that facilitate conversions using UTF-8 or other encodings. See also reencode-input-port and reencode-output-port for obtaining a UTF-8-based port from one that uses a different encoding of characters.
A locale captures information about a user’s language-specific interpretation of character sequences. In particular, a locale determines how strings are “alphabetized,” how a lowercase character is converted to an uppercase character, and how strings are compared without regard to case. String operations such as string-ci=? are not sensitive to the current locale, but operations such as string-locale-ci=? (see Strings) produce results consistent with the current locale.
A locale also designates a particular encoding of code-point sequences into byte sequences. Racket generally ignores this aspect of the locale, with a few notable exceptions: command-line arguments passed to Racket as byte strings are converted to character strings using the locale’s encoding; command-line strings passed as byte strings to other processes (through subprocess) are converted to byte strings using the locale’s encoding; environment variables are converted to and from strings using the locale’s encoding; filesystem paths are converted to and from strings (for display purposes) using the locale’s encoding; and, finally, Racket provides functions such as string->bytes/locale to specifically invoke a locale-specific encoding.
A Unix user selects a locale by setting environment variables, such as LC_ALL. On Windows and Mac OS, the operating system provides other mechanisms for setting the locale. Within Racket, the current locale can be changed by setting the current-locale parameter. The locale name within Racket is a string, and the available locale names depend on the platform and its configuration, but the "" locale means the current user’s default locale; on Windows and Mac OS, the encoding for "" is always UTF-8, and locale-sensitive operations use the operating system’s native interface. (In particular, setting the LC_ALL and LC_CTYPE environment variables does not affect the locale "" on Mac OS. Use getenv and current-locale to explicitly install the environment-specified locale, if desired.) Setting the current locale to #f makes locale-sensitive operations locale-insensitive, which means using the Unicode mapping for case operations and using UTF-8 for encoding.
When locale sensitivity is disabled by setting the parameter to #f, strings are compared, etc., in a fully portable manner, which is the same as the standard procedures. Otherwise, strings are interpreted according to a locale setting (in the sense of the C library’s setlocale). The "" locale is always an alias for the current machine’s default locale, and it is the default. The "C" locale is also always available; setting the locale to "C" is the same as disabling locale sensitivity with #f only when string operations are restricted to the first 128 characters. Other locale names are platform-specific.
String or character printing with write is not affected by the parameter, and neither are symbol case or regular expressions (see Regular Expressions).
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4