Part 14: Text

Text has been in every program since Hello, World, and so far it has been simple: a literal in quotes, printed with {}. This part looks underneath. A literal turns out to be a slice of UTF-8 bytes, which is why its length can surprise you; the Text package adds views that promise their bytes are text, Strings that own their bytes, and builders that grow them. Then the part turns outwards — Unicode's three ways of counting characters, laying out columns of numbers, rendering text you keep, parsing numbers back out of it, and reading lines a person types. It ends with whole texts: cutting them into lines and joining pieces back together, and searching them with regular expressions.

What you will learn

• What a string literal is — a read-only char8[..] of UTF-8 bytes — and the escapes it can hold.
• The three encodings, c8, c16 and c32, and why lengths count code units rather than characters.
• Borrowing text with StringView, owning it with String, and building it with StringBuilder.
• Checking and decoding UTF-8, and telling bytes, scalars and graphemes apart.
• Changing case correctly beyond ASCII, where one letter can become two.
• Placeholder specs for width, alignment, precision, bases, zeros and signs.
• Rendering into a String with Render, parsing numbers with ParseInt32, and reading lines with ReadLine.
• Walking the lines of a text with Lines, joining pieces with Join, and comparing text without regard to case with FoldCase.
• Regular expressions with the Regex package: testing and finding matches, capture groups, every match in turn, splitting and replacing.

The text types at a glance

flowchart LR
    lit(["A literal<br/>`char8[..]`"]) -- "`StringView::FromValidated`<br/>or `FromBytes` (checked)" --> view["`StringView`<br/>borrowed, known to be UTF-8<br/>(14.3)"]
    lit -- "`String::FromBytes`" --> str["`String`<br/>owns its bytes<br/>(14.4)"]
    sb["`StringBuilder`<br/>grows a buffer<br/>(14.5)"] -- "`IntoString`" --> str
    sb -- "`View`" --> view
    str -- "`View`" --> view
    r["`Render(pattern, values)`<br/>(14.11)"] --> str
    in["`ReadLine`<br/>(14.13)"] -- "appends to" --> sb
    view -- "`ParseInt32`<br/>(14.12)" --> num(["a number"])
    view -- "`Lines`<br/>(14.14)" --> view
    view -- "`Join`<br/>(14.14)" --> str

Lessons

Before you start

Finish Part 13: Generics. Text leans on almost everything before it: slices from Part 5, optionals and fallibles from Part 8 and Part 9 — nearly every text operation can fail — Destructor from Part 11 for the memory a String gives back, and Display and Iterator from Part 12. Each lesson's package is in the Examples repository's Text/ folder:

cd Examples/Text/StringLiteral
rux run

Several lessons pass an Allocator around without explaining it yet. For now it is simply where a String's bytes come from; Part 15: Memory opens it up.

After this part

Part 15: Memory explains the allocators and pointers this part used without looking inside — the @written of Unicode case and the Allocator every String was given. Part 16: Numbers then ends with the checkpoint projects Circle and Quadratic, which read numbers typed by the user with this part's ReadLine and the Format package's parsers.

For the language rules behind literals, see String literals, Literals and the character types in the Rux Reference, and the Text, Format, Io and Regex packages in the API reference.