Unicode in C++ - James McNellis - Meeting C++ 2016

Meeting Cpp · 14K views · 2016 · 1:04:08

Open on YouTube

C, C++, Rust, Go & systems languagesSoftware practice, open source & careerstalk

Why Stack Overflow recommends it

Stack Overflow users link it when answering questions about wide characters, accented letters, code pages and Unicode literals in C++, since it explains encodings and C++'s limited Unicode support.

Recommended by 3 people across 3 different months, 2016 to 2020.

What it is

A conference talk by James McNellis at Meeting C++ 2016 on how Unicode works and what C++ does and does not offer for handling it. It covers character set encodings, the UTF-8, UTF-16 and UTF-32 forms, and how to use them with C++ string types and literals.

What people said

Strong rec Christophe · 2016-12 · Stack Overflow · on “How to cast accented letters (wchar_t) to char?”

If 'é' can be stored on one character on Windows, your application was probably compiled without UNICODE and certainly with a Win 1252 [LINK] encoding. With the usual utf-8 encoding on linux, the 'é' should require 2 characters [LINK] . This should cause a warning from the compiler. And if you would use the character obtained, it would represent only a part of the encoding, so that the char by char comparison would be meaningless. If you want to keep your algorithms, using individual characters of a string, you'd better work with wchar_t and wstring (or event more portable: char32_t and u32string ). If you want to know more on character encoding and unicode with C++, I can only warmly recomm

Recommends Christophe · 2016-12 · Stack Overflow · on “Unicode literal - how does this even make sense?”

Unicode and C++ There are several unicode encodings: UTF-8 [LINK] encodes each unicode character into a sequence of one to four (8- bit) bytes ( char ) UTF-16 [LINK] (which can be BE and LE depending on endianness) encodes each unicode character into a sequence of one or two 16 bit words ( char16_t ). UTF-32 [LINK] (again BE or LE) encodes each unicode character into one 32 bit word ( char32_t ). Here is an excellent video tutorial on unicode with C++ [this video] by James McNellis. He explains everything you need to know on character set encoding, on unicode and its different encodings, and how to use it in C++. Your code "\u2654" is a a narrow string literal, that has the type array of char . T

Recommends Trevor Hickey · 2017-02 · Stack Overflow · on “C++ Correctly read files whose Unicode characters might be larger than a byte”

I'd recommend watching Unicode in C++ by James McNellis [this video] . That will help explain what facilitates C++ has and does not have when dealing with Unicode. You will see that C++ lacks good support for easily working with UTF8. Since it sounds like you want to iterate over each glyph (not just code points), I'd recomend using a 3rd pary library to handle the intricacies. utfcpp [LINK] has worked well for me.

Recommends Qaz · 2020-06 · Stack Overflow · on “Printing all ASCII characters from 0 to 255”

Speaking of code pages, the diagram you meant to link to (but got google's intermediate URL instead) is actually Code Page 437 [LINK] . There are many others and you can go see the differences. You can even change the console's code page and watch the same program produce completely different output. If this topic piques your interest, there's a great talk [this video] on it from a few years back.