unicode introduction
play

Unicode Introduction Ken Zook November, 2006 1 Unicode properties - PowerPoint PPT Presentation

Oct 31, 2023 •325 likes •483 views

Unicode Introduction Ken Zook November, 2006 1 Unicode properties 0041;LATIN CAPITAL LETTER A;Lu;0;L;;;;;N;;;;0061; A Representative glyph Code point: 0041 Name: LATIN CAPITAL LETTER A Semantic General category: Uppercase letter (Lu)

  1. Unicode Introduction Ken Zook November, 2006 1

  2. Unicode properties 0041;LATIN CAPITAL LETTER A;Lu;0;L;;;;;N;;;;0061; A Representative glyph Code point: 0041 Name: LATIN CAPITAL LETTER A Semantic General category: Uppercase letter (Lu) properties Canonical combining class: Standard spacing (0) Bidirectional category: Left-to-right (L) Mirrored: no (N) Lowercase mapping: 0061 November, 2006 Unicode Introduction 2

  3. Unicode code space Compatibility & General scripts East Asian specials 0000 FFFF Surrogates Symbols & punctuation Private Use Area (PUA) Basic multilingual plane (BMP) 0000 10FFFF Planes 1-16 accessed by surrogates when using UTF-16 November, 2006 Unicode Introduction 3

  4. Encoding Unicode UTF-32 = 10331 (1 32-bit value / code point) UTF-16 = D800 DF31 (FW/Win) (1-2 16-bit values / code point) UTF-8 = F0 90 8C B1 (XML) (1-4 8-bit values / code point) UTF-16 Surrogates: D800-DFFF High: D800-DBFF, Low: DC00-DFFF 0000 FFFF U+10331 GOTHIC LETTER BAIRKAN D800 DF31 10331 Surrogates used to access 10000-10FFFF in UTF-16 November, 2006 Unicode Introduction 4

  5. Private Use Area (SIL) International PUA: F100-F8FF (2,047) Entity PUA: E000-EFFF (4,095) PUA: E000-F8FF (6,400) E010 (Philippines) maps to F2010 E010 (Russia) maps to F1010 PUA: F0000-FFFFD, 100000-10FFFD (131K) Unique entity mappings in upper PUA November, 2006 Unicode Introduction 5

  6. Canonical equivalence 01FA LATIN CAPITAL LETTER A WITH RING ABOVE AND ACUTE 212B 0301 ANGSTROM SIGN COMBINING ACUTE ACCENT 00C5 0301 LATIN CAPITAL LETTER A WITH RING ABOVE COMBINING ACUTE ACCENT 0041 030A 0301 LATIN CAPITAL LETTER A COMBINING RING ABOVE COMBINING ACUTE ACCENT November, 2006 Unicode Introduction 6

  7. Normalization (NFD) 014D;LATIN SMALL LETTER O WITH MACRON;;0;;006F 0304 … 01ED;LATIN SMALL LETTER O WITH OGONEK AND MACRON;;0;;01EB 0304… 01EB;LATIN SMALL LETTER O WITH OGONEK;;0;;006F 0328 … 0304;COMBINING MACRON;;230… 0328;COMBINING OGONEK;;202… 006F 0328 0304 006F 0304 0328 ≡ 006F 0328 0304 014D 0328 ≡ 006F 0304 0328 ≡ 006F 0328 0304 01ED ≡ 01EB 0304 ≡ 006F 0328 0304 November, 2006 Unicode Introduction 7

  8. Normalization (NFC) 014D;LATIN SMALL LETTER O WITH MACRON;;0;;006F 0304 … 01ED;LATIN SMALL LETTER O WITH OGONEK AND MACRON;;0;;01EB 0304… 01EB;LATIN SMALL LETTER O WITH OGONEK;;0;;006F 0328 … 0304;COMBINING MACRON;;230… 0328;COMBINING OGONEK;;202… 006F 0328 0304 ≡ 01EB 0304 ≡ 01ED 006F 0304 0328 ≡ 006F 0328 0304 ≡ 01EB 0304 ≡ 01ED 014D 0328 ≡ 006F 0328 0304 ≡ 01EB 0304 ≡ 01ED 01ED ≡ 006F 0328 0304 ≡ 01EB 0304 ≡ 01ED November, 2006 Unicode Introduction 8

  9. Case mapping SpecialCasing.txt + UnicodeData.txt Unicode digraphs require title casing 01F1;LATIN CAPITAL LETTER DZ;Lu;;;;;;;01F3;01F2 01F2;LATIN CAPITAL LETTER D WITH SMALL LETTER Z;Lt;;;;;;;;;01F1;01F3; 01F3;LATIN SMALL LETTER DZ;Ll;;;;;;;;;01F1;;01F2 Case mapping is not reversible McConnel  mcconnel  MCCONNEL November, 2006 Unicode Introduction 9

  10. Case mapping Case mapping may produce strings of different length 01F0  004A 030C Case mapping may depend on the locale 0069  0049 English 0069  0130 Turkish/Azeri November, 2006 Unicode Introduction 10

  11. Case mapping Case mapping may depend on context 03A3 <letter>  03C3 03A3  03C2 November, 2006 Unicode Introduction 11

  12. Case mapping Some characters require special handling 1F80  1F88 or ...1F08 0399… 03B1 0313 0345  1F08 03B9 Case mapping may not preserve normalization 01F0 0323  004A 030C 0323 ≡ 004A 0323 030C NFC NFC November, 2006 Unicode Introduction 12

  13. Smart rendering: Arabic Keyboard: Code points: 0628 0628 064e 0628 064e 0628 0628 064e 0628 0650 0628 064e 0628 0650 0628 064e 0628 0650 0628 064e 0628 0650 0628 064e 0628 0650 babibu b babibu babib babi bab ba b 0628 064f 0020 0628 0628 064f 0020 0628 064f 0628 Screen: November, 2006 Unicode Introduction 13

  14. Smart rendering: Burmese Keyboard: Code points: 1000 1039 101b 1000 1039 101b 1000 1039 101b 1000 krui kru kr k 102f 102d 102f Screen: November, 2006 Unicode Introduction 14

  15. Smart rendering: Tamil Keyboard: Ur rU yU NU mU kU jU Ur rU yU NU mU kU Ur rU yU NU Ur rU yU NU mU kU j Ur rU yU NU mU k Ur rU yU NU mU Ur rU yU NU m Ur rU yU N Ur rU yU Ur rU y Ur rU Ur r Ur U Code b8a bb0 b8a bb0 bb0 bc2 baf bc2 baf points: ba3 bc2 bae bc2 bae b95 bc2 b95 ba3 b9c bc2 b9c Screen: November, 2006 Unicode Introduction 15

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

What is Unicode? Universal Character Set All of the major scripts Sinhala Unicode

What is Unicode? Universal Character Set All of the major scripts Sinhala Unicode

What is Unicode? Universal Character Set All of the major scripts Sinhala Unicode Simple and consistent manner Developer Workshop Alphabetic, syllabic and ideographic scripts Version 4.0 Muthu Nedumaran 50,000

686 views • 4 slides
Unicode Security Considerations (TR#36) Michel Suignard Senior Program Manager, Microsoft

Unicode Security Considerations (TR#36) Michel Suignard Senior Program Manager, Microsoft

Unicode Security Considerations (TR#36) Michel Suignard Senior Program Manager, Microsoft Technical Director, Unicode Consortium Unicode in short About 98000 characters allocated, cover all major writing systems, languages of the world

378 views • 14 slides
Unicode BCP47 Extensions Mark Davis http://goo.gl/owbBk Unicode Locale/Lang ID BCP47

Unicode BCP47 Extensions Mark Davis http://goo.gl/owbBk Unicode Locale/Lang ID BCP47

Unicode BCP47 Extensions Mark Davis http://goo.gl/owbBk Unicode Locale/Lang ID BCP47 Optional: only use where needed sl -Latn -IT -fonipa EXTENSIONS Variant(s) [digit4/alphanum5..8] Italy - ISO 3166 [alpha2] or UN M49* [digit3]

550 views • 10 slides
Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for

Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for

Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for Bangla Unicode Agenda for Bangla Bidyut Baran Chaudhuri Bidyut Baran Chaudhuri Society for Natural Language Technology Research Society for Natural Language Technology

274 views • 24 slides
Software for the world: latest developments in Unicode and CLDR Mark Davis President &amp;

Software for the world: latest developments in Unicode and CLDR Mark Davis President &amp;

Software for the world: latest developments in Unicode and CLDR Mark Davis President &amp; Co-founder Unicode Consortium Unicode Consortium All modern software: OSs, smartphones, XML, Core Globalization Standards and Data Encoding

888 views • 31 slides
Pango An open-source Unicode text layout engine Owen Taylor otaylor@redhat.com 25th

Pango An open-source Unicode text layout engine Owen Taylor otaylor@redhat.com 25th

Pango An open-source Unicode text layout engine Owen Taylor otaylor@redhat.com 25th Internationalization and Unicode Conference Washington, DC March/April 2004 Outline Introduction - why Pango? Application view Architecture Layout

688 views • 37 slides
Unicode Character Code A character is the smallest possible component of a tex t (e.g., A,

Unicode Character Code A character is the smallest possible component of a tex t (e.g., A,

Unicode Character Code A character is the smallest possible component of a tex t (e.g., A, B, and ) that has semantic value. Even the extended (8 bit) version of ASCII is not enough for international use. The Unicode

315 views • 14 slides
Unicode 4.0 In Common Unicode 4.0 In Common Lisp Lisp Adoption of Unicode In CLforJava

Unicode 4.0 In Common Unicode 4.0 In Common Lisp Lisp Adoption of Unicode In CLforJava

- - (defun the- -old old- - --fn (a fn (a d) d) (defun the - - (if (eql a d 2) 2 (* a d 2) 2 (* a d (the d (the- -old old- -

1.19k views • 57 slides
Unicode and Unicode and ISO/IEC 10646 ISO/IEC 10646 V.S. Umamaheswaran V.S. Umamaheswaran

Unicode and Unicode and ISO/IEC 10646 ISO/IEC 10646 V.S. Umamaheswaran V.S. Umamaheswaran

INTERNATIONAL ORGANIZATION FOR STANDARDIZATION L2/04-028 ORGANISATION INTERNATIONALE DE NORMALISATION ISO/IEC JTC 1/SC 2/WG 2 Universal Multiple-Octet Coded Character Set (UCS) ISO/IEC JTC 1/SC 2/WG 2 N 2696 2004-01-22 Title: Presentation

318 views • 16 slides
April 7, 2005 A Jonathan Kew SIL International 27th Internationalization and Unicode Conference

April 7, 2005 A Jonathan Kew SIL International 27th Internationalization and Unicode Conference

e Multilingual Lion: T EX learns to seak Unicode April 7, 2005 A Jonathan Kew SIL International 27th Internationalization and Unicode Conference Berlin, Germany, April 2005 Te Multilingual Lion: T EX learns to seak Unicode Background

635 views • 27 slides
Unicode Localization Data Interoperability TC Overview (ULI) Whats a word? Whats a

Unicode Localization Data Interoperability TC Overview (ULI) Whats a word? Whats a

Unicode Localization Data Interoperability TC Overview (ULI) Whats a word? Whats a sentence? Why is this business-relevant? Christian Lieske, SAP (Walldorf, Germany) Helena Shih Chapman, IBM (Waltham, Massachusetts, USA) META-FORUM 2013

214 views • 10 slides
lecture 8 http://www.asciitable.com/ ASCII: 8 bit (one byte) code Unicode is a 16 bit

lecture 8 http://www.asciitable.com/ ASCII: 8 bit (one byte) code Unicode is a 16 bit

lecture 8 http://www.asciitable.com/ ASCII: 8 bit (one byte) code Unicode is a 16 bit code which includes characters for most In fact, it only uses 7 of the 8 bits. written natural languages. [Correction: It is more complicated than

370 views • 4 slides
unicode.decode() lea king frm yo ur eye s lik e liq uid p ain

unicode.decode() lea king frm yo ur eye s lik e liq uid p ain

unicode.decode() lea king frm yo ur eye s lik e liq uid p ain ukasz Taczuk ukasz Taczuk The guessing game: f.write( u'tralala: {}' .format( 'foo' .encode( u'utf8' ))) ukasz Taczuk Python 2 / Python 3

1.25k views • 74 slides
7. International character sets Default character set: Unicode Characters correspond to

7. International character sets Default character set: Unicode Characters correspond to

7. International character sets Default character set: Unicode Characters correspond to numbers . Different encodings exist for these numbers. In Web servers HTTP metadata specifies the coding (so called MIME media type), e.g.

126 views • 12 slides
Mined : An Editor with Extensive Unicode and CJK Support for the Text-based Terminal Environment

Mined : An Editor with Extensive Unicode and CJK Support for the Text-based Terminal Environment

Mined : An Editor with Extensive Unicode and CJK Support for the Text-based Terminal Environment IUC 27, Berlin, 2005-04-08 Thomas Wol Mined : introduction A text editor suitable for editing text suitable for editing

177 views • 14 slides
An end-to end solution for using unicode with Blaise to support any language Alerk Amin

An end-to end solution for using unicode with Blaise to support any language Alerk Amin

An end-to end solution for using unicode with Blaise to support any language Alerk Amin (CentERdata) Richard Boreham (NatCen) Maurice Martens (CentERdata) Colin Miceli (NatCen) Overview Understanding Society Language

508 views • 16 slides
Multifidelity importance sampling methods for rare event simulation Benjamin Peherstorfer

Multifidelity importance sampling methods for rare event simulation Benjamin Peherstorfer

Multifidelity importance sampling methods for rare event simulation Benjamin Peherstorfer University of Wisconsin-Madison Karen Willcox and Boris Kramer MIT Max Gunzburger Florida State University July 2017 1 / 24 Introduction

600 views • 46 slides
Incremental and Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization

Incremental and Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization

Incremental and Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization Julien Mairal INRIA LEAR, Grenoble Gargantua workshop, LJK, November 2013 Julien Mairal Incremental and Stochastic MM Algorithms 1/28 A simple

583 views • 46 slides
Understanding the Effectiveness of Plutonium Surrogates for Waste and Stockpile Immobilisation

Understanding the Effectiveness of Plutonium Surrogates for Waste and Stockpile Immobilisation

2018-09-17 Understanding the Effectiveness of Plutonium Surrogates for Waste and Stockpile Immobilisation Lewis R Blackburn Immobilisation Science Laboratory University of Sheffield Supervisors: - Neil Hyatt (UoS) - Martin Stennett(UoS) -

298 views • 18 slides
Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computing Geoffrey

Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computing Geoffrey

Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computing Geoffrey Fox, many others ., Shantenu Jha* Rutgers University and Brookhaven National Lab. http://radical.rutgers.edu Outline Learning

450 views • 24 slides
Surrogate Dependencies (in NodeJS) @DinisCruz London, 29th Sep 2016 Me Developer for 25

Surrogate Dependencies (in NodeJS) @DinisCruz London, 29th Sep 2016 Me Developer for 25

Surrogate Dependencies (in NodeJS) @DinisCruz London, 29th Sep 2016 Me Developer for 25 years AppSec for 13 years Day jobs: Leader OWASP O2 Platform project Application Security Training JBI Training, others Part of

651 views • 32 slides
SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates June, 2019 Martin Engilberge, Louis

SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates June, 2019 Martin Engilberge, Louis

SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates June, 2019 Martin Engilberge, Louis Chevallier, Patrick Prez, Matthieu Cord SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates June, 2019 Martin Engilberge, Louis

558 views • 11 slides
Cross-platform fi le names in Rust WTF-8: a wonderful and horrifying hack! Simon Sapin, Mozilla

Cross-platform fi le names in Rust WTF-8: a wonderful and horrifying hack! Simon Sapin, Mozilla

Cross-platform fi le names in Rust WTF-8: a wonderful and horrifying hack! Simon Sapin, Mozilla Research !!Con, 2015-05-16 I work at Mozilla Research on Servo, a browser engine written in Rust. I also contribute a little bit to Rust. 1.0

469 views • 13 slides
Legislative Update Heidi Junge June 20, 2019 The webinar will begin shortly. Phone |

Legislative Update Heidi Junge June 20, 2019 The webinar will begin shortly. Phone |

Welcome to todays webinar! Legislative Update Heidi Junge June 20, 2019 The webinar will begin shortly. Phone | 1-800-619-3315 Passcode | 4597908 Q&amp;A Process Ask questions in two ways: 1. Use online chat feature during webinar 2.

1.25k views • 110 slides

More recommend