[PPT] - Accessibility Issues in Digital Mathemacal Libraries Petr Sojka, PowerPoint Presentation

SLIDE 1

Accessibility Issues in Digital Mathemacal Libraries

Petr Sojka, Michal Růžička, Maroš Kucbel, and Marn Jarmar

Masaryk University, Faculty of Informacs, Brno, Czech Republic <sojka@fi.muni.cz>, {mruzicka, kocka, 172981}@mail.muni.cz

Universal Learning Design, Brno, 13th February 2013

SLIDE 2

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Outline

. . 1 Introducon . . 2 PDF Processing . . 3 MathML Processing . . 4 Summary

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 3

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Introducon

Digital mathemacs libraries are on the rise.
The European Digital Mathemacs Library

(EuDML, <hps://eudml.org/>).

The Czech Digital Mathemacs Library (DML-CZ, <hp://dml.cz/>).
Serves not only metadata but also full texts with mathemacal

formulae.

PDF.
MathML.
*T

EX.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 4

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

PDF, T EX/L

AT

EX, MathML

Thanks to pdfT

EX, PDF is the de facto standard output format of the modern T EX distribuons.

L

AT

EX mathemacal notaon is well known and effecve.

Used not only in L

AT

EX documents but also in a variety of other projects such as Wikipedia.

L

AT

EX source code is usually a good choice for plain text representaon

f mathemacal expressions.
MathML is oen used as both machine and human readable language

for describing mathemacal notaons.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 5

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

PDF Processing

. . 1 Introducon . . 2 PDF Processing

PDF Processing PDF Enhancement

. . 3 MathML Processing . . 4 Summary

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 6

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MaxTract

A command line tool that reads a PDF and returns various types of

enriched output.

L

AT

EX for use with Tralics.

L

AT

EX for layered PDF with L

AT

EX and text layers.

L

AT

EX for annotated PDF with L

AT

EX annotaons.

A simple text file.
A text file with math in L

AT

EX.

Under development by the Scienfic Document Analysis Group at

School of Computer Science, University of Birmingham, UK.

Homepage:

<hp://www.cs.bham.ac.uk/research/groupings/reasoning/sdag/maxtract.php>

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 7

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MaxTract (cont.)

For successful analysis, the PDF file must make sole use of Type 1

fonts with embedded encodings.

MaxTract is wrien in OCaml and uses the pdk for decompressing

PDF files.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 8

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

InyReader OCR

Old documents are oen available in paper form only.
It is necessary to scan them and process by Opcal Character

Recognion (OCR) soware.

InyReader OCR soware has unique feature of recognion of

mathemacal expressions in scanned documents.

InyReader is part of the InyProject

(<hp://www.inyproject.org/>) under development by Masakazu Suzuki’s research and development group in Japan.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 9

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

InyReader OCR (cont.)

InyReader inputs and output various formats.

input TIFF, BMP, GIF, PNG, PDF.

utput L

AT

EX, XHTML+MathML, various XML formats.

Quality and resoluon of scans is crucial.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 10

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

CopyMath

The ActualText command of the PDF language is used to mark the

region of the mathemacal expression inside the PDF document.

We want the package to be as user friendly as possible – users should

not be forced to modify their mathemacal expressions in any way, \usepackage{copymath} should cater for all their needs.

The implementaon is not easy.
This requires nonstandard modificaons of the L

AT

EX mathemacal environments.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 11

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Standard PDF document

L

AT

EX source code:

Text $\Pi(x) = \pi(x) + \frac{1}{2}\pi(x^{1/2}) + \frac{1}{3}\pi(x^{1/3}) + \cdots$ text.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 12

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Standard PDF document

PDF code:

BT /F16 9.9626 Tf 148.712 707.125 Td [(T)83(ext)]TJ/F17 9.9626 Tf 23.247 0 Td [(\005\050)]TJ/F20 9.9626 Tf 11.346 0 Td [(x)]TJ/F17 9.9626 Tf 5.694 0 Td [(\051)-278(=)]TJ/F20 9.9626 Tf 17.158 0 Td [(\031)]TJ/F17 9.9626 Tf 6.036 0 Td [(\050)]TJ/F20 9.9626 Tf 3.875 0 Td [(x)]TJ/F17 9.9626 Tf 5.694 0 Td [(\051)-222(+)]TJ/F18 6.9738 Tf 17.247 3.923 Td [(1)]TJ ET Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 13

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Standard PDF document

Text obtained using Copy & Paste funcon of PDF reader:

Text Π(𝑦) = 𝜌(𝑦) + 1 2𝜌(𝑦1/2) + 1 3𝜌(𝑦1/3) + · · · text.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 14

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

CopyMath-enabled PDF document

L

AT

EX source code:

Text $\Pi(x) = \pi(x) + \frac{1}{2}\pi(x^{1/2}) + \frac{1}{3}\pi(x^{1/3}) + \cdots$ text.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 15

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

CopyMath-enabled PDF document

PDF code:

BT /F16 9.9626 Tf 148.712 707.125 Td [(T)83(ext)]TJ ET 1 0 0 1 171.959 707.125 cm /Span << /ActualText<245C506920287829203D205C706920287829202B205C66726163207B317D7B32 7D5C70692028785E7B312F327D29202B205C66726163207B317D7B337D5C70692028785E7B31 2F337D29202B205C63646F74732024> >> BDC 1 0 0 1 -171.959 -707.125 cm BT /F17 9.9626 Tf 171.959 707.125 Td [(\005\050)]TJ/F20 9.9626 Tf 11.346 0 Td [(x)]TJ/F17 9.9626 Tf 5.694 0 Td [(\051)-278(=)]TJ/F20 9.9626 Tf 17.158 0 Td [(\031)]TJ/F17 9.9626 Tf 6.036 0 Td [(\050)]TJ/F20 9.9626 Tf 3.875 0 Td [(x)]TJ/F17 9.9626 Tf 5.694 0 Td [(\051)-222(+)]TJ/F18 6.9738 Tf 17.247 3.923 Td [(1)]TJ ET Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 16

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

CopyMath-enabled PDF document

Text obtained using Copy & Paste funcon of PDF reader:

Text $\Pi (x) = \pi (x) + \frac {1}{2}\pi (x^{1/2}) + \frac {1}{3}\pi (x^{1/3}) + \cdots $ text.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 17

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

CopyMath Implementaon

We need to add \pdfliteral at the beginning and end of every

mathemacal environment.

The dollar sign ($) is acvated and redefined.
It is necessary to keep track of nested mathemacal environments.
Simple redefinion of A

MS-L

AT

EX mathemacal environments is not possible.

Sll experimental.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 18

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Processing

. . 1 Introducon . . 2 PDF Processing . . 3 MathML Processing

Making Maths Accessible MathML Processing

. . 4 Summary

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 19

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Reading Formulae Aloud

Under development by Maroš Kucbel at Faculty of Informacs

Masaryk University.

Input XML file is processed and every MathML block is converted to

plain text to be read aloud by a speech synthesizer soware.

English is currently the only supported language.
Output is again in a valid XML format.
Homepage: <hps://code.google.com/p/mathml-converter/>

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 20

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Converter Example

<?xml version=”1.0” encoding=”UTF-8”?> <math xmlns=”http://www.w3.org/1998/Math/MathML”> <mrow> <msubsup> <mo>∑</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>0</mn> </mrow> <mn>10</mn> </msubsup> <mrow> <mi>i</mi> <mo>+</mo> <mn>7</mn> </mrow> </mrow> </math>

𝑗 + 7

<?xml version=”1.0” encoding=”UTF-8”?> <math xmlns=”http://code.google.com/p/mathml-converter/”> sum over i from zero to ten of i plus seven </math>

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 21

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Mathemacal Formulae Braille Output

There is no simple language independent encoding of math in Braille.
MathML to Braille conversion tool development by Marn Jarmar at

Faculty of Informacs Masaryk University.

We have decided to support Czech version of Braille.
The tool is implemented as one of the UMCL (Universal Maths

Conversion Library) output drivers.

Available at: <hps://is.muni.cz/th/172981/fi_m/>

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 22

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Example of Czech Braille Conversion

<math xmlns=”http://www.w3.org/1998/Math/MathML”>

Czech Braille: 605500264710375506

<Cz> <symbol role=”braille” blank_after=”false” name=”mfrac”>60</symbol> <symbol role=”braille” name=”x” letter=”latin-low” prefix=”false”>55</symbol> <symbol role=”braille” blank_before=”true” blank_after=”false” name=”+” category=”over”>26</symbol> <symbol role=”braille” blank_after=”false” name=”number”>47</symbol> <symbol role=”braille” name=”1” letter=”number”>10</symbol> <symbol role=”middle” name=”mfrac”>37</symbol> <symbol role=”braille” name=”x” letter=”latin-low” prefix=”false”>55</symbol> <symbol role=”end” blank_before=”false” name=”mfrac”>06</symbol> </Cz> Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 23

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Math-Aware Search Engine

We believe important funcon of digital mathemacs libraries is

implementaon of math-aware search engine.

The ambiguies of MathML encoding are issues not only for

accessibility of mathemacal documents but also for math-aware search engines.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 24

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Sources ‘Hand made’ 𝑦2 + 𝑧2 MathML

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 25

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Sources L

AT

EXML 𝑦2 + 𝑧2 MathML

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 26

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Sources Matlab 𝑦2 + 𝑧2 MathML

generate::MathML(x^2 + y^2, Content = FALSE, Annotation = FALSE) <math xmlns=’http://www.w3.org/1998/Math/MathML’> <mrow xref=’No7’> <msup xref=’No3’> <mi xref=’No1’>x</mi> <mn xref=’No2’>2</mn> </msup> <mo>+</mo> <msup xref=’No6’> <mi xref=’No4’>y</mi> <mn xref=’No5’>2</mn> </msup> </mrow> </math>

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 27

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Sources InyReader 𝑦2 + 𝑧2 MathML

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 28

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

UMCL and MathML Canonicalizaon

Universal Maths Conversion Library (UMCL,

<hp://sourceforge.net/projects/umcl/>)

The main purpose is the transcripon of the MathML formulae to

Braille naonal codes.

Part of the translaon process is also canonicalizaon of input

MathML.

The canonicalizaon module implemented as a set of XSL

transformaons.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 29

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

UMCL Canonicalizaon Module Deficiencies

The UMCL canonicalizaon module appeared to have severe

deficiencies.

XSLT is not fast enough.
The UMCL canonicalizaon module proved to change semancs of

input formula during the transformaon.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 30

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

UMCL Canonicalizaon Module Deficiencies Input: (𝑏 + 𝑐)

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 31

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

UMCL Canonicalizaon Module Deficiencies Output: 𝑏 + 𝑐 + 𝑐𝑐

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 32

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer

We decided to implement our own MathML Canonicalizaon tool.
The main design imperave is the
modularity,
simplicity,
extensibility, and
flexibility.
The speed of the canonicalizaon applicaon is also a crical

parameter.

In our MREC corpora there is 168,000,000 formulae to canonicalize.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 33

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases <mphantom> Omission

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 34

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases Unnecessary Aributes

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 35

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases Unifying Fences

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 36

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases <mrow> Minimizing

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 37

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases Sub-/Superscripts Handling

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 38

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases Applying Funcons

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 39

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

MathML Canonicalizer Use Cases Applying Funcons

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 40

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Summary

. . 1 Introducon . . 2 PDF Processing . . 3 MathML Processing . . 4 Summary

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 41

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Summary

Digital mathemacs libraries usually serves full texts in PDF format

with various accessibility issues. Maths is usually encoded in MathML

r *T

EX language.

Born-digital PDF documents can be enriched or converted to L

AT

EX by MaxTract.

Scanned PDF documents can be processed by InyReader OCR

soware and converted to MathML.

MathML can be read aloud or converted to Czech Braille.
We are developing MathML canonicalizaon tool for math-aware

search engine.

Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 42

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Conclusion

There are accessibility issues in digital mathemacs libraries given especially by shortcomings of available data formats but tools for handling these formats are on the way.

Quesons?

Acknowledgements: This work has been parally supported by the European Union through its Compeveness and Innovaon Programme (Informaon and Communicaons Technologies Policy Support Programme, “Open access to scienfic informaon”, Grant Agreement No. 250503). Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013

SLIDE 43

. . Introducon . . . . . . . . PDF Processing . . . . . . . . . . MathML Processing . . Summary

Conclusion

There are accessibility issues in digital mathemacs libraries given especially by shortcomings of available data formats but tools for handling these formats are on the way.

Quesons?

Acknowledgements: This work has been parally supported by the European Union through its Compeveness and Innovaon Programme (Informaon and Communicaons Technologies Policy Support Programme, “Open access to scienfic informaon”, Grant Agreement No. 250503). Accessibility Issues in Digital Mathemacal Libraries Universal Learning Design, Brno, 13th February 2013