Recurrent Neural Networks using TensorFlow Jindich Libovick - - PowerPoint PPT Presentation

▶

Jan 31, 2024 106 likes •194 views

Recurrent Neural Networks using TensorFlow Jindich Libovick December 5, 2018 B4M36NLP Introduction to Natural Language Processing Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless

SLIDE 1

Recurrent Neural Networks using TensorFlow

Jindřich Libovický

December 5, 2018

B4M36NLP Introduction to Natural Language Processing

Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise stated

SLIDE 2

The Taks

Train a model that will correctly decide whether you should write ‘i’ or ‘y’ in a Czech sentence.

Recurrent Neural Networks using TensorFlow

1/7

SLIDE 3

Prepare Python Environment

create a new environment

virtualenv -p python3 env

activate the environment

source env/bin/activate

install TensorFlow and Jupyter

pip3 install tensorflow pip3 install jupyter

Recurrent Neural Networks using TensorFlow

2/7

SLIDE 4

Alternatively using Anaconda

download and install Anaconda

wget http://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh bash Anaconda3-4.2.0-Linux-x86_64.sh export PATH=$PATH:$HOME/anaconda3/bin

create a new environment

conda create -n tf python=3.5 anaconda

activate the environment

source activate tf

install TensorFlow 0.12

pip3 install tensorflow pip3 install jupyter

Recurrent Neural Networks using TensorFlow

3/7

SLIDE 5

Download the Lab Notebook

download the notebook

wget http://ufallab.ms.mff.cuni.cz/~libovicky/ctu_lab.ipynb

run jupyter in the same directory

jupyter notebook

Recurrent Neural Networks using TensorFlow

4/7

SLIDE 6

The Data

500k sentences from Czech Wikipedia (in general the more, the better)
only character from Czech alphabet, sentence-splitted, lower-cased
randomly shuffmed, seperated validation data

The text:

aristotelés dále určil poloměr země, kterí ale odhadl na dvojnásobek… v aristotelovském modelu země stojí a měsíc se sluncem a hvězdami krouží… mišlenki aristotelovi rozvinul ve 2. století našeho letopočtu klaudios…

Correct solution:

00001000000000000000000000000000001000000100000000000000000000001000000100000… 02000002000100000000200000100000000000000001000000000000000000000001000000000… 00000000000000000010000000000001000002000000000000000000020000000000000000000…

1 = ‘i’, 2 = ‘y’, 0 = ‘others’

Recurrent Neural Networks using TensorFlow

5/7

SLIDE 7

Baselines

just leave ‘i’ everywhere

70 %

simple rules: ‘y’ after ‘h’, ‘k’, ‘r’ and for words starting with ‘v’

80 %

remember the most frequent spelling for each word

90 %

Recurrent Neural Networks using TensorFlow

6/7

SLIDE 8

Learning curves

500 5000 50000 500000 5000000 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Most frequent spelling Recurrent network Baseline Simple rules

# Training sentences (log scale) Accuracy

Recurrent Neural Networks using TensorFlow

7/7