How far down the digital road will EL assessment go? TECHNOLOGY FOR TEACHERS IN ASSESSMENT – THE IMMEDIATE FUTURE 1&2 November, 2018 Alex Thorp Lead Academic - Europe Trinity College London English qualifications for real-world communication
Overview 1. Back to the start – Introduction 2. Introducing AI – history and definitions 3. AI and language – NLP 4. Chatbots 5. AI and Language assessment (Speaking focus) 6. Case study – Communicative competence 7. Summary 8. Test evaluation – the 3 c’s 9. Future considerations
Introduction
Introduction – true or false? Current AI still hugely limited, processing equivalent to a 2 year old AI and, more particularly NLP, can now offer a fully automated 4-skill assessment solution AI dates back as far as the 1950s The human brain provided the model for modern machine learning That which humans find easy, computers find difficult – and vice versa Elon Musk labelled AI ‘a fundamental risk to the existence of civilization’ Machine scoring is more reliable than human scoring I’ve utilized AI this morning!
Spot the odd one out?
Computers as ‘tutors or tools’? Name of tool Developer Language learning/testing Write & Improve English Language iTutoring Learning Write & Improve +Class View English Language iTutoring Learning Write & Improve +Test Zone English Language iTutoring Testing Read & Improve (coming soon) English Language iTutoring Learning Duolingo Duolingo Learning / testing e-Rater ETS Testing Writing Mentor ETS Learning Language Muse Activity Palette ETS Learning / testing AuraLang AuraLang Learning BetterAccentTutor Better Accent Learning TriplePlayPlus Syracuse Language Systems Learning Test of English Language Learning Pearson Testing Intelligent Essay Assessor Pearson Testing IntelliMetric Vantage Learning Testing MyAccess! Vantage Learning Learning Project Essay Grade MI Learning / testing Summary table of identified commercially-available language learning and language testing tools. Gillings et al. 2018
Introducing AI
Back to basics - Communication cycle Coding is the application of linguistic resource through a range of cognitive processes to generate meaning – often described as competences
Back to basics - Communication cycle Coding is the application of linguistic resource through a range of cognitive processes to generate meaning – often described as competences
Back to the beginning 29’086 measures barley 37 months. Kushim A clay tablet with an administrative text from the city of Uruk, c.3400 – 3000 BC. Probably our first ever recorded code. If Kushim was indeed a person, he may be the first individual in history whose name is known to us! Y N Harari 2015
Let’s go back Full scripts As societies developed external Numerical partial script codes required to cope with became the language of sociological demands to support Partial scripts advancement larger collectives Unto the era of computers….
Can computers think like humans? H Simon and A Newell – Pittsburgh 1955. A thinking machine?
Can computers think like humans? How to overcome Combinational Explosion? How to give intelligence to make good decisions? Turing developed rules to guide. Alan Turing 1948 1st chess programme
The birth of Classical AI A problem defined, a set of Could plan complex Could deliver maximum programmed rules applied operations in highly efficiency and economy (Heuristics) controlled environments But classical AI couldn’t engage with it’s environment
Our world is a little more… chaotic
Enter Machine learning System’s ability to learn for themselves from raw data • Image recognition (training datasets) • Voice recognition • Optical character recognition • Advanced customisation System’s learn from • Intelligent data analysis first principles – from • Sensory data analysis structure in data, and seeks potential solutions to problems -Model (predicts) based on Parameters - Input to inform (training data) - Learner (adjusts parameters through differences in prediction and actual)
Enter Machine learning 1990’s: S hift from 1960’s – Bayesian knowledge to a >1990s: Support methods 1 980’s – back data driven Vector machines 2010>: ANN and introduced for propagation approach – analysis and Recurrent Deep learning probabilistic of large amount of Neural Networks inference data Machine learning: Algorithms that parse data, learn from that data, and then apply what they’ve learned to make informed decisions. The algorithm needs to be told how to make an accurate prediction
The Moravec Paradox The things that our brains find The things that our brains find easy to cope difficult to cope with, that require with, that require a little conscious mental a lot of conscious mental effort, effort, like making sense of what we see like chess, were simple for AI. and hear, or movement, were very difficult for AI “We are prodigious Olympians in perceptual and motor areas… abstract thought though is a new trick.. We’ve not yet mastered it” (Moravec 1988)
How does ML work? Enter Artificial Neural Networks You recognised a dog instantaneously, by the firing of choral assemblies of neural networks
Enter Artificial Neural Networks Neural Networks consist of the following components • An input layer , x • An arbitrary amount of hidden layers • An output layer , ŷ • A set of weights and biases between each layer, W and b • A choice of activation function for each hidden layer, σ .
Artificial Neural Networks Training data Is there Is there SVOCA a a full ? subject stop? ? S entence sample Sentence Is there Is there a OC? an capital? object? Sentence sample Non sentence Is it at Is there start of VA? a para? noun?
AI ANN : taught then develops Training data – each time we tell it what it’s looking at, it tweaks the connections to better recognise what it’s looking for. AI is now booming • Optimise harvesting • Interpret medical images • Grading students • Id financial opportunities • Driverless cars 10’s of 1000’s of simulations every second and chooses to do the best one
Enter Deep Learning Solve intelligence. Use it to make the world a better place. (Mission statement – DeepMind) Demis Hassabis - CEO Entering a process (e.g. playing a Uses Representation Learning – game) through a ‘learning automatically discovers algorithm’ that changes millions of characteristics needed for feature connections in a neural network to detection or classification of raw reinforce or stop an action to data, that is then used to perform a improve the desired outcome (not task task-based algorithm) Deep learning: ML requires input – DL can learn by itself through learning algorhithm. E.g. Automatic light – ML accepts only ‘dark’, DL would learn ‘I can’t see’
Let’s ‘Go’ In DL systems, the algorithm learns how to make accurate predictions through its own data processing (ML needs to be told). Could a DL neural network system go beyond human understanding? AlphaGo played a completely unpredictable move – can come up with a new idea beyond the remit of human thought….
AI Limitations • • Patterns in complex data Convert data into meaningful concepts • • Process ‘predictable’ (images / ‘Understand’ content or images – easily outcomes) tricked • • Operate autonomously – based Data engagement beyond on training datasets human capacity Can find patterns in, and learn from, data, but no real understanding of what those patterns actually mean, there is no meaningful conceptual thinking. With no real conceptual understanding of patterns – hardest challenge of all is ability that relies on exactly this - language Prof Al Khalili
AI and language
Recognise these? NLP DMS AI NLG NLU ASR SDS Chatbot
Communication cycle Coding is the application of linguistic resource through a range of cognitive processes to generate meaning – often described as competences
AI in language - NLP [Response driven] Text recognition Text generation (NLG) Automated Speech Speech generation Recognition (ASR) When was Elvis born?
AI in language – Speech Recognition Limited until advent of AI and Machine Collect waveforms Learning techniques (phonetic input) Converts to text – Fast Fourier Transform ‘best fit hypothesis’ = spectogram Labels ‘Formants’ Identifies resonances recognising phonemes, of production words and phrases
ASR Challenges – Who ate all the cake? I think David ate all the delicious chocolate cake. Tonic / Keywords / Onset – Volume / Pitch / Length / Pausing Remarkable number of variables - immense amount of comparative data to be processed to arrive at correct hypothesis as to meaning beyond denotation. Yet any communication act is a combination of oral production and non- verbal cues, paralinguistics and contextual parameters.
AI in language – Speech Recognition Formants – limited with 44 phonemes and syntactic training If only it were that easy:-) Requires a ‘Language model’
Recommend
More recommend
