Supervision by observation using inductive programming José Hernández-Orallo (project leader: Carlos Monserrat) Departament de Sistemes Informàtics i Computació, Universitat Politècnica de València, Spain. jorallo@dsic.upv.es AAIP’2015 – Approaches and Applications of Inductive Programming, Dagstuhl Seminar 15442, October 25-30 1
• Task automation vs. task supervision • Project SuPERVaSION • Application domain: surgical training Outline • Capture and representation: first ideas • Related works, expressions of interest 2
Task automation vs. task supervision One of the major applications of inductive programming is the automation of repetitive tasks from examples. A significant progress has recently taken place. Many problems still look too challenging for current techniques. 3
Task automation vs. task supervision Automatic task supervision when the superviser learns from expert examples and compares with an apprentice. Don’t forget the other cup! 4
Task automation vs. task supervision Different problems (but related). Key differences. For task supervision: In principle, there is no need of learning the task completely , just some key steps that can be traced and identified when a novice or an operator is doing it. Several ways of solving the task are possible . The supervisor must be able to consider all types of solutions. Correction and feedback is also a possibility in supervision. 5
Project SUPERVaSION SuPERVaSION : “Automated supervision by observation: pervasive technology for autonomous skill acquisition and procedure execution assistance” . Funded as an Explora project (2015-2016) for risky, challenging ideas. “We envisage automated assistants that after the observation of how an expert performs a task are able to supervise whether other humans are performing the task correctly, also by observation” Some IP applications, especially in the area of learning assistants and education, have addressed this kind of problem 6
Project SUPERVaSION Need and impact: “Many tasks may be easily spoilt or may lead to suboptimal results by a human mistake that deviates from the procedure or the demonstration. A supervision system would be able to detect and advice the operator in real time”. “Many tasks are learnt by humans more efficiently if these have continuous supervision and get meaningful comprehensible feedback”. “Quality control and teaching planning could be improved significantly by a recollection of how the procedures are performed at a high level and the effect of the feedback over the operators”. Coding supervisors manually is repetitive and expensive, and may not cover all the possible ways of carrying out the task. 7
Application domain: surgical training Training Minimally Invasive Surgery (Laparoscopes): Students learn the procedure from a description and a demo. Students must repeat the procedure several times. Students learn faster with supervision. Practice is done with a virtual simulator or a box-trainer. Box-trainers are much cheaper and tactile feedback is real. Virtual simulators are very expensive. Not enough availability for students. Virtual simulators usually incorporate tasks and low-level supervision. 8
Application domain: surgical training Automatic supervision in this domain for virtual surgery simulators has been attempted in different ways: Markov processes String similarity (longest common subsequence algorithm): “Automatic supervision of gestures to guide novice surgeons during training” C. Monserrat, A. Lucas, J. Hernández-Orallo, M. José Rupérez, Surgical Endoscopy (2014) Using a character coding for gestures: This is very low-level. Ignores the high-level description. 9
Application domain: surgical training Exercises have more information: DESCRIPTION: VIDEO: 10
Application domain: surgical training Approach with inductive programming Provide start-up declarative knowledge about the domain, Analyse these logs and suggest segmentation, Identify groups and find repetitive structures, Turn them into high-level actions that represent a program, Possibly make them available to the expert, Perform a similar approach for each trainee performing the same task, locating matches and mismatches, and Producing high-level online feedback for the user. 11
a 12
Capture and representation Scene capture Using track dots in the trainer-box when recording. Positions are analysed and recorded for the key objects. High-level knowledge representation Event-Action declarative languages, such as event calculus and variants. For instance, Nikos Katzouris, Alexander Artikis, Georgios Paliouras “Incremental learning of event definitions with Inductive Logic Programming” Machine Learning 100:555 – 585, 2015. Uses XHAIL: Ray, O. (2009). Nonmonotonic abductive inductive learning. Journal of Applied Logic , 7 (3), 329 – 340. 13
Related works, expressions of interest These (Explora) projects are meant to analyse a challenging problem and see whether the proposed approach is feasible. If the initial analysis (and possible prototype) is successful they usually lead to larger consortia and projects. Similar projects, ideas, approaches and papers are welcome! Or… it can be seen as a CHALLENGE for the IP community! Anyhow, approach me during the coffee break! 14
Recommend
More recommend
