Remote Teleoperation Tejas Sathe The Matrix The Papers Usability - - PowerPoint PPT Presentation

Remote Teleoperation

Tejas Sathe

The Matrix

The Papers

Usability Guidelines for the Design of Robot Teleoperation: A Taxonomy An Experimental Analysis of Cyber Security Threats Against Teleoperated Surgical Robotics

Usability Guidelines for the Design of Robot Teleoperation: A Taxonomy

Abstract

A taxonomy of design guidelines for robot remote teleoperation
 A list of UI design guidelines, classified by open card sorting
 Taxonomy further validated by closed card sorting

Classification of guidelines

Platform architecture and scalability Error prevention and recovery Visual design Information presentation Robot state awareness Interaction effectiveness and efficiency Robot surroundings awareness Cognitive factors

Introduction

Robots can guide themselves with limited human expertise encoding
 For high criticality situations, partial encoding my not be enough
 For example, agricultural robot cannot adjust spraying according to specific needs of the plant
 Suitable human-robot interface needed

Open card sorting

Participants given cards with no groupings and asked to classify
 Participants: 6 female and 16 male experts
 Apparatus: Sorting done using WebSort online service
 Procedure: Drag and drop cards in appropriate groups

Closed card sorting

Used to test and refine proposed taxonomy obtained from earlier method
 Give cards with primary groups and classify into preestablished groups (from earlier)

Closed card sorting

Participants: 20 female and 18 male experts (23 HCI + 15 HRI)
 Apparatus: OptimalSort online service
 Procedure: guidelines and categories presented in random order; drag and drop

Analysis of Open Sorting

Used for generating taxonomy
 WebSort delivers categorizations generated both individually and cumulatively
 Three of the authors created their own categorizations for the data

Analysis of Open Sorting

One author used nominalizations to merge several groupings
 e.g. ‘Look’, ‘Orientation’, ‘Video Stream’ etc. were collectively grouped as ‘Viewing and Navigation’
 This resulted in 6 metacategories
 The 70 guidelines were then placed in these

Analysis of Open Sorting

e.g. Metacategory A contains 1, 2 and 3
 For each guideline, a sum indicates how many times it was placed in one of these
 If the sum was maximum among all subcategories, the guideline was assigned to A

Analysis of Open Sorting

The second author used the dendogram created by WebSort to group labels
 e.g. “Error Prevention” and “Provide Feedback” were placed under the standardized label “Design for error prevention and recovery”

Analysis of Open Sorting

The third author merged linguistically and conceptually similar groups into one single standardized group
 e.g. “Interface efficiency”, “Design efficiency”, “Interaction efficiency” etc. were placed under “User experience efficiency”

Analysis of Open Sorting

An analysis of a matrix with guidelines as rows, categories as columns and cells as percentage of participants who placed that guideline in that category was conducted
 Column-wise exploration of the same in conjunction with WebSort’s dendogram identified the group consistently formed by the participants consisting of the same guidelines

Analysis of Open Sorting

e.g. Guidelines like “Extensibility of system” and “Support evolution

• f platforms” were placed under “Platform architecture and

scalability”
 Row-wise exploration was used to place the guidelines in the newly formed group

Analysis of Closed Sorting

An overall agreement of 86% was observed between the results of the open and closed sortings
 For each guideline, a comparison of percentage of participants was done who placed it in different group in the open and closed sortings
 A statistically significant difference was found for only 10 out of 70 categories; this was found using two-sided two-proportion z-test, p<0.05 as the standard value

Final taxonomy

The final taxonomy had to be refined
 This was done by moving those ten guidelines into categories that the maximum number of participants chose for each of them

Application study

The proposed taxonomy was applied to an agricultural sprayer robot, AgriRobot
 The evaluated prototype implemented functionality for robot navigation and targeted spraying
 It was implemented using the Summit XL mobile platform, with three cameras for human-robot awareness

Application study

Four HRI experts conducted heuristic evaluation of the AgriRobot using the proposed taxonomy and severity rating proposed by Nielsen
 First, they studied the proposed taxonomy and carried out a free exploration on their own to identify usability problems
 Each expert then compiled a list containing problem description, the heuristic violated and a corresponding severity rating

Application study

They then shared notes, removed duplicates and refined problem descriptions and reordered the list by decreasing order of severity

Conclusion

This is the first endeavor to produce a taxonomy that takes inputs from HRI as well as HCI experts
 This taxonomy focuses specifically on the UI design aspect of robotics
 Several problems that were missed by UI experts in the AgriRobot were identified using this taxonomy

Audience response

What did we like about this paper?
 What didn’t we like?
 Are the conclusions appropriate?
 How would we improve on this paper?

An Experimental Analysis of Cyber Security Threats Against Teleoperated Surgical Robotics

Abstract

Surgical robots are being tested and used; what if they were hacked and turned into weapons?
 This paper analyzes possible cyber security attacks against Raven II, an advanced teleoperated surgical robotics system
 They identify various threats and evaluate scopes and impacts
 The also investigate methods of mitigating the damage

Introduction

There has been a 20% increase yearly in the number of surgical robots sold
 These are expected to use of existing public networks and temporarily available ad-hoc wireless and satellite networks to transmit audio and video
 These can provide immediate relief in under-developed and remote rural areas; also in the battlefield

Security concerns

Security hasn’t been a major concern for medical robotics so far
 But worms like Stuxnet can affect any devices that use a PLC
 So, security does become an important aspect of remote surgery (but so is the case with literally everything)

Stuxnet

A worm that exploited four zero-day vulnerabilities in the Windows OS; introduced into the device using a flash drive
 It propagates across the network, looking for Siemens Step7 software that controls a PLC
 If it doesn’t find it, it lays dormant, with the possibility of transmitting every time a flash drive is inserted into that device

Stuxnet

If a Siemens Step7 is found, Stuxnet introduces an infected rootkit on the PLC and Step7
 This modifies the code and gives unexpected commands and disrupts the functioning
 It also simultaneously transmits acceptable readings to the monitoring system
 Camera feed example

The Problem

Currently, surgical roboticists face two dimensions of lack of understanding
 The first is how such a system can be attacked
 The second, what the motivation behind doing this might be

The Analysis

Attack identification and characterization
 Vulnerability analysis
 Risk assessment and defense directions
 Challenges specific to teleoperated procedures

Related Work

The Raven II is a teleoperated robotics system to support research in advanced robotic surgery
 It can support both software development, experimental testing, and surgical training

Raven II

Raven II

The Raven II consists of two 7-degrees-of-freedom surgical manipulators
 The motion axes are: shoulder joint, elbow joint, tool insertion/ retraction, tool roll, tool grasping, tool wrist actuations 1 and 2

Raven II

The surgeon control inputs are collected through a surgical control console
 Control inputs and robot feedback are transmitted using a communication standard designed specifically for surgical teleoperation
 It’s called the “Interoperable Telesurgery Protocol”, or ITP

Raven II

The Raven II has been tested in extreme environments, like the Mojave desert
 It was controlled through the Internet, with the final link being a UAV- enable wireless network
 The following network states were identified as critical for performance: comm latency, jitters, packet delays/out-of-order arrival/losses, device failures

Vulnerability Analysis: Attacker Model

Vulnerability Analysis: Attacker Model

Two attack vectors are identified: endpoint compromise and network/communication based attack
 In the former, a surgeon’s control console or the robot can be compromised
 In the latter, the attacker may intercept network traffic and inject malicious packets

Vulnerability Analysis: Attacker Model

Endpoint compromise is a physical attack; you have to be present at an endpoint to compromise it
 This can be remedied by physical monitoring
 Network/communication based attack is a non-physical attack; it can be done remotely
 Due to the variety of such attacks and compromise points, remedying this can be much more intellectually challenging

Attack Classification

Intention modification: Intention of the command modified by modifying sent packets
 Intention manipulation: Manipulation of robot feedback that makes surgeon send a wrong command
 Hijacking: Robot completely ignores surgeon’s commands and instead performs some other, potentially harmful operation

Attack Classification

Network observer: Miscreant initially eavesdrops on network communication, and starts inserting false messages from both ends (like Stuxnet)
 Network intermediary: MitM attack, prohibits direct communication between surgeon and robot; can be done using ARP poisoning

Experimental Analysis: Setup

Experimental Setup: Description

Data collected from twenty human participants
 All were Electrical Engineering and CS students; aged 19 to 28
 They were put through the FLS Block Transfer test

Experimental Setup: Description

This test involves the subject using a robot’s arm to move blocks one at a time from the left side of a peg board to the right
 Then they are moved back to the left
 In the first, the block is picked up with the left arm, transferred in the air to the right, and placed with the right arm
 Exactly opposite for the reverse process

Experimental Setup: Description

Since the security was under the microscope rather than the subjects’ proficiency, they were asked to perform a reduced version

• f the test


They were asked to move only three blocks, only from left to right, without needing to transfer midair

Intent Modification Attacks

Reordering: simple random reordering of messages sent by the surgeon; Raven skips out-of-order messages, resulting in jerky action
 Intent Loss: randomly drop packets, resulting in jerky motion; loss rate

• f above 0.55 makes it difficult to use, and is practically inoperable

above 0.9

Intent Delay

Intent Modification Attacks

Modify the surgeon’s packets on the fly
 Forward them to Raven through the attacker’s proxy
 Changing command changes in positions and rotations, inverting grasping states, inverting right and left, random scaling of commanded changes in position and rotation

Intent Modification Attacks

Hijacking attacks

Sequence number leading: read a packet, extract sequence number, add random

• ffset and send; robot attributes this to large packet loss; executes leading packets,

which are malicious
 Force reset: prevent robot arms from moving too fast or outside an area; whenever a command that needs this is issued, system-wide halt is imposed, aka E-stop
 E-stop is actually designed to protect the robot from mechanical and electric failure; but this can be misused by sending malicious packets that order such behavior

Mitigation Strategies

Encrypt communication stream end-to-end, rendering MitM attacks impossible
 Since dedicated staff can exist at both ends, encryption and authentication are both possible; out-of-band communication like texting
 AES was used with 128, 192 and 256-bit keys; no significant increase in CPU usage was observed; there was increase in memory usage, which is acceptable

Implications of presented attacks

Attacks pose significant risk to surgeons, patients and robots; a compromised robot can cause a lot of damage in even a routine surgery
 Patient recovery can slow down due to incorrect procedures being performed
 Hacking robot feedback can violate patient privacy

Implications of presented attacks

For surgeons, a compromised robot can cause legal repercussions
 Intent modification can make it look like the surgeon deliberately tried to harm the patient
 This can make them lose their medical license

Implications of presented attacks

For the robot, malfunction can cause it to break
 It may also damage other, unrelated equipment
 From the patient’s perspective, the advantages of teleoperated surgery are not worth the risk of being operated on by a compromised robot

Challenges and Recommendations

The E-stop which is a protective feature, may be used in a malignant way; it’s challenging to reconcile the benefits of the E-stop with the potential of it being used as an attack
 Receiving packets in bursts may cause jerky motion and pose a threat to the patient; to protect against burst attacks, the packet processing rate should be limited
 For fast operation, datagram protocols are used; while packet losses can be tolerated, an attacker may cause large packet losses which poses a serious threat

Challenges and Recommendations

Encrypting video feedback may induce delays which can be catastrophic in time critical surgeries; at the minimum, each packet should be authenticated
 ITP should be continuously updated to avoid sequence number and checksum attacks
 The communication link and network should be continuously monitored by competent people

Conclusion

Raven II is vulnerable in several places, all of which the authors were able to breach
 Some of these can be easily prevented by encryption and authentication
 Trade-off between speed, security and usability needs to be effectively handled

Audience response

What did we like about this paper?
 What didn’t we like?
 Are the conclusions appropriate?
 How would we improve on this paper?

Thank You!