Peripheral State Persistence and Interrupt Management For - - PowerPoint PPT Presentation

Peripheral State Persistence and Interrupt Management For Transiently Powered Systems

Gautier Berthou, Tristan Delizy, Kevin Marquet, Tanguy Risset, Guillaume Salagnac

Citi Lab, INSA Lyon, France

NVMW, San Diego, March 12th 2018

Context: Transiently Powered Systems

Internet of Tiny Things

• Internet of Things ◮ networked embedded systems
• No battery ◮ harvest power

smart cards RFID tags wearable sensors

• G. Berthou

2/23

Context: Program checkpointing

Power in Energy Harvester + buffer

CPU RAM NVM

Power failure detection

Program Checkpointing:

• Anticipate power failures
• Save program state to a

non-volatile memory

• Restore state on next boot

◮ Idea: add “OS” code to hide checkpointing to the application

Supply voltage Time Vboot Vdeath Off time Lifecycles Vtrigger

• G. Berthou

3/23

Typical checkpoint structure

Application state

• Copy of variables
• Copy of application stack
• Copy of CPU registers
• Contains all relevant data to

enable application persistence

• Stored in non-volatile memory
• G. Berthou

4/23

Checkpointing for Transiently Powered Systems

CPU RAM

Flash

Mementos [Ransford et al ’13]

CPU

Quickrecall [Jayakumar et al ’14]

NVRAM

DINO [Lucia & Ransford ’15] HarvOS [Bhatti & Mottola ’16]

CPU RAM

NVRAM

[Ait Aoudia et al ’14] [Balsamo et al ’15, ’16]

NV CPU

NVRAM

[Liu et al ’15]

• G. Berthou

5/23

Outline

Introduction: Context and State of the Art Peripheral State Persistence Peripheral State: Volatility Problem Peripheral Access: Atomicity Problem Interrupt handling Experimental Results Conclusion and perspectives

• G. Berthou

6/23

The Peripheral State Volatility Problem

Application code

void main(void){ sensor_init(); rf_init(myconfig); for(;;){ v = sensor_read(); rf_send(v); ... } } App Drv HW rf send

Restoring memory content will not restore device state

• G. Berthou

7/23

Our approach: distinct roles for OS and drivers

Each driver provides:

• restore() function
• device context

Operating System:

• checkpoints device contexts
• calls every restore() function
• checkpoints application state

OS Drv HW App restore

• G. Berthou

8/23

The Peripheral Access Atomicity Problem

Application code

void main(void){ sensor_init(); rf_init(myconfig); for(;;){ v = sensor_read(); rf_send(v); ... } } Drv HW App rf send

In most cases, resuming execution in the middle of a hardware access does not make sense

• G. Berthou

9/23

Our approach: make driver calls atomic

Encapsulate driver functions into OS wrappers. Each driver provides a save() function that copies device context into checkpoint image. On wrapper entry:

• save arguments + function called
• switch to volatile stack

On wrapper exit:

• save device contexts
• clear arguments
• switch back to application stack

OS App wrapper entry Drv + HW wrapper exit

• riginal

driver function

drv save()

Interrupted driver calls are retried and not just resumed.

• G. Berthou

10/23

Outline

Introduction: Context and State of the Art Peripheral State Persistence Peripheral State: Volatility Problem Peripheral Access: Atomicity Problem Interrupt handling Experimental Results Conclusion and perspectives

• G. Berthou

11/23

Interrupt-related problems

Problems not specific to transiently-powered systems:

• Concurrency
• Race conditions

◮ Solution: critical sections with interrupts enabled. Specific to transiently-powered systems:

• Interrupt data volatility
• G. Berthou

12/23

Interrupt data volatility

• Interrupt occurrence is

volatile data.

• Peripheral data, e.g., radio

packet content, are also volatile data. App

• Int. A

handler Interrupt A Peripheral data is ready

• G. Berthou

13/23

Our approach: extend solution to peripheral state volatility problem

OS-managed top halves and user-managed bottom halves

• Each driver provides an
• n_interrupt() routine.
• Each top half calls the
• n_interrupt() routine of

relevant drivers.

OS App Drv + HW extracts data

drv save()

• n interrupt

save scheduler bottom half

• G. Berthou

14/23

Checkpoint structure

Application state

• Copy of variables
• Copy of application stack
• Copy of CPU registers

OS state

• Running driver call if any
• Address and arguments
• Interrupt occurrence

Drivers state

• Driver A device context
• Driver B device context
• ...
• OS and drivers states allow

peripheral persistence

• G. Berthou

15/23

Outline

Introduction: Context and State of the Art Peripheral State Persistence Peripheral State: Volatility Problem Peripheral Access: Atomicity Problem Interrupt handling Experimental Results Conclusion and perspectives

• G. Berthou

16/23

Sytare Evaluation Setup

• MSP430FR5739: 16-bit

CPU 24MHz, 1kB SRAM, 15kB FeRAM

• RF-chip: CC2500

Application

void main(void){ syt_sensor_init(); syt_rf_init(myconfig); for(;;){ v = syt_sensor_read(); compute(); syt_rf_send(v); ... } }

• G. Berthou

17/23

Evaluation methodology

Experimental setup

• Varying parameter: lifecycle

duration Ground-truth

• Same application without OS

layer

• Stable supply without outage

Supply voltage Time Vboot Vdeath Off time Lifecycles Vtrigger

• G. Berthou

18/23

Evaluation methodology

Performance metrics

• Duration of shortest usable lifecycle
• Temporal execution efficiency

Efficiency(x) = TGT T(x) x Lifecycle duration TGT Application runtime under ground-truth conditions T(x) Application runtime with OS layer when the platform is ON

• G. Berthou

19/23

Efficiency results

200 400 600 800 1,000 0.2 0.4 0.6 0.8 1 Tmin = 3 ms TGT Lifecycle duration (milliseconds) Efficiency

• G. Berthou

20/23

Results: Driver call temporal overhead

Driver calls are encapsulated into wrappers Driver call Overhead (%) Led toggle 1263 ADC read 27 Radio sleep 137 Radio wake-up 8 Radio send 1

• G. Berthou

21/23

Outline

Introduction: Context and State of the Art Peripheral State Persistence Peripheral State: Volatility Problem Peripheral Access: Atomicity Problem Interrupt handling Experimental Results Conclusion and perspectives

• G. Berthou

22/23

Conclusion and perspectives

Peripheral State Persistence for Transiently Powered Systems

• Volatility: device contexts
• Atomicity: retry rather than resume
• Interrupts: save data as soon as possible

Project sources: https://gitlab.inria.fr/citi-lab/sytare-public

Perspectives

• Reduce overhead of driver calls
• Adapt peripheral checkpointing to existing OS (RIOT, Contiki)
• WCEC evaluation of peripheral accesses
• Energy-based decision making
• G. Berthou

23/23

Checkpoint structure - Completed

Application state

• Copy of variables
• Copy of application stack
• Copy of CPU registers

OS state

• Running driver call if any
• Address and arguments
• Interrupt occurrence

Drivers state

• Driver A device context
• Driver B device context
• ...

Current Last

• Information about interrupt
• ccurrence are kept in the OS

section of the checkpoint image.

• Data carried by interrupts are

kept in the relevant device

• drivers. Ex: radio packet content

is owned by the radio chipset driver.

• G. Berthou

0/-1

Complex driver call

App OS drv A drv B App Drivers Modified drivers App Drivers Kernel App Drivers Kernel

Sequence diagram SRAM state NVRAM Checkpoint Next Image state NVRAM Checkpoint Last Image state

App 0 App 1 App 1 App 1 App 1 App 1 App 1 App 2 ∅ { B } ∅ ∅ ∅ { B, A } ∅ ∅ A 0 B 0 A 0 B 0 A 1 B 0 A 0 B 0 A 1 B 1 A 2 B 1 A 2 B 1 A 2 B 1 ∅ A 0 B 0 App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ App 0 A 0 B 0 ∅|∅ user main() syt drvA fn(x) drvA fn(x) drvB fn(y) ∅ A 0 B 0 ∅ A 0 B 0 drvA fn|x ∅ A 0 B 0 drvA fn|x ∅ A 0 B 0 drvA fn|x ∅ A 0 B 0 drvA fn|x ∅ ∅|∅ A 2 B 1 ∅|∅ ∅|∅ ∅ ∅|∅ A 2 B 1 Modified state Unmodified state mark dirty() mark dirty() drvB save() drvA save()

• G. Berthou

0/-1

Top halves and bottom halves

Interrupt handling is split into OS-managed top halves and user-managed bottom halves.

• Power loss is handled by top halves, making it possible to

hide power loss from user. Design choices: two axes

1 Bottom half nestedness? No 2 Allowance of hardware operations being called from

bottom halves ? Yes

• G. Berthou

0/-1

Data race conditions

Application code

static int x; bottom_half() { ++x; } main() { x = 0; ... if(x == 4) { ... } }

• G. Berthou
• 1/-1

Data race conditions

Problem

• Bottom halves may share data with user application: global

variables. Solution

• User-defined critical sections that disable bottom halves

but keep interrupts enabled.

• Bottom halves are delayed until the end of the critical

section.

• Interrupts are enabled, which makes the sytem reactive

upon power loss detection.

• G. Berthou
• 1/-1

Peripheral access race conditions

OS Drv SPI App syt spi config(A) spi config(A) Interrupt A

Top half

spi config(B)

SPI has config A?

• G. Berthou
• 1/-1

Peripheral access race conditions

Problem

• Interrupts might occur during a hardware access initiated

by user.

• Top half might use the same peripheral and put it in an

inconsistent state with respect to the application. Solution

• Provide lock mechanism, accessible from the user, who

indicates to the OS which peripherals are locked.

• When an interrupt occurs, if the top half tries to use a

locked peripheral, both the top and bottom halves are discarded.

• G. Berthou
• 1/-1

Peripheral access atomicity

Problem

• Interrupts might occur during a hardware access initiated

by user. Solution

• Rerun driver call call from the beginning when returning

from interrupt.

• Makes the driver call management policy consistent with

power loss detection occurring during driver call execution.

• Pessimistic approach that leads to time and energy
• verhead.
• G. Berthou
• 1/-1

Operating System architecture - Completed

Drvcall App Interrupt Empty queue and app running Interrupt Empty queue and syscall running Drv op. Ret Kernel Interrupt top half Bottom half Interrupt

• r

bot. half completion

• Bot. half Drvcall

Interrupt Drv op. Ret

• Bot. half queued or

top half completion Top half completion Interrupted during App Drvcall

• Bot. half
• Bot. half

Drvcall Boot Restore Init Checkpoint Power loss detection Not first boot First boot Actual power loss + Hardware reboot

Kernel Driver User application

• G. Berthou
• 1/-1

System boot sequence

Port Clock ADC SPI Radio Hardware boot 0.75 ms App state restoration 45µs Device context restoration 27µs Peripheral state restoration 1.17ms Next checkpoint initialization 30µs power-up application execution

• G. Berthou
• 1/-1

TPS example architecture : ST23ZL48 microcontroller

Ai12565

Internal Bus 2 I/Os IART GND 8/16-bit CPU Core 3 x 8-bit timers CRC Module CLK Clock Gene- rator Module Vcc True Random Number Gene- rator RAM ST ROM

(Boot software)

Security Monitor- ing and Control RESET EEPROM EDES Accelerator User ROM ST ROM Firewall NESCRYPT NESCRYPT RAM MPU

http://www.st.com/en/secure-mcus/st23zl48.html

• G. Berthou
• 1/-1
• 16-bits CPU

(27MHz)

• 8kB RAM
• 300kB ROM
• 48kB EEPROM