IO virtualization Michael Kagan Mellanox Technologies IO - - PowerPoint PPT Presentation

io virtualization
SMART_READER_LITE
LIVE PREVIEW

IO virtualization Michael Kagan Mellanox Technologies IO - - PowerPoint PPT Presentation

IO virtualization Michael Kagan Mellanox Technologies IO Virtualization Mission non-stop services to consumers Flexibility assign IO resources to consumer as needed Agility assignment of IO resources to consumer when


slide-1
SLIDE 1

IO virtualization

Michael Kagan Mellanox Technologies

slide-2
SLIDE 2

2

IO Virtualization

  • Mission – non-stop services to consumers

Flexibility – assign IO resources to consumer as needed Agility – assignment of IO resources to consumer when needed Isolation – deliver IO services to consumer without interference Fault-tolerance – deliver IO services despite HW failures

Reliability, flexibility & efficiency

  • Mean – de-couple service delivery from the Hardware

Overcome Hardware boundary constrains Resource assignment is a matter of mouse-click Consumer-transparent hardware replacement

De-couple consumers’ execution environment from underlying Hardware

slide-3
SLIDE 3

3

Simple (monolithic) IO model

  • Kernel-resident mngr owns HW
  • Initialization, configuration
  • Management interface (API)
  • OS is part of IO subsystem
  • Isolation, protection, QoS
  • Constrained performance
  • Kernel call, data copy etc.
  • Low QoS granularity
  • Agility? Flexibility? Fault tolerance?
  • Dumb IO device

cAPI – soc, MPI etc mAPI – PCI cfg, pwr etc

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

slide-4
SLIDE 4

4

Advanced IO model

  • Kernel-resident mngr owns HW
  • Initialization, configuration
  • IO delivery policy provisioning (QoS, protection)
  • IO delivered to consumers via IO channels – OS is not a part of IO subsystem
  • Abstract interface to post IO operation requests
  • HW-enforced cross-channel protection and isolation
  • Unlimited scalability
  • High performance
  • Kernel bypass, zero copy etc.
  • Fine QoS granularity
  • Agility? Flexibility? Fault tolerance?
  • Smart IO device

cAPI – soc, MPI etc mAPI – PCI cfg, pwr etc

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

slide-5
SLIDE 5

5

IO in virtual machines – take 1

  • Hypervisor-resident mngr owns HW
  • IO delivered to guest via IO device emulation

“mng” part is faked; “service” path used for IO operation

  • Improved agility, flexibility and fault tolerance
  • Additional SW layer on the IO service path
  • Cross-domain isolation and protection in Hypervisor
  • Virtualization cost – Guest OS and Hypervisor are part of IO subsystem
  • Lower IO performance
  • Higher CPU overhead
  • Limited scalability – architecture tied to HW
  • Low QoS granularity
  • Dumb IO device

IO device

Driver mng mAPI mngr

VM

cAPI

kernel user

cAPI

Hypervisor

Driver mng mAPI mngr Driver mng mAPI mngr

app app app app service

VM

cAPI

kernel user

cAPI

Driver mng mAPI mngr Driver mng mAPI mngr

app app app app

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

slide-6
SLIDE 6

6

IO in virtual machines – take 2

  • Hypervisor-resident mngr owns HW
  • IO device(s) configuration, IO policy provisioning
  • IO delivered to consumers via IO channels
  • IO delivery is an integral part of the platform
  • Uncompromised IO services delivery
  • Unlimited scalability (“IO channel for every socket”)
  • HW-enforced isolation and protection
  • Direct IO channel access (pass-through)

Hypervisor offload on data path, zero copy etc.

  • Fine QoS granularity
  • Allows on-demand resource allocation
  • Standard IO interface to enable transparent servers’ management
  • Smart IO device

cAPI – soc, MPI etc mAPI – PCI cfg, pwr etc

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app service

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app

IO device

app app Driver mng cAPI mAPI mngr

kernel user

cAPI app app

IO device

Driver mng mAPI mngr

VM

app app cAPI

kernel user

cAPI app app

Hypervisor

VM

app app cAPI

kernel user

cAPI app app

Driver mng mAPI mngr Driver mng mAPI mngr Driver mng mAPI mngr Driver mng mAPI mngr

slide-7
SLIDE 7

7

Open Fabrics Channel IO Interface

  • Industry-standard channel IO interface

Included in Linux kernel Endorsed by Microsoft

  • Supports multiple IO devices

InfiniBand HCAs Ethernet NICs

  • Standard IO provisioning interface

Assign QoS for an IO channel or group of channels

  • Efficient and versatile IO delivery

Enables direct access to IO HW – full Hypervisor offload Raw datagram IO channels Transport-offloaded IO channels RDMA IO channels

  • Unlimited scalability

Unlimited number of IO channels

slide-8
SLIDE 8

8

Channel IO in virtual machines

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user

Driver mng mngr

IO device

app app

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user Hypervisor

WR formatter

app app

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user

Driver mng mngr

IO device

app app

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user Hypervisor

app app mngr

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user

app

VM

core (framework) WR formatter

app

core (framework) WR formatter

app

kernel user Hypervisor

app app

IO device

Driver mng service

Pass Pass-

  • through

through Hypervisor acceleration Hypervisor acceleration Simple IO device Simple IO device

  • Hardware-independent consumer-transparent guest migration

Support direct access to HW

app

slide-9
SLIDE 9

9

Summary

  • Channel IO architecture implements IO virtualization mission

Flexibility, agility, isolation, management at high performance

  • Channel IO production HW available

Multiple generations, multiple vendors

  • Channel IO API standard materializing

Native support by standard OSes Supported on proprietary OSes

  • Channel IO serves IO consolidation trend

e.g. FC over Ethernet

  • Channel IO enables on-demand resource allocation

IO channels, IO resources, memory