Next generation Ethernet connect to All FLASH: iSER and NVMeF Subhojit Roy Senior Technical Staff Member, IBM IBM Systems
Agenda § Flash growth and dimensions of change § Shared SAN storage requirements § What is iSER? § What is NVMeF? § All FLASH on SAN – roadmap § What are we doing at IBM? IBM Systems | 2
Flash Growth and Dimensions of Change… Traditional Enterprise Workloads à New age Flash workloads Workloads (Tier0: SAP HANA, RT Analytics, Tier1: OLTP, VDI, Social Media Apps) Kernel Mode à User Mode (SPDK) Storage SW Architecture FCP à iSER & NVMeoF (Eth & FC) Upper Level Protocols (ULP) FC 8G/16G à Eth 10G à FC32G à Eth RDMA 25/40/50/100G Layer2 Interconnect (L2) HDD à SAS SSDs à PCIe NVMe à PM (3DXP) Storage Media AF Server à AF-HyperConverged à AF-Arrays à AF Disaggregated Storage Storage HW Architecture ü Change is multidimensional ü Flash is driving change ü Change would not be all at once. One dimension at a time IBM Systems
Shared SAN storage requirements Ø Multipath (for high availability) Ø Reservation (SCSI Persistent Group Reservation) Ø vVols (offload Copy Services to Storage Controller) Ø Data movement/migration without involving host CPU (XCOPY, ODX) Ø Unmap: Optimize space deallocation on Thin Provisioned storage Ø Atomic Compare And Write (CAW): Alternative to SCSI3 Reservations Ø Error Handling: Abort Task, Abort Task Set, LUN Reset, Target Reset etc. IBM Systems | 4
What is iSER? Operating System/Applications SCSI Layer FCP iSCSI Software Drivers iSER TCP Ω OFED IB verbs IP Fiber Channel Infiniband NIC Driver iWARP Driver RoCE Driver Driver Adapter Driver Hardware iWARP rNIC RoCE rNIC IB HCA FC HBA NIC 5 iSER FC iSCSI
iSER: Current state of affairs iSER is iSCSI with RDMA Data path Suitable for All FLASH over High Speed Eth Low Latency, Low CPU Utilization (Eliminates copies to/from TCP/IP buffers) (10, 25, 40, 50, 100 Gbps and beyond) No Changes to iSCSI administration No disruption to administration model (vSphere, Widows, OpenStack work as is) Vendor and Technology Independent Fits well into Software Defined Storage (works on iWARP , RoCE & Infiniband HCAs) (SDS) paradigm Works on Standard Ethernet equipment Cost Savings (10G and 25/50/100G switches) Enterprise applications just work! Ideal for shared storage (vVols, Clustering, Multipath etc.) (both for FLASH and HDD) iSER is ready for Shared All FLASH SAN storage today! IBM Systems 6
What is NVMeF? Operating System/Applications SCSI NVMeF iSCSI Software Drivers iSER OFED IB verbs FCAdapter Infiniband iWARP Driver RoCE Driver iWARP Driver RoCE Driver Driver Adapter Driver Hardware FC HBA iWARP rNIC RoCE rNIC IB HCA iWARP rNIC RoCE rNIC 7 iSER NVMeF
NVMeF: Current state of affairs Suitable for All FLASH over High Speed Eth Low Latency, Low CPU Utilization (Primarily scuts down on Host s/w stack) (10, 25, 40, 50, 100 Gbps and beyond) Changes to vSphere, Openstack etc. New Administrative Model Vendor and Technology Independent Yes and No! Need common user space (works on iWARP , RoCE & Fiber Channel) layer 2 Works on Standard Ethernet equipment Cost Savings (10G and 25/50/100G switches) Applications must change to exploit Applications need to transform yet parallelism NVMeF is still evolving to adapt to Shared Storage Applications! IBM Systems 8
All Flash SAN roadmap! Flash/3DXP Flash/3DXP Flash/NVMe Flash/NVMe HDD HDD/SSD HDD SCSI/NVMeoF NVMeoF SCSI SCSI SCSI SCSI SCSI Technology iSER/NVMf iSER/RDMA NVMf/RDMA FC FC FC iSCSI/Eth NVMeoF/Eth SCSI/FC iSCSI Flash PCIe NVMe iSER came as NVMeoF RDMA ruled the started environments Flash debuted an alternative specification matured for Enterprise penetration used SCSI as high to FC for and Usecase Shared Shared in low end with FC as an performance connecting technology Details Storage Storage market interconnect storage external Flash matured for Usecase World Tier0 usecase Timeline …. …. …. …. For 2012- 2019 < …. 2010 …. 2017 2016- 2015- Maturity 2016 …. 2010 +…. …. …. … Adoption Status 1990s – 2010 : HDD/SCSI/FC Rule 2007- 2016 SSD/Flash media evolution, maturity 2015 - … NVM evolution 2016 to 2020 – iSER/SCSI 2019/2022 –iSER/NVMeF IBM Systems
What are we doing at IBM? Host Host Host Host Host Ø Host Attach over iSER – Linux, VMWare Host attach over iSER Host SAN Ø Clustering over iSER Vdisk 1 Vdisk 1 Vdisk 1 Vdisk 1 Host Host Host Host SVC SVC SVC SVC Clustering of Nodes on iSER Device SAN RAID controller LUNS RAID RAID RAID RAID Ctlr Ctlr Ctlr Ctlr IBM Systems
iSER vs Fibre channel I/O iSER (40Gb) Fibre Channel (16Gb) 50 (us) 80 (us) Read 4KiB 139 (us) 195 (us) Write 4KiB 95 (us) 196 (us) Read 64KiB 209 (us) 337 (us) Write 64KiB iSER: Fiber Channel benefits minus the additional costs 11 IBM Systems
Thank You! IBM Systems 12
Recommend
More recommend
