Scaling NFV - Are containers the answer? Azhar Sayeed - asayeed@redhat.com Doug Smith - dosmith@redhat.com
Acknowledgements This is a result of mul7ple efforts in Red Hat on Containers and Container Networking. We would like thank everyone who helped us put this POC, demo and presenta7on together. A big thank you to Dan Williams (dcbw@redhat.com) and Rashid Khan (rkhan@redhat.com) for listening, • being pa7ent with us and for building a prototype that is really powerful Dan for whipping up slides and code in a maKer of 8 weeks. • Ajay Simha (asmiha@redhat.com) for his review and contribu7ons to the presenta7on • and the work he was doing with Doug Smith to build a POC Tomofumi Hayashi for his work on koko (Container Connector) basis for the demo • 2
Agenda Introduc7on • Telco Requirements for NFV scale • Containers - how can they help ? • Scale ques7ons • Do they solve the problem ? • Issues and Challenges • Demo • Summary • 3
Virtualiza7on Progression Applica7ons and Network Func7ons Virtualized Apps in - VMs Bare metal Virtualized Apps in Containers Containers in VMs and VMs in Containers?
NFV - Use cases and scale vCPE - vCPE/SDWAN Residen7al Wireline Mobile vEPC/vIMS/ vGiLAN VoLTE Business Consumer 5
NFV Use case - vCPE CPE virtualiza-on is not just about cost reduc-on but providing new services to customers at the pace of innova-on and Scale Virtualized Central Office OR Data Center Security & Firewall Enterprise vCPE Parental Control Residen7al vCPE Quota Management Home Automa7on Internet Residen7al NID NFVO SDN Controller VNFM(s) VIM Enterprise vCPE Security & Firewall Quality of Service (QoS) Traffic Shaping Device Management 6
NFV requirements Generic NFV Workload Requirements ● Flexibility of IP address assignment - Public IP, Private IP, IPv4 and IPv6 etc - many VNFs require no NAT ○ DHCP based address assignment ● Mul7ple Interface assignment - Rou7ng, Metering etc ● Mul7-Tenancy and Management of overlays ● Packet Forwarding Performance requirements - All workloads are not equal ○ NIC bonding ○ NUMA affinity - container scheduling ○ Huge Page Support ○ CPU pinning or par77oning ○ Jumbo frames support ● Hybrid VNFs (container and VMs) ● Mixed topologies containers and VMs ● Load sharing ● Elas7city - Orchestra7on
Scale metrics and factors A Mul7-dimensional problem for Telcos Total number of Sessions, subscribers scale • Service Density - VMs, Apps • Throughput scale • Orchestra7on scale • Number of comple7ons (Adds moves and deletes) • Management and troubleshoo7ng scale • Visibility and Traceability at scale • Audit Trail of Transac7ons • Development environments CI/CD • Introduc7on of new func7onality • It is not just about scale but also the speed of scale 8
Scale metrics and factors Example: vCPE For Residen7al Services Footprint - Subscriber density • Typical BNG Router serves 300K IP Sessions - Half rack dedicated hardware • Adding QoS and other bells and whistles => 150-200K IP Sessions • Throughput per subscriber • 10Gbps connec7ons common - but simultaneous users and subscribers average to • <0.5Gbps per subscriber 50K ac7ve subscribers => 25 x 100Gbps sustained throughput • Number of VMs per server - VNF Requirements on CPU, Memory and IO • Number of Subs per VMs • Number of Servers • Number of cores needed to serve that throughput using OVS+Accelera7on or VPP • etc HOW CAN WE SCALE THIS TO EVEN HIGHER DESNITIES 9
Why Containers? Containers: Sonware packaging concept that include an applica7on and all its run7me dependencies ● Low virtualiza7on overhead per VNF (applica7on) ● Low memory footprint ● Instant restart 7me ● Low Latency - due to a shared memory model ● Higher density per server/socket than VMs ● Encapsula7on of microservices ● Portability ● Determinis7c packaging ● Reasonable Isola7on can be accomplished easily
Comparing VMs and Containers Virtual Machine VMs Guest OS is needed per VM ● App1 App2 Each Virtual Machine is isolated by the hypervisor ● App1 App2 Interface and hardware emulated by the Hypervisor ● OS OS Distribu7on of app 7ed to OS ● Libs Libs OS commonly tuned to deliver app performance ● Hypervisor Docker Engine Containers There is no hypervisor in the container stack ● Host OS Host OS Docker Engine acts as the “hypervisor” ● Each applica7on runs as a process in user space ● Isola7on through cnames ● Hardware Hardware Considered “lightweight” compared to VMs ● Packet forwarding performance dependent on ● kernel stack Orchestra7on via Kubernetes ● Container Stack VM Stack Scale - >10x ●
VMs and Containers – Telco Evolu7on viewpoint VMs App App App App App App Libs & Run7me Libs & Run7me Guest OS Guest OS Libs & Run7me Libs & Run7me Guest OS Guest OS HYPERVISOR App App HYPERVISOR HOST OS HOST OS HW Libs & Run7me App App HW Containers in VM Guest OS (Tenant Isola7on) Libs & Run7me Hypervisor VM HOST OS HW App App Containers & VMs Libs & Run7me HOST OS HW Containers 12
Containers & NFV 13
Containers & NFV Applicability ● Use containers ala “VM” ● Leverage dockeriza7on of some func7ons - such as DHCP, IPAM, NAT, FW etc ● Not really separa7ng components within Network Func7ons (ala Microservices) as the network func7ons themselves are virtualized ● Intui7ve to apply and hence assumed easy to make it happen
Revisi7ng the NFV requirements With Containers - How do they fare? ● Flexibility of IP address assignment to containers - Public IP, Private IP, IPv4 and IPv6 etc - many VNFs require no NAT ● Mul7ple Interface assignment to a container ● Mul7-Tenancy and Management of overlays ● Performance requirements - All workloads are not equal ○ NIC bonding ? ○ NUMA affinity - scheduling ? ○ Huge Page Support ? ○ CPU pinning or par77oning ● Hybrid VNFs (container and VMs) ● Mixed topologies containers and VMs ? ● Load sharing and scale
Containers and NFV Telco provided defini7on ● Control plane heavy VNFs ○ High session count or control traffic Candidates for ○ Low data forwarding Containeriza7on ○ Latency and availability sensi7ve for network convergence ○ Examples - signaling, subscriber policy, control protocols ● Data plane heavy VNFs ○ Require large memory alloca7on ○ Large footprint applica7ons (CPU, memory, I/O) ○ High forwarding rate requirements ○ High volume of traffic ○ Examples - PGW, ePDG, DPI etc
Sizing NFV Example ● Simple VNF - vRouter with 2 interfaces ○ Simple IGP and BGP Configura7on ○ Stock images - Vyos distribu7on ○ Memory needed to run the VM with basic alloca7ons - 387MB ● Containers 6-10X density ○ Same configura7on ○ Stock Container image ○ Run using Docker ○ Per container - 34MB ● vCPU alloca7ons per VM ○ Core processing for DPDK ○ 6-12 cores for VNFs like vEPC, BNG ○ 16-32GB of memory Notes: Smaller configs result in smaller containers - Only 1 BGP session and an IGP results in 28MB per container
Forwarding performance with containers NFV = Line Rate Performance Requirement ● Use namespaces to isolate network func7ons ● Network namespaces for containers to see their resources ● Kernel performance becomes important ● Sonware switch - like macvlan ● Assign SR-IOV to network namespace ● Using DPDK accelera7on?
Container Orchestra7on Scale of Orchestra7on ● Kubernetes - Scale is Proven - Openshin ○ Today operates largest of DCs with millions of containers ○ Enterprise IT and OTT ● Scaling number of pods and nodes ○ Common to find 100 Nodes and 3000 Pods for VNF deployments ● Kolla - Ansible playbooks with Docker Containers to provide produc7on ready containers for openstack clouds ● NFV special requirements ○ Constraints on Kubernetes/Openshin ○ What about OAM management, Traceability, Performance, conformance, audit trail
OPENSHIFT – PLATFORM FOR CONTAINERS Atomic Host Networking Telemetry Security Storage Automa7on Clustering Kubernetes based OrchestraDon Docker Container Format
Openstack and Containers Managing containers in Openstack Environments ● Run Openshin/Kubernetes on Openstack Kuryr ● Magnum ● ● Run Openstack services in containers ○ Kolla
Subscriber Service Chaining – with Containers How does SFC work with containers? AAA Configura7on Policy Service Func7on Chain Parental Control VxLAN L2 NID HFC/GPON CPE vOLT VxLAN or IPSec Quota Firewall & NATing Management ⲙ VNF Applica7on or Content Cache Demarca7on point Cloud Boundary Central Office or DC • Containers at the remote site or central data center • S7tched together for service chaining – • same host –IPC • different hosts -VLAN/VXLAN • Port mapping architecture can be made to work here • Will this impact NSH or dynamic SFC? 22
Proof of Concept 23
Recommend
More recommend
