NetBSD Kernel Topics: IP Processing mbuf structure Loadable Kernel - - PowerPoint PPT Presentation

▶
netbsd kernel topics
SMART_READER_LITE
LIVE PREVIEW

NetBSD Kernel Topics: IP Processing mbuf structure Loadable Kernel - - PowerPoint PPT Presentation

Nov 01, 2022 576 likes •910 views

NetBSD Kernel Topics: IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous Washington January 9, 2001 Router Plugins (Crossbow) 1 WASHINGTON UNIVERSITY IN ST LOUIS Goals IP Understand the structure of IP

slide-1
SLIDE 1

January 9, 2001

Router Plugins (Crossbow)

1

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics:

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-2
SLIDE 2

January 9, 2001

Router Plugins (Crossbow)

2

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Goals

  • IP

– Understand the structure of IP processing in NetBSD – Understand the IP packet format – Become familiar with IP packet field access

» accessing src and dst addresses, etc…

  • mbufs

– Understand the mbuf structure – Become familiar with mbuf access routines

» packet size » data location

  • Stevens TCP/IP books

– impress upon you the importance of these ref. books

slide-3
SLIDE 3

January 9, 2001

Router Plugins (Crossbow)

3

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

What we are NOT going to talk about

  • IPv6

– Everything we talk about will be in terms of IPv4 – Router Plugins also supports IPv6 – The code can be confusing

» use IPv6 structures since they are superset of IPv4

  • IP Options
  • Fast Forward Path
  • Details of interaction between device drivers and IP

– actually, we’ll talk about some of this…

  • Packet Scheduler
  • Routing protocols
  • daemons: routed, rsvpd, …
slide-4
SLIDE 4

January 9, 2001

Router Plugins (Crossbow)

4

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics:

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-5
SLIDE 5

January 9, 2001

Router Plugins (Crossbow)

5

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Vocabulary: Terms Used in Stevens

  • message: Transport protocol data given to IP
  • datagram: message + IP header
  • fragment: if datagram is too large for network: IP splits it.

Each fragment contains its own IP header

  • packet: fragment or datagram small enough for network
  • frame: packet + data-link layer header
slide-6
SLIDE 6

January 9, 2001

Router Plugins (Crossbow)

6

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Protocol messages, IP Packets, Fragments, …

Derived from Figure 8.7 from Wright/Stevens, Vol. 2

fragment 1 fragment 2 fragment 3

IP UDP application data Application data IP Application data IP

packet

IP UDP application data link link link Application data IP Application data IP

frame

IP

datagram

UDP Application data

message

slide-7
SLIDE 7

January 9, 2001

Router Plugins (Crossbow)

7

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Format

IP Header (with no Options fields) Transport Protocol (e.g. TCP, UDP) Header 8 16 24 31

Vers HLen TOS ID Fragment Offset

Source Address Destination Address

Total length Flags TTL

Protocol

Header Checksum

Source Port . . . Destination Port

Packet Payload

slide-8
SLIDE 8

January 9, 2001

Router Plugins (Crossbow)

8

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IPv4 Forwarding in the NetBSD Kernel (APIC)

ipintrq: ipintr ip_forward ip_output ip6_forward ip6_output atmc_input apic_isr apic_intr ip6_input atmc_output

input packets interrupts

  • utput packets

tcp/udp_input tcp/udp_output socket layer apic_devoutput

L I N K D A T A N E T W O R K U P P E R L O W E R

apic_start packet_scheduler PS_Q

T R A N S P O R T

slide-9
SLIDE 9

January 9, 2001

Router Plugins (Crossbow)

9

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

User IP in the NetBSD Kernel

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output

input packets interrupts

  • utput packets

tcp/udp_input tcp/udp_output socket layer apic_devoutput

L I N K D A T A N E T W O R K U P P E R L O W E R

User Program

Software Interrupt Waits in Socket Buffer for User Program to pick up

apic_start packet_scheduler PS_Q

T R A N S P O R T

slide-10
SLIDE 10

January 9, 2001

Router Plugins (Crossbow)

10

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

T R A N S P O R T

SW Interrupt HW Interrupt

ipintrq: ipintr ip_forward ip_output ip6_forward ip6_output atmc_input apic_isr apic_intr ip6_input atmc_output

input packets interrupts

  • utput packets

User Space Kernel

tcp/udp_input tcp/udp_output socket layer apic_devoutput

IP Packet Handling in the CB Kernel

L I N K D A T A N E T W O R K U P P E R L O W E R

apic_start packet_scheduler PS_Q

Where are the gates?

sgate dgate

This is the gate that you will use. Here are the rest of the gates? Where is the route lookup done?

slide-11
SLIDE 11

January 9, 2001

Router Plugins (Crossbow)

11

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Important Source Files (usr/src/sys/netinet/ip*)

  • ip.h

– struct ip { } – #define’s

  • ip_input.c

– ipintr() (This is ip_input routine) – ip_forward()

  • ip_output.c

– ip_output()

slide-12
SLIDE 12

January 9, 2001

Router Plugins (Crossbow)

12

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel with APIC

  • Device Driver -- Device Dependent

– apic_intr()

» read INTR_ACK: 0 => not for us, return » apic_isr()

– apic_isr()

» read NOTIFY_LIST, next rcv channel needing attention

  • Process descriptors for RCV channel

– swap words in each received mbuf (APIC BUG) – link mbufs until we find end of frame – verify CRC (just check flag set by APIC) – atmc_input(packet) – repeat

  • repeat

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

slide-13
SLIDE 13

January 9, 2001

Router Plugins (Crossbow)

13

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel (continued)

  • Device Driver - Dev. Independent atmc_input():

– LLC/SNAP processing (if needed)

» extract type (IP/RATM/…)

– AAL5 processing

» extract length » strip trailer

– aiu_getafix(packet) (flow table will be discussed later)

» aiu_getafix stores a ptr to FTE in packet

– Enqueue in IP Queue

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

slide-14
SLIDE 14

January 9, 2001

Router Plugins (Crossbow)

14

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel (continued)

  • IP Input -- ipintr():

– Get next packet from IP Queue – Do some basic checks, header, length, checksum… – process IP options – aiu_dgate(&m,2) Router Plugins Dynamic Gate #2 – if packet is not for us and we can forward it:

» forward packet ip_forward() -- upcoming slide... » return

– aiu_dgate(&m,4) Router Plugins Dynamic Gate #4 – protocol specific input routine e.g. tcp_input()

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

slide-15
SLIDE 15

January 9, 2001

Router Plugins (Crossbow)

15

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel (continued)

  • IP Forwarding -- ip_forward():

– decide if we need to send any redirects to sender – aiu_dgate(&m,5) Router Plugins Dynamic Gate #5 – route lookup – ip_output()

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

Update for Fred’s new Gate also check details of if/when route lookup is done

slide-16
SLIDE 16

January 9, 2001

Router Plugins (Crossbow)

16

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel (continued)

  • IP Output -- ip_output()

– aiu_dgate(&m,3) Router Plugins Dynamic Gate #3 – if (no route yet)

» get route

– check for special processing

» ANEP Options

  • anep_output()

– if DAN then afd_handle_dan_packet() :ACTIVE PROCESSING

– send on interface

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

slide-17
SLIDE 17

January 9, 2001

Router Plugins (Crossbow)

17

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

IP Packet Handling in CB Kernel (continued)

  • Device Driver Device Indep.-- atmc_output():

– LLC/SNAP processing – AAL5 processing (if needed) – if packet_scheduling: enqueue for PS – if !packet_scheduling: send to devoutput()

  • Device Driver Device Dep. -- apic_devoutput()

– configure an APIC descriptor for each mbuf in packet – Resume APIC TX channel

ipintrq: ipintr ip_forward ip_output atmc_input apic_isr apic_intr atmc_output apic_devoutput

slide-18
SLIDE 18

January 9, 2001

Router Plugins (Crossbow)

18

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Note about Addresses

  • The address structures in the Crossbow Kernel are IPv6

– IPv6 address structures are a superset of IPv4 – IPv6 address are 128 bits

  • We will be using IPv4 addresses

– e.g. 192.168.5.2 – IPv4 addresses are 32 bits

  • Notation for using IPv4 address in IPv6:

– Use double colon before address:

::192.168.5.2

– Double colon tells the utilities to set everything to the left to 0’s

slide-19
SLIDE 19

January 9, 2001

Router Plugins (Crossbow)

19

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-20
SLIDE 20

January 9, 2001

Router Plugins (Crossbow)

20

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

The mbuf Data Structure

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags 0-108 MT_xxx

mbuf{}

0-100 MT_xxx

M_PKTHDR

2048-byte cluster mbuf{}

208-2048 MT_xxx

M_EXT

2048

2048-byte cluster mbuf{}

m_pkthdr.len m_pkthdr.rcvif m_ext.ext_buf m_ext.ext_free m_ext.ext_size 208-2048 MT_xxx

M_PKTHDR | M_EXT

2048

Normal mbuf Normal mbuf with Packet Hdr Cluster mbuf Cluster mbuf with Packet Hdr

slide-21
SLIDE 21

January 9, 2001

Router Plugins (Crossbow)

21

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

  • Highlight: (add Stevens’ like figures…)

– m_next vs m_nextpkt (figure 2.2 page 35)

» m_next: points to next mbuf in THIS packet » m_nextpkt: points to the first mbuf of the NEXT packet

– m_len vs. m_pkthdr.len – m_ext.ext_buf vs. m_data – Which mbuf structure we use with APIC

  • Example:

– do m_pullup – get data pointer – print out src addr, dst addr and protocol fields – get Transport header pointer – print out dst port Sorry: This was just one

  • f my notes slides that

should have been removed before printing

slide-22
SLIDE 22

January 9, 2001

Router Plugins (Crossbow)

22

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

The mbuf Data Structure

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_DATA

mbuf{}

0-100 MT_xxx

M_PKTHDR

2048-byte cluster mbuf{}

208-2048 MT_xxx

M_EXT

2048

2048-byte cluster mbuf{}

m_pkthdr.len m_pkthdr.rcvif m_ext.ext_buf m_ext.ext_free m_ext.ext_size 208-2048 MT_xxx

M_PKTHDR | M_EXT

2048

Sorry: This was just one

  • f my notes slides that

should have been removed before printing

slide-23
SLIDE 23

January 9, 2001

Router Plugins (Crossbow)

23

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

The mbuf Data Structure (example)

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_DATA m_pkthdr.len

m_pkthdr.rcvif 100 bytes

  • f data

100 NULL

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_DATA

50 bytes

  • f data

50 NULL NULL next mbuf in chain next mbuf in chain next packet in queue

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_DATA m_pkthdr.len

m_pkthdr.rcvif Ethernet header, IP header, UDP header

M_PKTHDR 192 NULL 42 mbuf packet header head of queue tail of queue

slide-24
SLIDE 24

January 9, 2001

Router Plugins (Crossbow)

24

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

The mbuf Data Structure (example, continued)

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_HEADER m_pkthdr.len

m_pkthdr.rcvif

M_PKTHDR 1514 NULL 54

mbuf{}

m_next m_nextpkt m_len m_data m_type m_flags MT_DATA m_pkthdr.len

m_pkthdr.rcvif

M_EXT 1460 NULL NULL

Ethernet header, IP header, TCP header

NULL 2048–byte cluster 1460 bytes of data

m_ext.ext_buf m_ext.ext_free m_ext.ext_size 2048

next mbuf in chain mbuf packet header tail of queue

slide-25
SLIDE 25

January 9, 2001

Router Plugins (Crossbow)

25

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-26
SLIDE 26

January 9, 2001

Router Plugins (Crossbow)

26

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Loadable Kernel Modules

  • Mechanism in NetBSD to dynamically load code into running kernel
  • NetBSD System Utilities:

– modload(8) – modunload(8) – modstat(8)

  • Module Types supported:

– System Call modules

» When unloaded, returns to original system call » Any system call can be replace » Take care when replacing ioctl(2) since LKM uses it to load/unload modules!!

– Virtual File System modules – Device Driver modules: Block and character device drivers – Execution Interpreters: For binaries not normally usable by OS – Miscellaneous modules

» No defined interfaces » Up to user/developer to provide hooks to get to the code » This is what Router Plugins uses

slide-27
SLIDE 27

January 9, 2001

Router Plugins (Crossbow)

27

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Loadable Kernel Modules: modload

  • Open /dev/lkm

– ioctl’s will be performed on the open file descriptor

  • Prelink module, open it and calculate size info

– system(“ld -A /netbsd -e _<entry> -o <outfile> -T <addr=0> <module>”);

  • Reserve Kernel memory:

– ioctl(lkm_fd, LMRESERV, size_info) – returns kernel load address

  • Relink at kernel load address
  • Open relinked module and load it into kernel

– ioctl(lkm_fd, LMLOADBUF, ...)

  • Adjusting symbol table entry pointers and load it.
  • sync();
  • Call the module’s entry function to test it.
  • Post-install if called for...

– ioctl(lkm_fd, LMSTAT, ...)

slide-28
SLIDE 28

January 9, 2001

Router Plugins (Crossbow)

28

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-29
SLIDE 29

January 9, 2001

Router Plugins (Crossbow)

29

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Interrupts

  • 8 levels

– spl0 normal – splsoftclock low-priority clock processing – splnet network protocol processing – spltty terminal I/O – splbio disk and tape I/O – splimp network device I/O – splclock high-priority clock processing – splhigh all interrupts blocked

  • Typical usage:

int i; i = splimp(); [do something at device I/O level] splx(i);

slide-30
SLIDE 30

January 9, 2001

Router Plugins (Crossbow)

30

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

NetBSD Kernel Topics

IP Processing mbuf structure Loadable Kernel Modules Interrupts Miscellaneous

slide-31
SLIDE 31

January 9, 2001

Router Plugins (Crossbow)

31

Washington

WASHINGTON UNIVERSITY IN ST LOUIS

Other kernel details you must know

  • No floating point
  • No #include in header files
  • More caveats and info from Fred later…