Developing SLA apps by using

ORACLE Solaris 11.3

DSCP flows
Samsi Serghei
blog: http://sscdvp.blogspot.com

Real-world SLA application

Codul surs disponibil online:

https://github.com/sscdvp/flow-mgmt

Implementat n termen de 30 de zile datorit tehnologiei anuntate

de ORACLE in 03.2015

Solaris Flow

Din manual flowadm(1M):

... a flow is defined as a set of attributes based on Layer 3 and Layer 4 headers,

which can be used to identify a protocol, service, or a virtual machine

... can be used on any type of data link, including physical links, virtual NICs, and link

aggregations

Caracteristice de baz ale flow

Gestioneaz QoS pentru stiva virtualizat de reea

Flow QoS este integrat n stiva de protocoale i nu este un layer separat

Diferenierea serviciilor se bazeaz pe atributele L3/L4:

protocol (UDP/TCP/SCTP/ICMP) se suport IPv6

adres IP (SRC/DST) se accept masca de reea

port (SRC/DST)

DS field se accept valoarea i masca

Efectuarea controlului de band cu un efort minim

Partajarea limei de band PNIC/VNIC ntre mai muli clieni. Beneficienii pot fi VM-uri sau chiar
socket-uri

Clasificarea traficului

Marcarea traficului prin DSCP

Integrat n Solaris Zones (administratorul zonei poate gestiona flow-urile aferente)

November 2015

Se adaug: marcarea DSCP, flow-uri unidirec ionale,

ridicarea constrngerilor privind combina ia atributelor flow pe un

Oracle Solaris 11.3.0.30.0

datalink, flow ranking

May 2015

Se adaug: marcarea DSCP

Oracle Solaris 11.2.8.4.0

August 2014

Se adaug: componentul SDN - application-driven flows

November 2011

Se adaug suport pentru Solaris Zones

ORACLE Solaris 11 11/11

November 2008

Prima apariie a elementului cheie n virtualizarea de re ea

OpenSolaris

(SO_FLOW_SLA), prioritizarea

Solaris flows: controlul limii de band, limea zero dac e dorit

sistarea traficului, motenirea set-urilor CPU de la datalink atribuit,

Oracle Solaris 11.2.0.0.42

(Crossbow)

stocarea configuraiei flow-urilor n fiiere pentru pstrare dup

restart

Cronologia evoluiei Solaris Flow

Arhitectura virtualizare de reea

ORACLE Solaris

Virtualization lane: conine resurse hardware i

software destinate pentru procesarea traficului

Resursele PNIC: ring-urile Tx i Rx

Resursele MAC: softring-urile

Resursele de transport: cozile de serializare

Resursele CPU

Izolarea i controlul resurselor n stiva

virtualizat de reea ORACLE Solaris

Componente stivei virtualizate

Rx ring-urile HW i SW

Interaciunea ntre TOE (TCP/IP Offload

Engine) i sistem

Afinitatea intreruperilor MSI (Receiver Side Scaling)

Afinitatea pachetelor ce constituie un flow (Receiver Packet

Steering)

Afinitatea la nivel de virtualization lane (Receive Flow Steering)

Pachetele sa fie procesate in batch

Filtrarea direct pe PNIC

Avantaje RFS

Independenta de NIC hardware

Orice protocol nou poate fi adoptat in filtrele software

Utilizeaza IPI si nu afecteaza IRQ

Rx-descriptor

Clasificator MAC Level

Clasificator High Level

Workflow aplicaiei SLA

SMF configuratia aplicatiei

din partea zonei Gateway

SMF configuratia aplicatiei

din partea zonei DNS

Captura traficului DNS

n timpul desfurrii atacului DoS

Captura traficul DNS parial marcat DSCP

Statistica flow-urilor din partea serverului

DNS (coloana IDROPS)

Rata de rejectare pentru trei flow-uri de agregare:

59%, 100%, 1%

Dificulti utiliznd libdladm API

Lipsa documentaiei API

Memory leak-uri (Soluionat: Oracle Solaris 11.3.3.6.0)

Constrngere n ioctl DLDIOC_WALKFLOW (exist

workaround)

Output MDB pentru aplicaie:

>::findleaks -dvf
findleaks: elapsed CPU time => 0.0 seconds
findleaks: elapsed wall time => 0.0 seconds
findleaks:
CACHE LEAKED BUFCTL CALLER
086f5010 1 0879dc40 libdladm.so.1`do_check_dscp+0x3c
086f5010 1 0879dbc8 libdladm.so.1`do_check_maxbw+0x34
-----------------------------------------------------------------------
Total 2 buffers, 32 bytes
umem_alloc_16 leak: 1 buffer, 16 bytes
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
879dc40 8798fa0 1e2f61f9b5d0b6 1
86f5010 0 0
libumem.so.1`umem_cache_alloc_debug+0x157
libumem.so.1`umem_cache_alloc+0x19d
libumem.so.1`umem_alloc+0x76
libumem.so.1`malloc+0x2d
libdladm.so.1`do_check_dscp+0x3c
libdladm.so.1`i_dladm_flow_proplist_extract_one+0x198
libdladm.so.1`dladm_flow_proplist_extract+0x37
libdladm.so.1`dladm_flow_add+0x83
do_add_flow+0x33a
main+0x118
_start+0x7d
umem_alloc_16 leak: 1 buffer, 16 bytes
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
879dbc8 8798fc0 1e2f61f9b50f33 1
86f5010 0 0
libumem.so.1`umem_cache_alloc_debug+0x157
libumem.so.1`umem_cache_alloc+0x19d
libumem.so.1`umem_alloc+0x76
libumem.so.1`malloc+0x2d
libdladm.so.1`do_check_maxbw+0x34
libdladm.so.1`i_dladm_flow_proplist_extract_one+0x198
libdladm.so.1`dladm_flow_proplist_extract+0x37
libdladm.so.1`dladm_flow_add+0x83
do_add_flow+0x33a

Output MDB pentru tool-ul flowadm:

#env LD_PRELOAD=/usr/lib/libumem.so.1 UMEM_DEBUG=default

/usr/bin/i86/mdb /usr/sbin/flowadm
>::load libumem
>::sysbp _exit
>:r add-flow -l vlink35 -a local_ip=10.10.7.7 -p dscp=38 test2
mdb: stop on entry to _exit
mdb: target stopped at:
0xec88cd88: nop
mdb: You've got symbols!
Loading modules: [ ld.so.1 libumem.so.1 libc.so.1 libuutil.so.1 ]
> ::findleaks
CACHE LEAKED BUFCTL CALLER
0852d290 1 0856ec40 libdladm.so.1`do_check_dscp+0x3c
0852d290 1 0856ebc8 libdladm.so.1`do_check_maxbw+0x34
-----------------------------------------------------------------------
Total 2 buffers, 32 bytes

CR #

Description

Fixed in version

SR date

Resolution
date

15606330
15806736

17649247

restriction on flow creation can be

Oracle Solaris

relaxed in some cases

11.3.0.30.0

some flow hash tables scale poorly

Oracle Solaris

with a large number of flows

11.3.0.30.0

inbound/outbound traffic only flows

Oracle Solaris

16.01.15

23.02.15

16.01.15

23.02.15

16.01.15

23.02.15

23.04.15

02.06.15

11.3.0.30.0
20981017

libdladm leaks memory while adding

Oracle Solaris

flows

11.3.3.6.0

Lista CR-urilor deschise

sau escaladate n MOS

Link-uri utile
https://docs.oracle.com/cd/E53394_01/html/E54847/ntwkg.html#SOLWNgpqhs
https://docs.oracle.com/cd/E53394_01/html/E54764/flowadm-1m.html
https://blogs.oracle.com/yenduri/entry/new_flowadm_features_in_s11
https://tools.ietf.org/html/rfc2474
ORACLE - Writing Device Drivers
http://docs.oracle.com/cd/E23824_01/html/819-3196/gkbnv.html#gld3-datapaths
Interrupt handlers in ORACLE Solaris
http://www.oracle.com/technetwork/server-storage/solaris10/interrupt-handlers-141289.html

Mulumesc pentru atenie!