OVS ile Biraz DB, Biraz Monitor/Troubleshoot , Biraz Openflow programlama

Simdi biraz daha ileri gidelim , uzun bir yazi oldu ama keyifli !

Oncelikle ihtiyacimiz olacak birkac sey ;

Bir onceki yazida ovsdb-client kullanarak OVS tablolarini listelemistik simdi aynisini bir benzerini asagidaki sekilde alalim.

noroot@kvm-ovs-server1:~$ sudo ovsdb-client get-schema

Not : bu komutu >  sudo ovsdb-client get-schema | python -m json.tool < seklinde kullanabilirsiniz veya ciktiyi  http://www.jsoneditoronline.org aktrip direk grafik uzerinden harekete edebilirsiniz .

Screen Shot 2015-12-15 at 17.13.10

Simdi elimizde bir DB var , OVS ilk kuruldugunda yaratilan , semasi yukardaki gibi olan ve icersunde kayitlar olan bir db. Haliyle buraya bir select bir insert atilamaz mi diye merak ediyor insan. Evet OVSDB uzerinde sorgulama islemi yapabiliriz bunu icinde XML-RPC’nin yerine JSON-RPC kullaniyor olacagiz . (Cok kil , ordan [ ac sonra kapa ] of array mi ] eksikmi derken😦 neyse )

Eger dump almak yerine sql query yazar gibi , mesela varsayili db adi “Open_vSwitch” buradan “Bridge” tablosundan sadece “fail_mode” secure olanlari almak isterseniz iste boyle yaziyorsunuz , allah kolaylik version “dump” dan grep lemeye devam🙂

noroot@kvm-ovs-server1:~$ sudo ovsdb-client transact ‘[“Open_vSwitch”,{“op”:”select”, “table”:”Bridge”, “where”: [ [“fail_mode”, “==”,”secure” ] ] }]’
[{“rows”:[{“_uuid”:[“uuid”,”0927ecc0-cea1-48f9-8b81-456a61747068″],”_version”:[“uuid”,”7ed38da0-3931-4c5d-9a36-1db4084ace30″],”controller”:[“set”,[]],”datapath_id”:”0000c2ec2709f948″,”datapath_type”:””,”external_ids”:[“map”,[]],”fail_mode”:”secure”,”flood_vlans”:[“set”,[]],”flow_tables”:[“map”,[]],”ipfix”:[“set”,[]],”mirrors”:[“set”,[]],”name”:”aa2″,”netflow”:[“set”,[]],”other_config”:[“map”,[]],”ports”:[“uuid”,”4d6a2baa-897c-4864-b378-97a6b563e511″],”protocols”:[“set”,[]],”sflow”:[“set”,[]],”status”:[“map”,[]],”stp_enable”:false}]}]

Daha guzel bir komut “monitor” parametresi, bu parametreyi yazip komutu calistirdiginizda sanki “tail -f” yapmis gibi olackasiniz, farkli bir terminalden bir adet bridge veya port yaratin ve ekrana neler dusuyor gorun

noroot@kvm-ovs-server1:~$ sudo ovsdb-client monitor Bridge

Iki tane vswitch yaratalim , ilki varsayili degerlerle diger ise biraz fakli olacak

vswitch1 -> adi “learning-switch”

noroot@kvm-ovs-server1:~$ sudo ovs-vsctl add-br learning-switch

vswtich2 -> adi “manuel-switch” . Burada ek olarak “–” diye bir komutun arkasina digerini yapistiriyoruz, buna “atomic transaction” diyorlar

noroot@kvm-ovs-server1:~$ sudo ovs-vsctl add-br manuel-switch — set Bridge manuel-switch fail-mode=secure

Yukardaki fark “fail-mode=secure” , bunun farki ne ona bakalim

Farkini gormek icin iki komut calistiracagiz ;

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl dump-flows learning-switch

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=466.719s, table=0, n_packets=8, n_bytes=648, idle_age=457, priority=0 actions=NORMAL

Yukardaki satiri aciklamak gerekir ise ;

  1. Bir adet flow
  2. table0 bizim icin onemli cunku tum paketler table0 dan islenmeye baslaniyor
  3. action=Normal , bu demek oluyor ki yarattigimiz switch L2 learning switch olarak hareket edecek.

ve

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl dump-flows manuel-switch

NXST_FLOW reply (xid=0x4):

Yukardaki satiri aciklamak gerekir ise

  1. Hic bisiy yok !
  2. Iletisim yok !
  3. Kimin kimle iletisime gececegini siz belirleyeceksiniz.

Simdi hidden flow’lar var mi yok mu onuda gorelim

noroot@kvm-ovs-server1:~$ sudo ovs-appctl bridge/dump-flows learning-switch

duration=2568s, priority=0, n_packets=8, n_bytes=648, priority=0,actions=NORMAL

table_id=254, duration=2568s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x3,actions=drop

table_id=254, duration=2568s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x1,actions=controller(reason=no_match)

table_id=254, duration=2568s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x2,actions=drop

noroot@kvm-ovs-server1:~$ sudo ovs-appctl bridge/dump-flows manuel-switch

table_id=254, duration=1954s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x3,actions=drop

table_id=254, duration=1954s, priority=0, n_packets=8, n_bytes=648, priority=0,reg0=0x1,actions=controller(reason=no_match)

table_id=254, duration=1954s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x2,actions=drop

Simdi 4 tane makine yaratacagim ve bunlari ikiserli sekilde “learning switch” ve “manuel-switch” e ekleyecegim

sudo ubuntu-vm-builder kvm trusty --addpkg linux-image-generic --addpkg openssh-server --arch amd64 --hostname S1-LS --dest S1-LS --libvirt qemu:///system --bridge learning-switch --mem 1024 --cpu 1 --ip xxx.yyy.zzz.ttt --mask 255.255.255.224 --net xxx.yyy.zzz.ttt --bcast xxx.yyy.zzz.ttt --gw xxx.yyy.zzz.ttt --dns 8.8.8.8;

noroot@kvm-ovs-server1:~$ sudo virsh list –all

setlocale: No such file or directory

Id    Name                           State

—————————————————-

–     S1-LS                          shut off

–     S1-MS                          shut off

–     S2-LS                          shut off

–     S2-MS                          shut off

S1-LS ve S2-LS calistirdim , simdi oncelikle sanal makinelerin bagli oldugu switch portlarini gorelim .

Sanal sunuculardan birini sanal switch’in 2. digerinide 4. portuna sanal olarak baglanmis (bakiniz koyu isaretlenmis yazilara). Burada belki LOCAL port ilginizi cekebilir kendisi aslinda ilk once “learning-switch” yaratigimizda olusan HOST un IP Stack’ine baglanmis olan interface.

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl dump-ports-desc learning-switch

OFPST_PORT_DESC reply (xid=0x2):

2(vnet0): addr:fe:54:00:bd:d1:a7

config:     0

state:      0

current:    10MB-FD COPPER

speed: 10 Mbps now, 0 Mbps max

4(vnet1): addr:fe:54:00:6c:74:56

config:     0

state:      0

current:    10MB-FD COPPER

speed: 10 Mbps now, 0 Mbps max

LOCAL(learning-switch): addr:7a:f7:8d:41:e4:4c

config:     0

state:      0

speed: 0 Mbps now, 0 Mbps max

Buda farkli bir gorunum , burada ayni zamanda error count larida izleyebilirsiniz.

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl dump-ports learning-switch

OFPST_PORT reply (xid=0x2): 3 ports

port  4: rx pkts=14, bytes=1124, drop=0, errs=0, frame=0, over=0, crc=0

tx pkts=6, bytes=476, drop=0, errs=0, coll=0

port  2: rx pkts=14, bytes=1124, drop=0, errs=0, frame=0, over=0, crc=0

tx pkts=22, bytes=1772, drop=0, errs=0, coll=0

port LOCAL: rx pkts=8, bytes=648, drop=0, errs=0, frame=0, over=0, crc=0

tx pkts=33, bytes=2634, drop=0, errs=0, coll=0

Yarattigimiz “learning-switch” NORMAL yani L 2mode’da oldugundan bir MAC Tablosu olusmus olmali bunu yine “ovs-appctl” ile gorecegiz , bunu yapmadan once mutlaka birbirin ping veya x servisine erismeye calisinki tabloya girsin.

noroot@kvm-ovs-server1:~$ sudo ovs-appctl fdb/show learning-switch

port  VLAN  MAC                Age

4   497  52:54:00:6c:74:56    0

2   497  52:54:00:bd:d1:a7    0

Simdi zurnanin zirttt dedigi yerdeyiz , iki sanal makine birbirini ping’ler iken datapath yani kernel‘dan bir flow olayina bakalim

noroot@kvm-ovs-server1:~$ sudo ovs-dpctl dump-flows

skb_priority(0),in_port(4),eth(src=52:54:00:6c:74:56,dst=52:54:00:bd:d1:a7),eth_type(0x0800),ipv4(src=xxx.yyy.zzz.156/0.0.0.0,dst=uuu.eee.www.155/0.0.0.0,proto=1/0,tos=0/0,ttl=64/0,frag=no/0xff), packets:16, bytes:1568, used:0.589s, actions:3

skb_priority(0),in_port(3),eth(src=52:54:00:bd:d1:a7,dst=52:54:00:6c:74:56),eth_type(0x0800),ipv4(src=uuu.eee.www.155/0.0.0.0,dst=xxx.yyy.zzz.156/0.0.0.0,proto=1/0,tos=0/0,ttl=64/0,frag=no/0xff), packets:16, bytes:1568, used:0.589s, actions:4

Guzel simdi ayni seyi tekrardan bu kez L2 switch olarak set edilmemis switch de yapalim

S1-LS ve S2-LS calistirdim , iki makineyi arasinda ping ile iletisim kurmaya calistim , bukez once datapath/kernel dan bir bakalim ne oluyor

noroot@kvm-ovs-server1:~$ sudo ovs-dpctl dump-flows

skb_priority(0),in_port(6),eth_type(0x0806), packets:0, bytes:0, used:never, actions:userspace(pid=4294963055,slow_path(controller))

Kernel’da herhangi birsey match etmediginden anlasilmasi icin slow_path yani userspace yani ovs-vswitchd’e firlatiyor …

Hemen openflow flow tablosuna bakalim , zaten birsey girmis olmadigimizdan gizli rule/flow lara bakalim direkt , asagidak gorebileceginiz gibi ikici flow (kalin yazili) a match ediyor , cunku yarattigimiz switch L2 mode ‘da degil ve ellede herhangi bir flow/kural/rule girmis degiliz.

noroot@kvm-ovs-server1:~$ sudo ovs-appctl bridge/dump-flows manuel-switch

table_id=254, duration=17254s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x3,actions=drop

table_id=254, duration=17254s, priority=0, n_packets=353, n_bytes=16386, priority=0,reg0=0x1,actions=controller(reason=no_match)

table_id=254, duration=17254s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x2,actions=drop

MAC tablosuda bos 

noroot@kvm-ovs-server1:~$ sudo ovs-appctl fdb/show manuel-switch

port  VLAN  MAC                Age

Bundan sonrasini http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=tutorial/Tutorial;hb=HEAD takip ederek ilerleyecegim . Buradaki herseyi birebir uygulamayacagim fakat ilgili testler dogrultusunda gereken kadariyla ilerleyecegim.

Bu arada benim senaryom biraz farkli ne Trunk port var neden portlarin openflow id lerini set etmek , amacim sadece iki adet sanal makinenin fiziksel switch e bagli iki sunucu gibi 497 vlan inda birbirleri ile konusmalar.

Zaten flow tablomuzda hicbirseyin olmadigini biliyoruz , buna istinaden dokumanda tek bir flow table degil birden fazla flow table kullanarakdan dizyan edilmis

 Table 0: Admission control. (Giris/Kabul/Resepsiyon)

 Table 1: VLAN input processing. (Iletisim icin izin verilen VLAN lari belirleme)

 Table 2: Learn source MAC and VLAN for ingress port. (Paket icersindeki MAC/VLAN degerini ogrenelim)

 Table 3: Look up learned port for destination MAC and VLAN. (Hedef port'u bulalim)

 Table 4: Output processing. (Paketi teslim edelim)

Adim 1 

Asagidaki hedef MAC adresi , STP paketleri icin , bunu drop ediyor , bridge/switch’STP paketlerine cevap vermesini istemiyor
Tablo 0 daha onceden dedigim gibi paketin ilk giris noktasi.

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=0, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop”

Geri kalan tum paketleri baska bir table’a aktarmak istiyor , resubmit anladigim kadariya GOTO , 1 de Tablo 1 anlaminda

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=0, priority=0, actions=resubmit(,1)”

Kontrol edelim

Openflow’da bir tablo’da tarama yaparken tum kayitlara bakildigini ve mathc edenlerden priority’si yuksek olanin uygulandigini unutmayin.

 

noroot@kvm-ovs-server1:~$ sudo ovs-appctl bridge/dump-flows manuel-switch

duration=788s, priority=32768, n_packets=0, n_bytes=0, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0,actions=drop

duration=167s, priority=0, n_packets=171, n_bytes=7182, priority=0,actions=resubmit(,1)

table_id=254, duration=19917s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x3,actions=drop

table_id=254, duration=19917s, priority=0, n_packets=2844, n_bytes=121008, priority=0,reg0=0x1,actions=controller(reason=no_match)

table_id=254, duration=19917s, priority=0, n_packets=0, n_bytes=0, priority=0,reg0=0x2,actions=drop

Simdi bunu test etmek lazim🙂 Nasil mi ? Yazmislar onuda yazmislar🙂

+++Rule satirina dikkat

Komut ve Cikti –>

noroot@kvm-ovs-server1:~$ sudo ovs-appctl ofproto/trace manuel-switch in_port=100,dl_dst=01:80:c2:00:00:05

Flow: metadata=0,in_port=100,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=01:80:c2:00:00:05,dl_type=0x0000

+++Rule: table=0 cookie=0 dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0

OpenFlow actions=drop

Final flow: unchanged

Relevant fields: skb_priority=0,in_port=100,dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0,dl_type=0x0000,nw_frag=no

Datapath actions: drop

Yukardaki ornek icin mac adersini –> 50:22:c2:00:00:55 vererek deneyin , bu durumda “OpenFlow actions=resubmit(,1)” aksiyonunu goreceksiniz. Bu arada son olarak hala paketin Table 1 de drop oldugunu gozden kacirmayin😉

Adim 2

Biz herkesin degilde sadece belli VLAN/MAC lerin gececegini hesapladigimizdan , tablo nunbasina sanki firewall kurali yazar gibi once any any bir deny yazalim

Komut –>

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=1, priority=0, actions=drop”

Haliyle allow olacak birsey’e ihtiyac var , asagida 3 ve 4 numarali portlari access port olarak set edelim

Bunu sanki bir cisco switch’de ilgili port’u ilgili vlan’a atayip access gibi set ettiginizi dusunun

switch# configure terminal
switch(config)# interface ethernet 1/7
switch(config-if)# switchport access vlan 2
switch(config-if)#
Komut –>

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=1, priority=99, in_port=3, vlan_tci=0, actions=mod_vlan_vid:497, resubmit(,2)”

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=1, priority=99, in_port=4, vlan_tci=0, actions=mod_vlan_vid:497, resubmit(,2)”

Note: ilgili port numaralarini ogrenmek icin “sudo ovs-ofctl dump-ports manuel-switch

Kontrol Edelim

Asagidaki komutda calistirdiginizda , 2 numarali tablo’ya istegi forward ettigini gorebilirsiniz.

noroot@kvm-ovs-server1:~$ sudo ovs-appctl ofproto/trace manuel-switch in_port=3

Flow: metadata=0,in_port=3,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000

Rule: table=0 cookie=0 priority=0

OpenFlow actions=resubmit(,1)

Resubmitted flow: unchanged

Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0

Resubmitted  odp: drop

Rule: table=1 cookie=0 priority=99,in_port=3,vlan_tci=0x0000

OpenFlow actions=mod_vlan_vid:497,resubmit(,2)

Resubmitted flow: metadata=0,in_port=3,dl_vlan=497,dl_vlan_pcp=0,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000

Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0

Resubmitted  odp: drop

No match

Final flow: unchanged

Relevant fields: skb_priority=0,in_port=3,vlan_tci=0x0000,dl_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl_type=0x0000,nw_frag=no

Datapath actions: drop

Eger bu porta tagged bir paket gonderir isek tablo 1 de drop olup kalacak

noroot@kvm-ovs-server1:~$ sudo ovs-appctl ofproto/trace manuel-switch in_port=3,vlan_tci=2

Flow: metadata=0,in_port=3,vlan_tci=0x0002,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x0000

Rule: table=0 cookie=0 priority=0

OpenFlow actions=resubmit(,1) # Burada tablo 2 ye aktarim yok !!

Resubmitted flow: unchanged

Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0

Resubmitted  odp: drop

Rule: table=1 cookie=0 priority=0

OpenFlow actions=drop

Final flow: unchanged

Relevant fields: skb_priority=0,in_port=3,vlan_tci=0x0002,dl_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl_type=0x0000,nw_frag=no

Datapath actions: drop

Adim 3

MAC adresini ogrenelim …

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=2 actions=learn(table=10, NXM_OF_VLAN_TCI[0..11], NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[], load:NXM_OF_IN_PORT[]->NXM_NX_REG0[0..15]), resubmit(,3)”

Tablo 10’u sorgulayalim , burada bu kez ofproto/trace degil , S1_MS den S2_MS ‘i pingledigimde asagidaki ciktiyi aldim.

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl dump-flows manuel-switch table=10

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=970.015s, table=10, n_packets=0, n_bytes=0, idle_age=970, hard_age=28, vlan_tci=0x01f1/0x0fff,dl_dst=52:54:00:04:ab:e2 actions=load:0x4->NXM_NX_REG0[0..15]

Yukarida –> load:0x4 <– bize 4 numarali port’dan (ingress port) istegin geldigini gosteriyor

Birde S2-MS den S1-MS ping’lemeye basladigimda kernel (datapath’i) kontrol etmeye basladiginizda dst mac adresi bilmedigimiz ortada🙂

noroot@kvm-ovs-server1:~$ sudo ovs-dpctl dump-flows

skb_priority(0),in_port(5),eth(src=52:54:00:45:a2:a4,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806), packets:1, bytes:42, used:0.772s, actions:drop

Adim 4

Yaklasiyoruz diger tarafi ping’lemeye

Hedef portu bulmak icin once tablo10 (yani daha onceden ogrendiysek zorlamay gerek yok) , eger orada yoksa tablo 4 e gidiyoruz (Keza gidiyoruz)

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl add-flow manuel-switch “table=3 priority=50 actions=resubmit(,10), resubmit(,4)”

Adim5

reg0=0 demek eger destination bilinmiyor ise flood et demek.

sudo ovs-ofctl add-flow manuel-switch “table=4 reg0=0 priority=99 dl_vlan=497 actions=strip_vlan,3,4”

Dikkat table 10 openflow ve kernel datapath flowlarina eklenti olacak ama hala S2-MS , S1-MS in MAC adresini bulamiyor durumda.

3 ve 4 numarali portlarimiz access portlar oldugundan tablo 4 de ilgili islemler yapilirken paket icersindeki vlan tag’larinin cikartilmasi lazim.

sudo ovs-ofctl add-flow manuel-switch “table=4 reg0=3 actions=strip_vlan,3”

sudo ovs-ofctl add-flow manuel-switch “table=4 reg0=4 actions=strip_vlan,4”

Eger flow’lari yanlis girdiyseniz ihtiyaciniz olacak  :)  (Gerektiginde Flow lari silmek)

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl del-flows manuel-switch table=10

noroot@kvm-ovs-server1:~$ sudo ovs-ofctl del-flows manuel-switch table=4

 

Ekstra

Eger olurda bridge’e eklediginiz interface’in (veth0) portunu sabitlemek isterseniz

ovs-vsctl add-port switch1 port1 -- set Interface port1 ofport_request=1

 

Guzel Linkler

https://en.wikipedia.org/wiki/JSON-RPC

Ornek Insert , Select

http://openvswitch.org/pipermail/discuss/2013-July/010619.html
https://mailman.stanford.edu/pipermail/openflow-discuss/2010-February/000744.html

OVSDB

http://networkstatic.net/getting-started-ovsdb/

http://www.revolutionlabs.net/2013/11/what-commands-are-called-during-startup_28.html

Multicast Address

http://www.cavebear.com/archive/cavebear/Ethernet/multicast.html

Posted on 17/12/2015, in OVS and tagged , , , , , , , , , , , , , , , , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: