Openstack REST API Kolayca Ogrenme

Tek yapmaniz gereken once cli’daki komutlari ogrenmek sonra basina –debug parametresini koymak


Ornek : aa1 adindan bir Domain yaratiyoruz …

stack@helion-cp1-c1-m1-mgmt:~$ openstack –debug domain create aa1

Istek ve cevap hersey asagidaki gibi , bunu gibi ornekleri cogaltip REST API gereksinimlerini ve kullanmini kolayca anlayip uygulayabilirsiniz.

REQ: curl -g -i –cacert “/etc/ssl/certs/ca-certificates.crt” -X POST -H “User-Agent: openstack” -H “Content-Type: application/json” -H “Accept: application/json” -H “X-Auth-Token: {SHA1}aa0e9ca9215d6fa8cd7f9f814d5e637e06e95655” -d ‘{“domain”: {“enabled”: true, “name”: “aa1”}}’

“POST /v3/domains HTTP/1.1” 201 177

RESP: [201] Content-Length: 177 Vary: X-Auth-Token Server: Apache/2.4.10 (Debian) Date: Tue, 06 Sep 2016 08:26:58 GMT Content-Type: application/json x-openstack-request-id: req-2627ac03-1ea2-477c-8297-84ccbd32112d

RESP BODY: {“domain”: {“enabled”: true, “id”: “7bdc8676ca4b4c1491dc7668cf6fa7cb”, “links”: {“self”: “”}, “name”: “aa1”}}


about time , ssl and other things …

To see the certification information from linux cli (k is optional for without verification)

curl –vvIk https://<FQDN>or<IP&gt;

To read the certificate file human readable (you can use -inform parameter to change the output format)

openssl x509 -in <your_certificate_file_name_maybe_txt_pem_whatever>

Sometimes you can have broken/corrupted pem or certificate files could be ! maybe wrong copy past , windows to linux copy , ftp copy issues that time using “curl” for only debug could not help pls try more tool to double check for example “wget”

I faced Socket error: [X509] PEM lib (_ssl.c:2751) and couldn’t find any solution and wget helped me, wget show us some of certifications files are not readable.

Another option is using openssl with s_client parameter

openssl s_client -host FQDN -port 443 -quiet

tzdata = time zone database

Sometime people are mixing UTC and GMT , GMT is time zone and UTC is time standard but both of them share same current time practice. No any country or territory use UTC as a local time.

Debian change time zone 

dpkg-reconfigure tzdata


sudo cp /usr/share/zoneinfo/xxx/xxx /etc/localtime

About how ntp work , what is stratum, what is reference clock pls check the links below. Generally reference clock is stratum-0 and its atomic time(cesium clock) some more levels there startum-1 , 2 up to 16 and looks like we are querying time from mostly  startum-2 servers.

To update time with date command perfect article from nixCraft

I mostly needed this one

date +%T -s “10:13:13”

To password less sudo privileges  

sudo su

echo “noroot ALL=(ALL) NOPASSWD: ALL” >> /etc/sudoers

sslvpnd can cause ha sync /Webinterface unresponsive issue? -another Fortinet story-

Today we faced Fortinet web interface become unresponsive,  we find out some articles and expect that killing/restarting httpd will be enough but we faced policy load issues for example try to list rules but we have empty response and after some time its gone and need to restart httpd to access webui again.

Then after some investigation we saw that cluster checksum is not consistent (command:diagnose sys ha cluster-csum)

Tried to sync ha config but not succeed (command:exec ha synchronize start) (for more pls check)

Then somehow we maybe did not prioritise  but cluster member which web interface is working but first snmp service stop response and then sslvpn connections are start to not work ! in this time what i remember we changed the password of sslvpn user but i don’t think that this help us but when we kill the sslvpnd magically non-responsive fortinet box become to run , after all checked ha csum its worked and snmp also start to work !

Actually if we did not try this (also vendor said that related firmware have a bug) we have to restart nodes and this will cause some downtime  . Version is 5.2.6

Some good link for debugging ha

diag  debug enable
diagnose  debug  console  timestamp enable
diag debug application hasync -1
diag debug application hatalk -1
execute ha synchronize start

When you can not kill process gently -another Fortinet story-

I expect that you know the pid but if its not you have two ways

Option 1

Walter (global) # diag test app snmpd 1

snmpd pid = 161

Option 2 (somehow related commands are not return some processes pids, then start to use fnsysctl )

List pid files then get pid id from related file

Walter (global) # fnsysctl ls /var/run/

Walter (global) # fnsysctl cat /var/run/


Then execute (somehow diag sys kill 11 <pid_id> do not kill related pid)

Walter (global) # fnsysctl kill -9 161

thats it !

When Fortigate ips engine and AV engine fuck everything !

Since the beginning we have always trouble about choosing wrong hardwares , developer issues like handle sessions with single core , then ASIC things , its always changing  NPx support something NPy support more , end of the day you will always have NPx/y/z/t/u what released but cost is another issue but don’t worry providers always things these are usual things. Hope one day with DPDK or another technology will help us to use firewalls without ASICs and without need always buy new hardwares. (I know you know SDDC , SDN also i know ! )

I know this device is UTM , UTM is somehow fancy things, you know you shouldn’t use , know what will happen but you have to because of some situations (for ISPs)

AV issues , IPS engine issues , conserve mode ! I don’t how you really protect my device and network behind

Big problem AV and memory , still can not understand small to bigger devices why more ram is not used ! Memory is expensive ? or maybe still 32 bit ! What developers can’t handle ?

at the end AV full the ram , whole device under stress and communication issues , BGPs or OSPF are gone ! :(((

For AV its not working like a linux services or something ? you should kill it ! Step by Step , each process , disabling AV is not helping every time

Login your device , switch to config global (it is also command ) then execute diagnose sys top see the processed press q and leave to console execute diagnose sys kill 11 <process_id> . Here we are using because maybe some outputs we could have then fallow console for mem usage.

Another issue is ips engine , so understandable command diag test🙂 lovely , its not meaningful for me but meaningful for developer or who maintain cli commands

diagnose test app ipsmonitor

You will see nice options and choose what you exactly want , restart , stop , start , get status

Also if you run cluster then consider do same things on slave🙂 to switch slave

  • config global
  • get system ha status
  • exec ha manage 1 (mostly)

Good fixes !


Windows Time Server Configuration

Really i hate to write out this article but i have to !

Finally i find out working one , hope its help for you too !

its really hard to understand where is gone time server configuration after promote active directory

its really hard to understand the way of clear configuration of time

its really hard to understand why multiple too many article explaining this and its working for some but not others

maybe its hard to understand me ! Okay forgot everything because its working now🙂

First clear all configs , all of time config if your are not sure , execute such commands step by step

Not: Unregister process for clear config , sometimes its giving an error but don’t worry its %100 working

C:\Users\Administrator>net stop w32time
The Windows Time service is stopping.
The Windows Time service was stopped successfully.
C:\Users\Administrator>w32tm /unregister
W32Time successfully unregistered.

C:\Users\Administrator>w32tm /register
W32Time successfully registered.

C:\Users\Administrator>net start w32time
The Windows Time service is starting…
The Windows Time service was started successfully.

Then tricky thing is someone execute commands and saying working but someone its not !

Step by step execute such commands ….

w32tm /config /syncfromflags:manual

w32tm /config /manualpeerlist:”” (Replace ntp server with your one ! )

w32tm /config /reliable:yes ( Fuck your reliability )

net stop w32time && net start w32time

Then for some troubleshooting tips

w32tm /query /peers (to see the peers are active)

w32tm /query /status (to see everything is fine)

w32tm /query /status (to force sync)

Thats it !

See you

docker cok kullanilan komutlar kisa kisa – Adim 3

Evet , hizlica cok kullanilir komutlari inceleyip gecelim ….

Elimizde docker container kuracak hangi imajlar var bakalim.

vahric:~ vahricmuhtaryan$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
redis               latest              0f0e96f1f267        10 days ago         177.5 MB
ubuntu              latest              b72889fa879c        2 weeks ago         188 MB

Ilgili kolonlari acikalamak gerekir ise repo adi olarak redis , sanki bir klasor acilmis ve icersine latest diye bir imaj koyulmus , her bir imajin tekil bir numarasi , ne zaman yaratildigi ve kapladigi alani belirtilmis.

Ek olarak isterseniz docker images <repo_adi> seklinde veya docker images <repo_adi>:tag gibide listeleme yapabilisiniz ama aradiginiz seyin %100 uyusmasi gerekiyor repo* veya re?po gibi bir arama mekanizmasi yok !

Simdi, bir imaj indirelim, diyelim centos en son surum ki burada ek birsey belirmeye gerek yok kendisi <tag> kismina bakarsaniz latest olarak belirtmis yani en son version centos.

vahric:~ vahricmuhtaryan$ docker pull centos
Using default tag: latest

fakat diyelim siz en son surumu degilde bir onceki veya olan 6.6 surumune sahip imaji indirmek istiyorsunuz. Asagidaki komutta bir ustteki satira gore farki kullandigimiz <tag> yani centos:6.6  . Tag lar “:” den sonra belirtiliyor.

vahric:~ vahricmuhtaryan$ docker pull centos:6.6
6.6: Pulling from library/centos

Read the rest of this entry

Neymis bu Graphite … (Giris)

En son  soyleyecegimi basta soylemek gerekir ise Graphite aynen RRDtool gibi geldi , uygulamanizdan, sunucularinizdan hatta metrik toplayabileceginiz x seyden mesela networkden aldiginiz degerleri saklayip bunu icin grafikler cizebiliyor size, bu sekilde hem performans hemde durum bilgilerini toparlayip bir ekrandan veya kendinize ozel ekranlar yaratabilir ve diger ucuncu parti yazilimlarla ornegin cabot ile alarm uretebiliyorsunuz.

Benim Graphite’yi incelememdeki amac tam olarak “Tracking Dynamic Host and Application Metrics at Scale” bunla beraber sirada ilgili kitap da Docker monitor etmek icin yontemlerden biri Graphite hakkinda bilgiler veriyor.

Graphite’ye bir sekilde metrikleri yollamaniz gerekiyor, bildigim ozel bir metrik toplayicisi yok bunla ilgili bir liste var fakat ben bu makalede sadece telnet ve/veya python kullanarak nasil ve ne yaratabiliriz Graphite uzerinde onu deneyecegim.

Read the rest of this entry

VisorFSObj: 1954: Cannot create file /var/lib/vmware/hostd/journal/1462853708.8163 for process hostd-worker because the inode table of its ramdisk (root) is full.

Problem : No vMotion , No possibility to start VM

in vmware.log on ESXi host is like below

VisorFSObj: 1954: Cannot create file /var/lib/vmware/hostd/journal/1462853708.8163 for process hostd-worker because the inode table of its ramdisk (root) is full.

All articles talking about snmpd but snmpd was not working for us but inodes table is still full .

Command to see inode usage is : esxcli system visorfs ramdisk list

To Solve the problem ;

You can use any way to handle it until find out this article but better solution is try to stop unused services like if its HP you can stop some of it like ams or stop CIM services , vm or something like that to free up inodes and then try to move vms on that server to go !

I know its not good solution but this is the last thing we can do to solve the problem before find better solution😉