about time , ssl and other things …

To see the certification information from linux cli (k is optional for without verification)

curl –vvIk https://<FQDN>or<IP&gt;

To read the certificate file human readable (you can use -inform parameter to change the output format)

openssl x509 -in <your_certificate_file_name_maybe_txt_pem_whatever>

Sometimes you can have broken/corrupted pem or certificate files could be ! maybe wrong copy past , windows to linux copy , ftp copy issues that time using “curl” for only debug could not help pls try more tool to double check for example “wget”

I faced Socket error: [X509] PEM lib (_ssl.c:2751) and couldn’t find any solution and wget helped me, wget show us some of certifications files are not readable.

Another option is using openssl with s_client parameter

openssl s_client -host FQDN -port 443 -quiet

tzdata = time zone database

Sometime people are mixing UTC and GMT , GMT is time zone and UTC is time standard but both of them share same current time practice. No any country or territory use UTC as a local time.

Debian change time zone 

dpkg-reconfigure tzdata

or

sudo cp /usr/share/zoneinfo/xxx/xxx /etc/localtime

About how ntp work , what is stratum, what is reference clock pls check the links below. Generally reference clock is stratum-0 and its atomic time(cesium clock) some more levels there startum-1 , 2 up to 16 and looks like we are querying time from mostly  startum-2 servers.

https://ntpserver.wordpress.com/2008/09/10/ntp-server-stratum-levels-explained/

http://www.ntp.org/ntpfaq/NTP-s-algo.htm

To update time with date command perfect article from nixCraft http://www.cyberciti.biz/faq/howto-set-date-time-from-linux-command-prompt/

I mostly needed this one

date +%T -s “10:13:13”

To password less sudo privileges  

sudo su

echo “noroot ALL=(ALL) NOPASSWD: ALL” >> /etc/sudoers

sslvpnd can cause ha sync /Webinterface unresponsive issue? -another Fortinet story-

Today we faced Fortinet web interface become unresponsive,  we find out some articles and expect that killing/restarting httpd will be enough but we faced policy load issues for example try to list rules but we have empty response and after some time its gone and need to restart httpd to access webui again.

Then after some investigation we saw that cluster checksum is not consistent (command:diagnose sys ha cluster-csum)

Tried to sync ha config but not succeed (command:exec ha synchronize start) (for more pls check)

Then somehow we maybe did not prioritise  but cluster member which web interface is working but first snmp service stop response and then sslvpn connections are start to not work ! in this time what i remember we changed the password of sslvpn user but i don’t think that this help us but when we kill the sslvpnd magically non-responsive fortinet box become to run , after all checked ha csum its worked and snmp also start to work !

Actually if we did not try this (also vendor said that related firmware have a bug) we have to restart nodes and this will cause some downtime  . Version is 5.2.6

Some good link for debugging ha http://kb.fortinet.com/kb/documentLink.do?externalID=FD36494

diag  debug enable
diagnose  debug  console  timestamp enable
diag debug application hasync -1
diag debug application hatalk -1
execute ha synchronize start

When you can not kill process gently -another Fortinet story-

I expect that you know the pid but if its not you have two ways

Option 1

Walter (global) # diag test app snmpd 1

snmpd pid = 161

Option 2 (somehow related commands are not return some processes pids, then start to use fnsysctl )

List pid files then get pid id from related file

Walter (global) # fnsysctl ls /var/run/

Walter (global) # fnsysctl cat /var/run/snmpd.pid

161

Then execute (somehow diag sys kill 11 <pid_id> do not kill related pid)

Walter (global) # fnsysctl kill -9 161

thats it !

When Fortigate ips engine and AV engine fuck everything !

Since the beginning we have always trouble about choosing wrong hardwares , developer issues like handle sessions with single core , then ASIC things , its always changing  NPx support something NPy support more , end of the day you will always have NPx/y/z/t/u what released but cost is another issue but don’t worry providers always things these are usual things. Hope one day with DPDK or another technology will help us to use firewalls without ASICs and without need always buy new hardwares. (I know you know SDDC , SDN also i know ! )

I know this device is UTM , UTM is somehow fancy things, you know you shouldn’t use , know what will happen but you have to because of some situations (for ISPs)

AV issues , IPS engine issues , conserve mode ! I don’t how you really protect my device and network behind

Big problem AV and memory , still can not understand small to bigger devices why more ram is not used ! Memory is expensive ? or maybe still 32 bit ! What developers can’t handle ?

at the end AV full the ram , whole device under stress and communication issues , BGPs or OSPF are gone ! :(((

For AV its not working like a linux services or something ? you should kill it ! Step by Step , each process , disabling AV is not helping every time

Login your device , switch to config global (it is also command ) then execute diagnose sys top see the processed press q and leave to console execute diagnose sys kill 11 <process_id> . Here we are using because maybe some outputs we could have then fallow console for mem usage.

Another issue is ips engine , so understandable command diag test:) lovely , its not meaningful for me but meaningful for developer or who maintain cli commands

diagnose test app ipsmonitor

You will see nice options and choose what you exactly want , restart , stop , start , get status

Also if you run cluster then consider do same things on slave:) to switch slave

  • config global
  • get system ha status
  • exec ha manage 1 (mostly)

Good fixes !

VM

Windows Time Server Configuration

Really i hate to write out this article but i have to !

Finally i find out working one , hope its help for you too !

its really hard to understand where is gone time server configuration after promote active directory

its really hard to understand the way of clear configuration of time

its really hard to understand why multiple too many article explaining this and its working for some but not others

maybe its hard to understand me ! Okay forgot everything because its working now:)

First clear all configs , all of time config if your are not sure , execute such commands step by step

Not: Unregister process for clear config , sometimes its giving an error but don’t worry its %100 working

C:\Users\Administrator>net stop w32time
The Windows Time service is stopping.
The Windows Time service was stopped successfully.
C:\Users\Administrator>w32tm /unregister
W32Time successfully unregistered.

C:\Users\Administrator>w32tm /register
W32Time successfully registered.

C:\Users\Administrator>net start w32time
The Windows Time service is starting…
The Windows Time service was started successfully.

Then tricky thing is someone execute commands and saying working but someone its not !

Step by step execute such commands ….

w32tm /config /syncfromflags:manual

w32tm /config /manualpeerlist:”ntp1.ulakbim.gov.tr” (Replace ntp server with your one ! )

w32tm /config /reliable:yes ( Fuck your reliability )

net stop w32time && net start w32time

Then for some troubleshooting tips

w32tm /query /peers (to see the peers are active)

w32tm /query /status (to see everything is fine)

w32tm /query /status (to force sync)

Thats it !

See you

docker cok kullanilan komutlar kisa kisa – Adim 3

Evet , hizlica cok kullanilir komutlari inceleyip gecelim ….

Elimizde docker container kuracak hangi imajlar var bakalim.

vahric:~ vahricmuhtaryan$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
redis               latest              0f0e96f1f267        10 days ago         177.5 MB
ubuntu              latest              b72889fa879c        2 weeks ago         188 MB

Ilgili kolonlari acikalamak gerekir ise repo adi olarak redis , sanki bir klasor acilmis ve icersine latest diye bir imaj koyulmus , her bir imajin tekil bir numarasi , ne zaman yaratildigi ve kapladigi alani belirtilmis.

Ek olarak isterseniz docker images <repo_adi> seklinde veya docker images <repo_adi>:tag gibide listeleme yapabilisiniz ama aradiginiz seyin %100 uyusmasi gerekiyor repo* veya re?po gibi bir arama mekanizmasi yok !

Simdi, bir imaj indirelim, diyelim centos en son surum ki burada ek birsey belirmeye gerek yok kendisi <tag> kismina bakarsaniz latest olarak belirtmis yani en son version centos.

vahric:~ vahricmuhtaryan$ docker pull centos
Using default tag: latest

fakat diyelim siz en son surumu degilde bir onceki veya olan 6.6 surumune sahip imaji indirmek istiyorsunuz. Asagidaki komutta bir ustteki satira gore farki kullandigimiz <tag> yani centos:6.6  . Tag lar “:” den sonra belirtiliyor.

vahric:~ vahricmuhtaryan$ docker pull centos:6.6
6.6: Pulling from library/centos

Read the rest of this entry

Neymis bu Graphite … (Giris)

En son  soyleyecegimi basta soylemek gerekir ise Graphite aynen RRDtool gibi geldi , uygulamanizdan, sunucularinizdan hatta metrik toplayabileceginiz x seyden mesela networkden aldiginiz degerleri saklayip bunu icin grafikler cizebiliyor size, bu sekilde hem performans hemde durum bilgilerini toparlayip bir ekrandan veya kendinize ozel ekranlar yaratabilir ve diger ucuncu parti yazilimlarla ornegin cabot ile alarm uretebiliyorsunuz.

Benim Graphite’yi incelememdeki amac tam olarak “Tracking Dynamic Host and Application Metrics at Scale” bunla beraber sirada ilgili kitap da Docker monitor etmek icin yontemlerden biri Graphite hakkinda bilgiler veriyor.

Graphite’ye bir sekilde metrikleri yollamaniz gerekiyor, bildigim ozel bir metrik toplayicisi yok bunla ilgili bir liste var fakat ben bu makalede sadece telnet ve/veya python kullanarak nasil ve ne yaratabiliriz Graphite uzerinde onu deneyecegim.

Read the rest of this entry

VisorFSObj: 1954: Cannot create file /var/lib/vmware/hostd/journal/1462853708.8163 for process hostd-worker because the inode table of its ramdisk (root) is full.

Problem : No vMotion , No possibility to start VM

in vmware.log on ESXi host is like below

VisorFSObj: 1954: Cannot create file /var/lib/vmware/hostd/journal/1462853708.8163 for process hostd-worker because the inode table of its ramdisk (root) is full.

All articles talking about snmpd but snmpd was not working for us but inodes table is still full .

Command to see inode usage is : esxcli system visorfs ramdisk list

To Solve the problem ;

You can use any way to handle it until find out this article but better solution is try to stop unused services like if its HP you can stop some of it like ams or stop CIM services , vm or something like that to free up inodes and then try to move vms on that server to go !

I know its not good solution but this is the last thing we can do to solve the problem before find better solution😉

VM

Hatalari Yakalayalim Beyler …

Iste , hata mesajimiz bu ve ciktinin onemli yeride “ValueError“. Asagidaki cikti yas soruldugunda string girildiginde alindi.

/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Users/vahricmuhtaryan/PycharmProjects/fonksiyonlar1/test1.py
Yasin Kacdfdf
Traceback (most recent call last):
File “/Users/vahricmuhtaryan/PycharmProjects/fonksiyonlar1/test1.py”, line 5, in <module>
a = int(raw_input(“Yasin Kac”))
ValueError: invalid literal for int() with base 10: ‘dfdf’

Process finished with exit code 1

Baska bir ornek , buradada onemli olan “urllib3.exceptions.MaxRetryError” . Aciklamasi normal boye bir site yok baglanilacak.

/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Users/vahricmuhtaryan/PycharmProjects/fonksiyonlar1/test1.py
Traceback (most recent call last):
File “/Users/vahricmuhtaryan/PycharmProjects/fonksiyonlar1/test1.py”, line 14, in <module>
istek = http_w_pool.request(‘get’,’http://www.yupyu42342342p.com&#8217;)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/request.py”, line 69, in request
**urlopen_kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/request.py”, line 90, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/poolmanager.py”, line 165, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/connectionpool.py”, line 628, in urlopen
release_conn=release_conn, **response_kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/connectionpool.py”, line 628, in urlopen
release_conn=release_conn, **response_kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/connectionpool.py”, line 628, in urlopen
release_conn=release_conn, **response_kw)
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/connectionpool.py”, line 608, in urlopen
_stacktrace=sys.exc_info()[2])
File “/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/urllib3/util/retry.py”, line 273, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host=’www.yupyu42342342p.com’, port=80): Max retries exceeded with url: / (Caused by NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x101def590>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known’,))

Process finished with exit code 1

Burada program hataya dusmeden bir uyari yaratmamiz gerekiyor , aslinda try icin gerekli kismi yaptik ve ValueError/urllib3.exceptions.MaxRetryError denen hatalari yakaladik onun icin kodu soyle duzenleyip bir sonraki adima gecebiliriz.
Read the rest of this entry

docker-machine – Adim 2

docker-machine windows ve Mac OSX gibi isletim sistemleri uzerinden birden fazla Docker Host’u manage etmek icin kullanilan bir binary .

Ben Mac OSX kullanicisiyim ve makinemde Docker Toolbox kurulu ki icersinde  docker-machine iceriyor , yok ise edinin kendisi ek olarak Linux ve Windows icinde mevcut.

docker-machine tek bir noktadan networkunuzdeki docker host’lari yonetebildigi gibi Digital Ocean , Openstack ve diger cloud provider’lardaki kaynak havuzunuzda docker host’lar hazirlayabilir ve yonetebilirsiniz.

Bu makalede generic driver kullanarak hali hazirda uzerinde ubuntu kurulu bir VM i sisteme tanitacagim (hatta oncesinde uzerinde Docker da kurulmus yani existing docker host’u ekliyoruz)

Simdi asil amacim docker-machine ile docker host’lari yonetmekten cok karsilastigim hatalar ve cozumleri belirtmek.

Herkes yaratarak baslar , biz silerek baslayalim ama oncesinde bir listeleyelim ne var diye ….

vahric:.ssh vahricmuhtaryan$ docker-machine ls

NAME       ACTIVE   DRIVER    STATE     URL                        SWARM   DOCKER    ERRORS

docker01   –        generic   Running   tcp://10.111.21.210:2376           Unknown   Unable to query docker version: Unable to read TLS config: open /Users/vahricmuhtaryan/.docker/machine/machines/docker01/server.pem: no such file or directory

docker02   –        generic   Running   tcp://10.111.21.210:2376           Unknown   Unable to query docker version: Unable to read TLS config: open /Users/vahricmuhtaryan/.docker/machine/machines/docker02/server.pem: no such file or directory

Sonra silelim Read the rest of this entry

Follow

Get every new post delivered to your Inbox.

Join 134 other followers