[ASK]:: LOAD Server 100% + HTTPD

twistedshells · 16 Jan 2012

@TS: coba copy paste hasil perintah-perintah berikut:

top -c
vmstat 3 3
iostat 3 3

priz_ · 16 Jan 2012

twistedshells said:
@TS: coba copy paste hasil perintah-perintah berikut:

top -c
vmstat 3 3
iostat 3 3

OK mas, saya pantau dlu,

Terakhir saya lihat di httpd/error_log, berikut penampakannya

[Mon Jan 16 09:59:58 2012] [notice] child pid 8908 exit signal Segmentation fault (11)
[Mon Jan 16 09:59:58 2012] [notice] child pid 9701 exit signal Segmentation fault (11)
[Mon Jan 16 09:59:58 2012] [notice] child pid 11090 exit signal Segmentation fault (11)
[Mon Jan 16 09:59:59 2012] [notice] child pid 30130 exit signal Segmentation fault (11)
[Mon Jan 16 09:59:59 2012] [notice] child pid 30513 exit signal Segmentation fault (11)
[Mon Jan 16 09:59:59 2012] [notice] child pid 32003 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:03 2012] [notice] child pid 32119 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:03 2012] [notice] child pid 12055 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:03 2012] [notice] child pid 12056 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:03 2012] [notice] child pid 12057 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:04 2012] [notice] child pid 12058 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:06 2012] [notice] child pid 32114 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:06 2012] [notice] child pid 8909 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:06 2012] [notice] child pid 8925 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:06 2012] [notice] child pid 12046 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:06 2012] [notice] child pid 12060 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:07 2012] [notice] child pid 12059 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:07 2012] [notice] child pid 12924 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:08 2012] [notice] child pid 12925 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:12 2012] [notice] child pid 12930 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:12 2012] [notice] child pid 12932 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:12 2012] [notice] child pid 12933 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:12 2012] [notice] child pid 12934 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:13 2012] [notice] child pid 12935 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12926 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12936 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12937 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12938 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12939 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12940 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:14 2012] [notice] child pid 12941 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:15 2012] [notice] child pid 12942 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12943 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12944 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12945 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12946 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12947 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:17 2012] [notice] child pid 12948 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:18 2012] [notice] child pid 12949 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:18 2012] [notice] child pid 12950 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12951 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12952 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12953 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12954 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12955 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:28 2012] [notice] child pid 12956 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:29 2012] [notice] child pid 12957 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:29 2012] [notice] child pid 12958 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:34 2012] [notice] child pid 12961 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:34 2012] [notice] child pid 12962 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:34 2012] [notice] child pid 12965 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:35 2012] [notice] child pid 12966 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:35 2012] [notice] child pid 12967 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:36 2012] [notice] child pid 12968 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:36 2012] [notice] child pid 12969 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:36 2012] [notice] child pid 12970 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:37 2012] [notice] child pid 12971 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:37 2012] [notice] child pid 12972 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:37 2012] [notice] child pid 12976 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:37 2012] [notice] child pid 12977 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:41 2012] [notice] child pid 12978 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:41 2012] [notice] child pid 12981 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:41 2012] [notice] child pid 12982 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12979 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12980 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12983 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12985 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12986 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12987 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:42 2012] [notice] child pid 12989 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:43 2012] [notice] child pid 12990 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:43 2012] [notice] child pid 12991 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:43 2012] [notice] child pid 12992 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:44 2012] [notice] child pid 12993 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:44 2012] [notice] child pid 12994 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:44 2012] [notice] child pid 12995 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:44 2012] [notice] child pid 12996 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:44 2012] [notice] child pid 12997 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:45 2012] [notice] child pid 12999 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:45 2012] [notice] child pid 13000 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:45 2012] [notice] child pid 13001 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:45 2012] [notice] child pid 13002 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13003 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13004 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13006 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13007 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13008 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13009 exit signal Segmentation fault (11)
[Mon Jan 16 10:00:46 2012] [notice] child pid 13010 exit signal Segmentation fault (11)

priz_ · 16 Jan 2012

sigmabisnis said:
@TS, iya itu yang dicoret merah nama username nya ya??
Sepertinya user ID tersebut ngerunning banyak process ID gitu...(kayak terjadi flood gitu), BTW hingga Load Average-nya hingga 278 , OMG.

Kemungkinan ada sesuatu program/webscript yang executed dari user account tsb yang menyebabkan Load Server nya jadi tinggi.

Iya mas itu yg saya contreng merah adalah nama usernya,

kalau misalnya ada program/webscript yang menyebabkan load server jadi tinggi, ada tidak mas command yg bisa memberikan bukti. jadi lebih enak kasih penjelasang ke yg punya website.

Thanks

twistedshells · 16 Jan 2012

Pak Priz, kalau dilihat dari pesan error httpd_log sepertinya apachenya ada masalah. Entah masalah karena corrupt atau ada DoS. Coba dicek pakai ini pak:

netstat -tan | grep ":80"

hasilnya copy paste kesini coba pak

priz_ · 16 Jan 2012

Ini mas, hasilnya,

netstat -tan |grep ":80"
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:1826 SYN_RECV
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:1825 SYN_RECV
tcp 0 0 202.xxx.xxx.xxx:80 222.124.249.89:64606 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 222.124.249.89:64607 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.210:13053 ESTABLISHED
tcp 0 13140 202.xxx.xxx.xxx:80 202.162.205.249:1824 ESTABLISHED
tcp 0 10220 202.xxx.xxx.xxx:80 202.162.205.249:1827 ESTABLISHED
tcp 0 13140 202.xxx.xxx.xxx:80 202.162.205.249:1828 ESTABLISHED
tcp 0 10220 202.xxx.xxx.xxx:80 202.162.205.249:1829 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 180.76.6.222:2856 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 222.124.249.89:64608 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 114.24.41.151:64797 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.4.37.21:49650 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:49502 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 222.124.249.89:64521 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.208:32673 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 182.4.37.21:49642 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.208:33707 ESTABLISHED
tcp 0 12401 202.xxx.xxx.xxx:80 210.79.217.13:9853 FIN_WAIT1
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14312 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14315 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.208:15497 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26679 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26678 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26677 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26676 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26675 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.76.244:26680 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.210:63590 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 186.93.35.129:57325 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 180.76.6.230:4021 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3263 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2486 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2488 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2489 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2491 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3249 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2492 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3248 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3250 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3247 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 114.24.41.151:60051 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 120.165.7.137:49690 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 186.93.35.129:57278 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.208:12860 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 202.162.205.249:2497 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3268 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3265 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3264 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3267 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.1.154.0:3266 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 223.255.226.206:56501 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 223.255.226.206:56502 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 223.255.226.206:56504 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 223.255.226.206:56507 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14451 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14452 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 114.24.41.151:65237 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14456 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 69.171.228.246:35269 TIME_WAIT
tcp 1 13140 202.xxx.xxx.xxx:80 182.0.134.152:10082 CLOSE_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 125.162.66.230:14459 TIME_WAIT
tcp 0 204 202.xxx.xxx.xxx:80 124.121.130.110:29167 LAST_ACK
tcp 0 0 202.xxx.xxx.xxx:80 118.96.140.195:29089 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 180.76.5.92:13953 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 118.96.140.195:29057 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 118.96.140.195:29059 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 118.96.140.195:29058 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.209:52952 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 182.11.129.18:51805 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59316 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 176.9.87.106:57853 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 176.9.87.106:57580 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 110.137.120.125:1386 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 110.137.120.125:1384 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.120.125:1385 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.125.226:35831 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 110.137.120.125:1382 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 110.137.120.125:1383 FIN_WAIT2
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59270 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59275 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59276 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 110.137.125.226:35861 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 206.53.148.209:61809 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59456 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59457 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59458 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59449 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59451 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59452 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59453 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59454 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59455 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 118.96.140.195:29055 ESTABLISHED
tcp 0 0 202.xxx.xxx.xxx:80 82.145.208.55:59416 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.11.129.18:44273 TIME_WAIT
tcp 0 0 202.xxx.xxx.xxx:80 182.11.129.18:54766 TIME_WAIT
tcp 0 0 :::80 :::* LISTEN

Ket :
* 202.xxx.xxx.xxx = ip server

priz_ · 16 Jan 2012

Kondisi sekarang masih terlihat aman mas, tapi nanti saya tampilkan ketika loadnya tinggi

twistedshells · 16 Jan 2012

@Mas Priz,

dari hasil netstat memang kelihatan normal

yang perlu dijaga ada jenis serangan yang cukup efektif untuk apache yaitu slowris, hanya kalau saya lihat dari hasil netstat, terlihat normal.

Biasanya apache segmentation fault diakibatkan oleh DoS (atau DDoS dari beberapa sumber) meski layanan lain terlihat normal. Tetapi jika melibatkan cpu high biasanya berhubungan dengan iowait

priz_ · 17 Jan 2012

Terakhir, uninstall varnish.

malah ada 1 user yg gak bisa dibuaka websitenya mas,

========
Fatal error: Class 'JPath' not found in /home/namauserxx/domains/namauserxx.com/public_html/libraries/joomla/database/table.php on line 91
========

kenapa itu yha mas,, hadeh2.

twistedshells · 18 Jan 2012

@TS: coba dilihat code file /home/namauserxx/domains/namauserxx.com/public_html/libraries/joomla/database/table.php terutama baris ke 91 atau coba ketik grep JPath /home/namauserxx/domains/namauserxx.com/public_html/libraries/joomla/database/table.php

[ASK]:: LOAD Server 100% + HTTPD

twistedshells

Apprentice 1.0

priz_

Beginner 1.0

priz_

Beginner 1.0

twistedshells

Apprentice 1.0

priz_

Beginner 1.0

priz_

Beginner 1.0

twistedshells

Apprentice 1.0

priz_

Beginner 1.0

twistedshells

Apprentice 1.0