Load too high and many files including iptables missing: Is my server under attack?

ananthan asked:

My server was under heavy load around 400 and above. Here is the link of Server Fault question

I was able to see the rm command and xargs running in top output, in which I was the one and only user logged in.

I tried to Kill that process but that didn’t work.

I tried to write an iptables rule to make a default policy to drop and allow only my IP to communicate, but before doing that iptables went missing. I installed it again but it showed:

FATAL: Could not load /lib/modules/2.6.32-5-vserver-amd64/modules.dep:
No such file or directory iptables v1.4.14: can’t initialize iptables
table `filter’: Table does not exist (do you need to insmod?) Perhaps
iptables or your kernel needs to be upgraded.

When I tried to shutdown the server, I was getting time out messages. Rebooting also didn’t work.

Once load came down I performed a chrootkit scan and here is the result. It shows many missing modules and hidden files.

Searching for suspicious files and dirs, it may take a while... The
following suspicious files and directories were found:
/usr/lib/pymodules/python2.6/.path /usr/lib/pymodules/python2.7/.path
/usr/lib/node_modules/npm/.npmignore
/usr/lib/node_modules/npm/.travis.yml
/usr/lib/node_modules/npm/node_modules/fast-list/.npmignore
/usr/lib/node_modules/npm/node_modules/fast-list/.travis.yml
/usr/lib/node_modules/npm/node_modules/fstream/.npmignore
/usr/lib/node_modules/npm/node_modules/fstream/.travis.yml
/usr/lib/node_modules/npm/node_modules/graceful-fs/.npmignore
/usr/lib/node_modules/npm/node_modules/lru-cache/.npmignore
/usr/lib/node_modules/npm/node_modules/minimatch/.travis.yml
/usr/lib/node_modules/npm/node_modules/node-uuid/.npmignore
/usr/lib/node_modules/npm/node_modules/nopt/.npmignore
/usr/lib/node_modules/npm/node_modules/slide/.npmignore
/usr/lib/node_modules/npm/node_modules/tar/.npmignore
/usr/lib/node_modules/npm/node_modules/tar/.travis.yml
/usr/lib/node_modules/npm/node_modules/.bin
/usr/lib/node_modules/npm/test/packages/npm-test-files/.npmignore
/usr/lib/node_modules/npm/test/packages/npm-test-ignore/.npmignore
/usr/lib/node_modules/npm/node_modules/.bin

Checking `lkm’… You have 3086 process hidden for readdir command
SIGINVISIBLE Adore found
chkproc: Warning: Possible LKM Trojan installed

Do I need to investigate further to confirm that it was an attack?

How can I get details about the way attacker got in?

My answer:


It appears that chkrootkit doesn’t like your Node.js installation since it has a large number of hidden files. Most of those look normal for a Node installation to me, though. The Python ones don’t look normal, but that could just be because you’re using Debian. Check into those.

As for the iptables problem, you are at the mercy of your VPS provider there. Since OpenVZ and Linux-VServer use a shared kernel, you can only use iptables if the provider loads it for you. In particular, Linux-VServer has very limited or no support for iptables in guest containers.

I hope that by now you have moved away from that crappy OpenVZ based VPS that you were on. That’s certainly the root cause of all the problems you have been having.


View the full question and answer on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.