First 15 minutes
Live system triage — Log all output with `exec > >(tee -a /cases/$(hostname)-triage-$(date +%Y%m%d_%H%M%S).log) 2>&1`; document system state with `uname -a; uptime; id; who; w; last -10 -F`, active connections with `ss -tunap`, and process tree with `ps auxf`.
Detection of deleted but running binaries
Check for fileless/rootkit indicators with `ls -la /proc/*/exe 2>/dev/null | grep "(deleted)"`; a positive result is a signal of root-level compromise and requires escalation.
Auth.log / secure triage
Ubuntu: `/var/log/auth.log`; RHEL: `/var/log/secure`; extract successful logins with `grep "Accepted" /var/log/auth.log | awk '{print $1,$2,$3,$9,$11}' | sort | uniq -c | sort -rn` and brute-force sources with `grep "Failed password\|Invalid user" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -20`.
Auditd persistence queries
Query cron modifications with `ausearch -k cron_modify --interpret 2>/dev/null | tail -30`, systemd modifications with `ausearch -k systemd_unit_modify --interpret 2>/dev/null | tail -30`, SSH key modifications with `ausearch -k ssh_key_modify --interpret 2>/dev/null | tail -20`, and LD_PRELOAD writes with `ausearch -k ldso_preload --interpret 2>/dev/null`.
Journald triage
Check error-level events with `journalctl --since "24 hours ago" -p err..emerg --no-pager | tail -50`, sudo usage with `journalctl _COMM=sudo --since "24 hours ago" --no-pager`, and previous reboots with `journalctl --list-boots`.
Cron persistence triage
Check user crontabs with `for user in $(cut -f1 -d: /etc/passwd); do f="/var/spool/cron/crontabs/$user"; [ -f "$f" ] && echo "USER $user:" && cat "$f"; done`, system cron files with `ls -la /etc/cron.d/ /etc/cron.daily/ /etc/cron.weekly/ /etc/cron.monthly/`, and recently added cron files with `find /etc/cron* /var/spool/cron -type f -newer /etc/passwd 2>/dev/null` — Ubuntu crontab path is `/var/spool/cron/crontabs/<user>`, RHEL is `/var/spool/cron/<user>`.
Systemd persistence triage
List enabled services with `systemctl list-unit-files --type=service --state=enabled`; identify recently modified unit files with `find /etc/systemd /usr/lib/systemd ~/.config/systemd -name "*.service" -o -name "*.timer" 2>/dev/null | xargs -I{} stat --format="%n %y" {} 2>/dev/null | sort -k2`; an `ExecStart` field pointing to `/tmp`, `/dev/shm`, or a hidden directory is a strong anomaly.
Shell startup file triage
Find recently modified startup files with `find /home /root -maxdepth 2 \( -name ".bashrc" -o -name ".bash_profile" -o -name ".profile" -o -name ".zshrc" -o -name ".zshenv" \) -newer /etc/passwd 2>/dev/null`; high-priority anomalies during content analysis include `curl | bash`, `wget -O- | sh`, base64 decode chains, and `LD_PRELOAD=` assignments.
SSH authorized_keys triage
Inspect all authorized_keys files with `find /home /root -name "authorized_keys" 2>/dev/null | while read f; do echo "=== $f ==="; stat "$f"; cat "$f"; done` and recently added key files with `find /home /root -name "authorized_keys" -newer /etc/passwd 2>/dev/null`; attackers may also add a `ProxyCommand` inside `~/.ssh/config`.
LD_PRELOAD triage
Check file presence and content with `cat /etc/ld.so.preload 2>/dev/null` — the file should normally be empty or absent; `/etc/ld.so.preload` requires root privileges and affects all processes (including setuid); the `LD_PRELOAD` environment variable is ignored by glibc for setuid binaries, but `/etc/ld.so.preload` is not subject to this restriction.
SUID / capability triage
List all SUID/SGID files with `find / -type f \( -perm -4000 -o -perm -2000 \) -not -path "/proc/*" 2>/dev/null | sort`, recently added SUID files with `find / -type f -perm -4000 -newer /etc/passwd 2>/dev/null`, and dangerous capability sets (`cap_setuid+ep`, `cap_sys_admin+ep`, `cap_dac_read_search+ep`) with `getcap -r / 2>/dev/null | grep -v "^/proc"`.
Account and privilege analysis
Check users with a login shell with `grep -v "nologin\|false\|sync\|shutdown\|halt" /etc/passwd | awk -F: '{print $1, $3, $7}'`, NOPASSWD entries with `cat /etc/sudoers | grep -v "^#" | grep -v "^$"` and `cat /etc/sudoers.d/*`, and file modification times with `stat /etc/passwd /etc/shadow /etc/sudoers 2>/dev/null | grep -E "File:|Modify:"`.
Bash_history and editor histories
Check the size of history files with `find /home /root -name ".*history" 2>/dev/null | xargs ls -la 2>/dev/null`; examine secondary trace sources the attacker may have overlooked with `find /home /root \( -name ".viminfo" -o -name ".lesshst" -o -name ".mysql_history" -o -name ".python_history" \) 2>/dev/null`.
Log integrity check
Query zeroed log files with `find /var/log -type f -size 0 2>/dev/null | grep -v ".gz"`, journald integrity gaps with `journalctl --verify 2>&1 | grep -i "error\|fail\|missing\|gap"`, and auditd log deletion attempts with `ausearch -k log_tampering --interpret 2>/dev/null | tail -20`.
Compromise decision gate
Proceed to the Contain phase if two or more of the following are present: unknown authorized_keys entry, new/modified cron/systemd unit, suspicious command in .bashrc/.profile, unknown SUID or new capability, executable in /tmp / /dev/shm, evidence of log clearing, suspicious network connection within the attack window.
