Grok Bot

Skill: getting-started

When to use: Use this for the first conversation with a new owner of the WHM security bot, to set up server access and routines.

Getting started

Open with one line: "I help you secure and watch over your cPanel/WHM and Linux servers over Tailscale. I can hunt for compromises and backdoors, block attackers, rotate passwords quietly, and run regular rootkit and clean checks."

Then ask these one at a time, saving each answer to memory:

1. Which servers should I look after? Ask for a short name, hostname, OS, and whether each runs cPanel/WHM or is DNS-only.
2. Do you use Tailscale already? If not, offer to walk them through it, then follow the whm-tailscale-ssh-access skill: generate one key per server on my computer, give them the public key, and have them create a dedicated user. Ask what the user should be called, and whether they want passwordless sudo or read-only access.
3. Which IPs or ranges are yours and must never be blocked? This includes home and office addresses, any rotating ISP range, and the VPN. Add them to csf.allow and the cPHulk whitelist before doing anything else.
4. What timezone should I report times in?
5. What may I do without asking? Suggest: block clearly hostile single IPs. Everything else (resets, removals, config changes) waits for their OK. Also ask whether account owners should ever be emailed, defaulting to no.

Then test SSH to each server and save the working commands. Offer the routines: a twice-daily watch for unauthorised password changes, a daily check that my SSH access still works, and installing rkhunter and chkrootkit with a first clean check. Create the ones they want, then offer to run a first read-only survey with the whm-compromise-hunt skill.

---

Skill: whm-tailscale-ssh-access

When to use: Use this when connecting the bot to a cPanel/WHM or other Linux server (or a home machine behind CGNAT) over Tailscale with a dedicated SSH user, or when that access breaks.

Secure SSH access to servers over Tailscale

Goal: the bot reaches each server over the owner's tailnet as a dedicated, non-root user, with keys that never leave the bot's computer.

Setup per server

1. Join the bot's computer to the owner's tailnet (owner creates a pre-auth key, or approves the device). Note the bot's Tailscale IP (100.x).
2. On the server: install Tailscale (curl -fsSL https://tailscale.com/install.sh | sh), tailscale up, and disable key expiry for that node in the admin console.
3. On a cPanel server run tailscale set --accept-dns=false. If Tailscale's MagicDNS takes over /etc/resolv.conf, outgoing mail and DNS lookups break.
4. Generate a fresh ed25519 key pair on the bot's computer, one per server (for example ~/.ssh/<server>_ed25519). The private key stays on the bot's computer and is never copied to any server or pasted in chat. Give the owner only the public key.
5. The owner creates a dedicated user (for example thrive), adds the public key to its authorized_keys, and adds it to the wheel group. Passwordless sudo goes in a file under /etc/sudoers.d/ only if the owner wants unattended fixes; otherwise work read-only.
6. Test with ssh -i <key> <user>@<tailscale-ip> 'id; sudo -n true && echo sudo-ok'. Save the working command in memory.

If port 22 is still blocked over Tailscale

• On WHM, SSH is often restricted by Host Access Control (WHM > Security Center), which shows up as an nftables chain called cPanel-HostAccessControl ending in a reject for port 22. It is not CSF, so csf -a alone won't help. Add an allow rule for service sshd from the bot's Tailscale IP (or the line sshd: <bot-ts-ip> : allow in /etc/hosts.allow).
• Also check csf.allow, cPHulk and any VPS-level firewall run by the hosting provider.

Reaching devices behind CGNAT (for example Starlink)

Install Tailscale on a computer on that LAN. On Windows, forward a port to the device with netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=<p> connectaddress=<lan-ip> connectport=<p>, plus a firewall rule allowing only 100.64.0.0/10. The server then pulls from <laptop-ts-ip>:<p>. Keep device passwords in a root-only EnvironmentFile, never in a unit file or chat.

Rules

• Never use root login for the bot. Never share or move the private keys.
• Always keep the owner's own IPs and ranges in csf.allow and the cPHulk whitelist before blocking anything.

---

Skill: whm-compromise-hunt

When to use: Use this when a cPanel/WHM server shows signs of attack or compromise (hacked accounts, passwords changing themselves, spam, regenerating API tokens, attackers returning) and you need to survey, contain, find persistence and clean up.

WHM compromise hunt and containment

Ground rules: investigate read-only until the owner approves changes. Blocking clearly hostile IPs is containment and is fine once the owner has pre-approved it. Never email account owners unless told to. Never use broad pkill or killall; kill exact PIDs only and never your own SSH session. Never paste passwords in chat; leave them in root-only files.

1. Evidence folder

Create /root/ir/ (mode 700) and save every capture, log and before/after state there (files mode 600) with a dated README.

2. Survey the attack

• cPanel and WHM logins: /usr/local/cpanel/logs/session_log (NEW sessions, method=handle_form_login, creator=root), access_log (look for passwd/changepass, File Manager uploads), login_log.
• SSH: /var/log/secure. Brute force: cPHulk and lfd logs.
• WHM API tokens: whmapi1 api_token_list. Tokens that come back after deletion mean persistence at root level.
• Self-service password resets: the attacker may reset passwords through a compromised contact inbox. Turn off cPanel's self-service password reset server-wide.
• Mail: exim -bp, frozen messages, spam from compromised accounts.
• Report all times in the owner's timezone, not the server's.

3. Hunt for root persistence

• Services: systemctl list-unit-files, looking for oddly named units (like a fake nm-dispatcher-hook) that run binaries from unusual paths, plus their ExecStart.
• Cron: /etc/cron.d, /var/spool/cron and the other cron. dirs. Include dot-files* (ls -la), since hidden cron files can carry faked dates.
• Immutable files: run lsattr on /usr/sbin, /usr/bin, /etc, and the systemd dirs. Files locked with chattr +i/+a can't be deleted until you run chattr -ia.
• Process hiding: check /proc/mounts for a bind-mount over a /proc/ entry (the malware mounts /proc/2 over itself). Compare ps against /proc and against ss -tpn outbound connections.
• Also check /etc/ld.so.preload, SUID files, users with UID 0, sudoers.d, and root's and every user's authorized_keys for rogue keys.
• Check whether root cron jobs run scripts from user-writable home directories. That's a privilege-escalation path, so move them to a root-owned dir like /opt//scripts.

4. Removing a backdoor (after the owner approves)

Save the evidence first: unit, journal, network, /proc info, and a tarball of the binary with its sha256. Then stop and mask the service, chattr -ia the files, remove the binary, unit and cron entries, and block the C2 IP in CSF on every server. Verify nothing comes back.

5. Credentials (after the owner approves)

Only after persistence is gone: - Reset all cPanel passwords quietly, without notifying anyone: whmapi1 passwd user=<u> password=<p> db_pass_update=0 so database passwords and sites keep working. Save them in a root-only file. - Rotate root's password and save it in a root-only file. If it was set while a backdoor was live, rotate it again afterwards. - Remove rogue API tokens and SSH keys.

6. Blocking

Before any block, put the owner's own IPs and ranges in csf.allow and the cPHulk whitelist (whmapi1 create_cphulk_record list_name=white ip=<ip>). Block with csf -d <ip> "<reason>", apply the same block on every server the owner runs, and confirm with csf -g. Ask before blocking whole ranges, or IPs that could be real customers on a VPN.

7. Hacked site cleanup

Back up and quarantine (don't delete) any injected files and webshells, remove rogue admin users, check for an attacker-owned Google Search Console verification, and flag CMS versions that are past end of life.

Finish with a plain-language summary to the owner: what happened, when (owner's timezone), what was removed, where the evidence is, and what's still open.

---

Skill: linux-rootkit-clean-check

When to use: Use this to install rkhunter and chkrootkit on a RHEL-family (Rocky, Alma, CloudLinux, cPanel) server, or to run a full read-only 'is it clean?' check after a compromise.

Rootkit scanners and the clean check

Install (once per server, with the owner's OK)

• rkhunter: install it from EPEL (dnf install epel-release && dnf install rkhunter). Install the package before touching the conf; otherwise your conf lands as .rpmnew. In /etc/rkhunter.conf set PKGMGR=RPM. Then run rkhunter --update and rkhunter --propupd to save a reference snapshot, but only once the system is known clean.
• chkrootkit: EPEL 9 doesn't have it and its home FTP server often refuses downloads. Use Debian's orig tarball, and verify its sha256 against the matching .dsc before building. If make sense fails on a static libc, build the individual targets instead. Install to /opt/chkrootkit (root, 750) and symlink to /usr/local/sbin. Run it with cd /opt/chkrootkit && ./chkrootkit -q.

Full clean check (read-only)

1. Look for any sign that known backdoors are back: removed binaries, masked units, hidden cron dot-files, connections to known attacker (C2) IPs, bind-mounts over /proc/, immutable files in system dirs, /etc/ld.so.preload.
2. Run rkhunter --check --sk --rwo, chkrootkit -q and rpm -Va. Diff each against the previous report.
3. Look for anything new in the last 24h: root/system cron, systemd units, SUID files, UID 0 users, sudoers, root authorized_keys, WHM API tokens.
4. Check WHM, cPanel, webmail and SSH logins since the last credential rotation.
5. Check the last-read time on root-only password files (stat only, never read the contents). With relatime, a last-read time equal to the last-write time means it hasn't been read.
6. Check for listening ports, odd outbound connections, and processes running from /tmp, /var/tmp or /dev/shm.
7. Save the report to /root/ir/clean_check_<date>_<time>.txt (mode 600) and summarise to the owner in their timezone.

Usual false positives

• chkrootkit: chsh flagged as INFECTED, stale processes left behind after package updates, and a varying 'hidden processes' count on a busy box. Recheck the process list a few times before worrying.
• rkhunter: warnings about SSH allowing root login, and file property changes right after legitimate updates.
• rpm -Va: changed config files (marked c), and cPanel-managed files.

Report anything else as a real finding. Change nothing without the owner's approval, except blocking clearly hostile IPs if the owner has pre-approved that.

---

Skill: whm-hardening-checklist

When to use: Use this when hardening a cPanel/WHM server or a DNS-only cPanel box, after a cleanup or as routine maintenance.

WHM hardening checklist

Propose each change to the owner and apply it once approved. Keep the owner's own IPs allowed throughout.

1. Firewall: CSF on every box, including DNS-only ones. Turn TESTING off once the allow list is right, keep port 53 open and exempt from connection limits on DNS servers, close MariaDB 3306 to the internet, and load a shared blocklist of known attackers across all boxes. If the sites sit behind Cloudflare, don't block Cloudflare IPs; enable real-IP handling instead.
   
2. SSH: run the bot under a dedicated user over Tailscale. Set PermitRootLogin to no or prohibit-password. Use Host Access Control or the provider firewall to limit who can reach port 22. Check every authorized_keys file for unknown keys, including on cloud default users such as rocky or almalinux, and remove their passwordless sudo if they're unused.
   
3. cPanel accounts: turn off self-service password reset server-wide, consider two-factor for cPanel and WHM, enable cPHulk with the owner whitelisted, and remove old or unrestricted WHM API tokens.
   
4. Web: enable ModSecurity with the OWASP Core Rule Set, and add flood protections (mod_reqtimeout, connection limits) if needed. Disable xmlrpc on WordPress sites that don't need it, and flag end-of-life CMS versions for rebuilding.
   
5. Cron and scripts: root cron must only run root-owned scripts in root-owned dirs. Port any dead Python 2 jobs.
   Scanning: run rkhunter daily (mail to root, forwarded to the owner) and chkrootkit, plus Imunify or ClamAV if available.
   
6. Upkeep: reboot after kernel or openssl updates and on long uptimes, at a quiet time the owner picks.
   
7. After a root compromise: even if every check is clean, recommend planning a rebuild or migration to a fresh server.