Linux & Shell¶

Essential commands, tips, and patterns for SRE/DevOps work in Linux environments.

🔎 System Information¶

hostnamectl              # System info
uname -a                 # Kernel version & architecture
uptime                   # How long system has been running
who -a                   # Logged-in users & sessions

📊 Processes & Resources¶

top                      # Live system view
htop                     # Enhanced top (install if needed)
vmstat 1 5               # CPU, memory, IO stats
free -m                  # Memory usage in MB
ps aux --sort=-%cpu | head
ps aux --sort=-%mem | head

🗂 Filesystem & Disk¶

df -hT                   # Filesystem usage & type
lsblk -o NAME,SIZE,TYPE,MOUNTPOINT
du -sh * | sort -h       # Directory sizes
find /var/log -type f -size +200M
iostat -xz 1 5            # Disk I/O stats (sysstat package)

Permissions & Ownership¶

chmod 640 file.txt
chown app:app file.txt
getfacl file.txt
setfacl -m u:alice:r file.txt

🔧 Services (systemd)¶

systemctl status nginx
systemctl restart nginx
systemctl enable nginx
journalctl -u nginx -n 100 -f

🌐 Networking¶

ip a                     # IP addresses
ip route                 # Routing table
ss -tulpn                # Listening sockets
dig example.com          # DNS lookup
nslookup example.com
ping -c 3 1.1.1.1
traceroute google.com
mtr -rwzbc100 google.com # Network path diagnostics

Firewall¶

# ufw
sudo ufw status
sudo ufw allow 22/tcp

# firewalld
sudo firewall-cmd --list-all
sudo firewall-cmd --add-service=http --permanent && sudo firewall-cmd --reload

📡 Networking Command Reference¶

A deeper reference for interface, connectivity, DNS, and diagnostic tools — what each one does, its usage, and a working example.

Interfaces & Addressing¶

ip addr — Show IP addresses assigned to every interface. The modern, scriptable replacement for ifconfig.

ip addr                  # all interfaces
ip addr show eth0        # just one interface
ip -brief addr           # compact table view

ip link — Show or change interface state at the link (L2) layer — no IP info, just existence/state/MAC address.

ip link show
sudo ip link set eth0 up
sudo ip link set eth0 down

ip route — Show or manage the kernel routing table: which gateway/interface traffic to a given destination goes through.

ip route                        # full table
ip route get 8.8.8.8            # which route a specific destination would use
sudo ip route add default via 192.168.1.1

ifconfig — Legacy interface configuration tool (from net-tools). Deprecated in favor of ip addr/ip link, but still common on older systems.

ifconfig                 # all up interfaces
ifconfig eth0
sudo ifconfig eth0 up

Reachability & Path Testing¶

ping <host/ip> — Send ICMP echo requests to test whether a host is reachable and measure round-trip latency.

ping -c 4 8.8.8.8         # 4 packets then stop
ping -c 4 example.com

traceroute <ip/host> — Show the sequence of routers (hops) a packet passes through to reach a destination, with latency per hop.

traceroute example.com
traceroute -n 8.8.8.8      # -n: skip reverse DNS, faster output

tracepath <host> — Does the same job as traceroute, but doesn't require root/setuid privileges — the practical default when you don't have sudo.

tracepath example.com

mtr <host> — Combines ping and traceroute into one continuously-updating view per hop. The best single tool for pinpointing where latency or packet loss is actually happening on a path.

mtr example.com
mtr -rwzbc100 example.com   # report mode, 100 pings, wide/numeric - good for pasting into a ticket

DNS Lookups¶

dig <domain> — Detailed DNS query tool showing the full response, including TTLs and authority records. The standard for real DNS troubleshooting.

dig example.com               # A record
dig example.com MX            # mail exchanger records
dig +short example.com        # just the answer, no headers
dig @8.8.8.8 example.com      # query a specific resolver directly

nslookup <domain/ip> — Simpler, older DNS lookup tool. Works both directions (forward and reverse) and is available almost everywhere, including Windows.

nslookup example.com
nslookup 8.8.8.8            # reverse lookup

host <domain/ip> — The terse cousin of dig/nslookup — one-line output, handy for quick scripting checks.

host example.com
host 8.8.8.8

Sockets & Connections¶

ss -tulnp — List listening TCP/UDP sockets with the owning process. The modern, much faster replacement for netstat on busy hosts. Flags: t=TCP, u=UDP, l=listening only, n=numeric ports, p=show process.

ss -tulnp                    # everything listening
ss -tulnp | grep :443        # what's bound to a specific port

netstat -tulnp, netstat -anp — Legacy socket/connection listing tool (from net-tools); same flag meanings as ss. -anp shows all connections, not just listening ones, with the owning process.

netstat -tulnp               # listening sockets, like ss -tulnp
netstat -anp                 # all connections, established + listening

ARP & Neighbor Discovery¶

arp -a — Show the ARP cache: which MAC address the kernel has resolved for each IP on the local network segment.

arp -a

ip neigh — The modern replacement for arp -a — same neighbor table, but works for both IPv4 (ARP) and IPv6 (NDP).

ip neigh
ip neigh show dev eth0

Hostname & Public IP¶

hostname — Print (or, run as root, set) the system's hostname.

hostname

hostname -I — Print every IP address currently assigned to the host, space-separated — a fast way to get your own IP without parsing ip addr output.

hostname -I

curl ifconfig.me — Ask an external service what IP address your traffic is actually leaving from — your public IP as seen from the internet. Useful behind NAT to confirm what an external allowlist would need.

curl ifconfig.me
curl -4 ifconfig.me        # force IPv4

NetworkManager¶

nmcli device status — Show every network device NetworkManager knows about and its current state (connected, disconnected, unmanaged).

nmcli device status
nmcli device show eth0     # full detail for one device

systemctl status|restart|start NetworkManager — Check or control the NetworkManager service itself — useful when nmcli reports devices as "unmanaged" or networking silently stops working after a config change.

systemctl status NetworkManager
sudo systemctl restart NetworkManager
sudo systemctl start NetworkManager

Packet Capture & Interface Tuning¶

tcpdump -i eth0 — Capture raw packets on an interface — the ground-truth tool for seeing exactly what's on the wire instead of trusting what an application claims is happening.

sudo tcpdump -i eth0                      # everything on the interface
sudo tcpdump -i eth0 port 443             # filter to one port
sudo tcpdump -i eth0 -w capture.pcap      # write to a file for Wireshark

ethtool eth0 — Show or change low-level NIC settings: link speed, duplex, and whether the link is actually up at the driver level — useful when ip link reports UP but throughput is wrong.

ethtool eth0                 # speed, duplex, link status
sudo ethtool -s eth0 speed 1000 duplex full

iwconfig — Show or configure wireless interface parameters (SSID, signal strength, bitrate). Legacy tool, largely superseded by iw, but still widely available.

iwconfig
iwconfig wlan0

🧾 Logs¶

journalctl -xe                               # System logs
journalctl -u app --since "10 min ago"       # Service logs
grep -RniE "error|fail|timeout" /var/log     # Search logs

🐚 Shell Patterns¶

# Retry with exponential backoff
for s in 1 2 4 8 16; do cmd && break || sleep $s; done

# Tail latest log file in a directory
ls -Art /var/log/app/ | tail -1 | xargs -I{} tail -f /var/log/app/{}

# Count top callers by IP
awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

🧪 Text Processing¶

tail -500 app.log | grep -iE "timeout|500|exception"
sed -i.bak 's/DEBUG/INFO/g' config.yml
awk -F, '{sum+=$3} END {print sum}' data.csv
jq '.items[] | {name, status}' payload.json

📦 Packages¶

# Debian/Ubuntu
sudo apt update && sudo apt -y upgrade
sudo apt install -y htop curl jq

# RHEL/CentOS/Rocky
sudo dnf update -y
sudo dnf install -y htop curl jq

🔐 Users & SSH¶

sudo useradd -m -s /bin/bash app
sudo passwd -l app
mkdir -p ~app/.ssh && chmod 700 ~app/.ssh && chown -R app:app ~app/.ssh

# Add sudo permission for specific command
echo "app ALL=(ALL) NOPASSWD:/usr/bin/systemctl" | sudo tee /etc/sudoers.d/app

🛡️ Security Basics¶

Disable password SSH for privileged users; use keys + MFA
Keep systems patched
Enable audit logging
Apply noexec,nodev,nosuid to non-essential mounts
Drop unnecessary capabilities

🧰 Performance Debugging¶

mpstat -P ALL 1 5         # Per-CPU usage
pidstat 1 5               # Per-process stats
dmesg | tail -50          # Kernel messages
sar -n DEV 1 5            # Network interface stats
lsof -p <pid> | wc -l     # FD count for a process

🗓 Scheduling¶

crontab -e
crontab -l
# Example job: daily backup at 3 AM
0 3 * * * /usr/local/bin/backup.sh >>/var/log/backup.log 2>&1

📦 Tars & Transfers¶

tar czf backup_$(date +%F).tgz /etc /var/app
tar xzf backup.tgz -C /restore
rsync -avz --delete src/ dst/
scp file user@host:/path

🧯 Incident Triage Quick Commands¶

date -Is
uptime
journalctl -u app -n 200 --no-pager
ss -tulpn
curl -sS -m 5 -o /dev/null -w '%{http_code} %{time_total}\n' https://service.health/ready

🧱 Kernel & Limits¶

ulimit -a
ulimit -n 1048576
sysctl net.core.somaxconn=4096

✅ Readiness Checklist¶

[ ] set -euo pipefail in scripts
[ ] /healthz endpoint for services
[ ] Correlation IDs in logs
[ ] Local ~/bin toolkit with essentials

Master these commands to troubleshoot faster and keep systems healthy.