π‘οΈ Blue Team Operations - Complete Reference
Your comprehensive guide to security operations, incident response, and threat hunting
π OSINT & Reconnaissance
Open-Source Intelligence (OSINT) involves collecting and analyzing publicly available information. Essential for threat intelligence, attack surface mapping, and initial reconnaissance.
πΈοΈ Maltego - Visual Link Analysis
Overview
Maltego is a powerful data mining tool that provides a graphical interface for analyzing relationships and connections between entities (domains, IPs, people, organizations, etc.).
Installation
# Download from official website
# https://www.maltego.com/downloads/
# Install using Chocolatey (alternative)
choco install maltego
# Community Edition (free) available
# Requires registration for API key
Installation
# Download from official website
wget https://downloads.maltego.com/maltego-v4/linux/Maltego.v4.X.X.deb
# Install (Ubuntu/Debian)
sudo dpkg -i Maltego.v4.X.X.deb
sudo apt-get install -f
# Kali Linux (pre-installed)
maltego
Installation
# Download from official website
# https://www.maltego.com/downloads/
# Install via Homebrew (if available)
brew install --cask maltego
# Or drag .app to Applications folder
Basic Usage
- Create new graph: File β New β Blank Graph
- Add entity: Drag from Entity Palette (left sidebar)
- Run transform: Right-click entity β Run Transform
- Common transforms: To DNS Name, To IP Address, To Email, To Domain
Key Transforms for SOC
| Transform | Purpose | Example Use |
|---|---|---|
| To DNS Name [DNS] | Resolve IP to domains | Find domains hosted on C2 IP |
| To IP Address [DNS] | Resolve domain to IPs | Map phishing domain infrastructure |
| To Whois [Domain] | Get registration info | Find registrant details |
| To Email [Whois] | Extract emails from WHOIS | Link domains by registrant |
π§ theHarvester - Email & Subdomain Enumeration
Overview
theHarvester is a simple but effective OSINT tool for gathering emails, subdomains, hosts, employee names, open ports, and banners from different public sources.
Installation
# Kali Linux (pre-installed)
theHarvester -h
# Install from source
git clone https://github.com/laramies/theHarvester
cd theHarvester
python3 -m pip install -r requirements.txt
python3 theHarvester.py -h
# Install via apt
sudo apt install theharvester -y
Installation
# Install from source
git clone https://github.com/laramies/theHarvester
cd theHarvester
pip3 install -r requirements.txt
python3 theHarvester.py -h
Installation
# Install Python first
# Download from: https://www.python.org/downloads/
# Install from source
git clone https://github.com/laramies/theHarvester
cd theHarvester
pip install -r requirements.txt
python theHarvester.py -h
Basic Usage
# Basic domain enumeration
theHarvester -d example.com -b google
# Multiple sources
theHarvester -d example.com -b google,bing,yahoo
# Limit results
theHarvester -d example.com -b google -l 500
# All sources
theHarvester -d example.com -b all
# Export to HTML
theHarvester -d example.com -b google -f results.html
Available Sources
| Source | Data Collected | API Required |
|---|---|---|
| Emails, subdomains | No | |
| bing | Emails, subdomains | No |
| certspotter | Subdomains from CT logs | No |
| crtsh | Subdomains from crt.sh | No |
| shodan | Hosts, ports, banners | Yes |
| virustotal | Subdomains | Yes |
-l to limit results and avoid being blocked.
π¦ Wireshark - Network Protocol Analyzer
Overview
Wireshark is the world's foremost network protocol analyzer. It lets you capture and interactively browse traffic running on a computer network.
Installation
# Download installer
# https://www.wireshark.org/download.html
# Install using Chocolatey
choco install wireshark
# Install using winget
winget install WiresharkFoundation.Wireshark
# Requires WinPcap or Npcap (included in installer)
Installation
# Ubuntu/Debian
sudo apt update
sudo apt install wireshark -y
# Add user to wireshark group (no sudo for capture)
sudo usermod -aG wireshark $USER
# Apply group changes (logout/login or):
newgrp wireshark
# RHEL/CentOS/Fedora
sudo yum install wireshark wireshark-qt -y
Installation
# Install via Homebrew
brew install --cask wireshark
# Or download DMG from official website
# https://www.wireshark.org/download.html
# ChmodBPF required for capture (installed with Wireshark)
Display Filters - Essential
# IP Address Filtering
ip.addr == 192.168.1.100 # Any traffic to/from IP
ip.src == 192.168.1.100 # Source IP
ip.dst == 192.168.1.100 # Destination IP
# Port Filtering
tcp.port == 80 # TCP port 80 (any direction)
udp.port == 53 # UDP port 53 (DNS)
tcp.srcport == 443 # Source port
tcp.dstport == 8080 # Destination port
# Protocol Filtering
http # All HTTP traffic
dns # All DNS traffic
tls # All TLS/SSL traffic
ssh # All SSH traffic
# HTTP Specific
http.request.method == "POST" # HTTP POST requests
http.request.uri contains "login" # URIs containing "login"
http.response.code == 200 # HTTP 200 responses
http.host == "example.com" # Specific host
http.cookie contains "session" # Cookies with "session"
# DNS Specific
dns.qry.name == "malware.com" # DNS query for domain
dns.qry.type == 1 # A records
dns.qry.type == 28 # AAAA records
dns.flags.response == 1 # DNS responses only
# TLS/SSL
tls.handshake.type == 1 # Client Hello
tls.handshake.type == 2 # Server Hello
ssl.record.content_type == 23 # Application Data
# String Search
frame contains "password" # Frame contains string
tcp contains "flag" # TCP payload contains string
http.request.uri contains ".php" # URI contains .php
# Logical Operators
ip.addr == 10.0.0.1 && tcp.port == 80 # AND
ip.addr == 10.0.0.1 || ip.addr == 10.0.0.2 # OR
!(http) # NOT (exclude HTTP)
# Advanced Filters
tcp.flags.syn == 1 && tcp.flags.ack == 0 # SYN packets (port scan)
tcp.analysis.retransmission # Retransmissions
tcp.analysis.zero_window # Zero window (performance issue)
tshark - Command Line Wireshark
# Read PCAP file
tshark -r capture.pcap
# Apply display filter
tshark -r capture.pcap -Y "http.request"
# Extract specific fields
tshark -r capture.pcap -T fields -e ip.src -e ip.dst -e tcp.port
# Statistics - Protocol Hierarchy
tshark -r capture.pcap -q -z io,phs
# Statistics - Conversations
tshark -r capture.pcap -q -z conv,tcp
# Statistics - HTTP requests
tshark -r capture.pcap -q -z http,tree
# Export HTTP objects
tshark -r capture.pcap --export-objects http,./exported/
# Follow TCP stream
tshark -r capture.pcap -z follow,tcp,ascii,0
# DNS queries only
tshark -r capture.pcap -Y "dns.flags.response == 0" -T fields -e dns.qry.name
# Extract unique IPs
tshark -r capture.pcap -T fields -e ip.src | sort -u
tshark -r capture.pcap -T fields -e ip.dst | sort -u
# Count packets by IP
tshark -r capture.pcap -T fields -e ip.src | sort | uniq -c | sort -rn
tcp.stream eq X to filter by TCP stream number for easier conversation analysis.
π Zeek (Bro) - Network Security Monitor
Overview
Zeek (formerly Bro) is a powerful network analysis framework that converts network traffic into structured logs. Unlike Wireshark, Zeek focuses on producing logs for analysis rather than packet-level inspection.
Installation
# Ubuntu/Debian
sudo apt install cmake make gcc g++ flex bison libpcap-dev libssl-dev \
python3 python3-dev swig zlib1g-dev -y
# Install from source
curl -L https://download.zeek.org/zeek-5.0.0.tar.gz | tar xz
cd zeek-5.0.0
./configure
make -j$(nproc)
sudo make install
# Add to PATH
export PATH=/usr/local/zeek/bin:$PATH
# Verify installation
zeek --version
Installation
# Install via Homebrew
brew install zeek
# Verify installation
zeek --version
Basic Usage
# Analyze PCAP file (creates logs in current directory)
zeek -r capture.pcap
# Analyze with specific scripts
zeek -r capture.pcap protocols/http/detect-sqli
# Output to specific directory
zeek -r capture.pcap LogAscii::output_to_file=./zeek_logs/
# List generated log files
ls *.log
# Common log files:
# conn.log - Connection summary
# http.log - HTTP requests/responses
# dns.log - DNS queries
# ssl.log - SSL/TLS certificates
# files.log - Files transferred
# weird.log - Unusual network activity
Log Analysis Examples
# View HTTP requests
zeek-cut method host uri < http.log
# DNS queries to specific domain
zeek-cut query answers < dns.log | grep malware.com
# SSL certificates
zeek-cut server_name < ssl.log | sort -u
# Top talkers by bytes
zeek-cut id.orig_h orig_bytes < conn.log | \
awk '{ip=$1; bytes+=$2} END {for (i in ip) print bytes[i], i}' | \
sort -rn | head -n 10
# Suspicious connections (no data transferred)
zeek-cut id.orig_h id.resp_h id.resp_p < conn.log | \
awk '$4 == 0' | head
π§ Volatility3 - Memory Forensics Framework
Overview
Volatility is the world's most widely used framework for extracting digital artifacts from volatile memory (RAM) samples.
Installation
# Install dependencies
sudo apt install python3 python3-pip git -y
# Clone Volatility3
git clone https://github.com/volatilityfoundation/volatility3.git
cd volatility3
# Install requirements
pip3 install -r requirements.txt
# Create alias (optional)
echo "alias vol='python3 $(pwd)/vol.py'" >> ~/.bashrc
source ~/.bashrc
# Test installation
python3 vol.py -h
Installation
# Install Python from python.org first
# Install Git for Windows
# https://git-scm.com/download/win
# Clone Volatility3
git clone https://github.com/volatilityfoundation/volatility3.git
cd volatility3
# Install requirements
pip install -r requirements.txt
# Run
python vol.py -h
Installation
# Install Python3 via Homebrew
brew install python3
# Clone Volatility3
git clone https://github.com/volatilityfoundation/volatility3.git
cd volatility3
# Install requirements
pip3 install -r requirements.txt
# Run
python3 vol.py -h
Essential Plugins
# Identify OS (always run first)
vol -f memory.dmp windows.info
vol -f memory.dmp linux.info
# Process Analysis
vol -f memory.dmp windows.pslist # List processes
vol -f memory.dmp windows.pstree # Process tree
vol -f memory.dmp windows.psscan # Scan for processes (finds hidden)
vol -f memory.dmp windows.cmdline # Command lines
vol -f memory.dmp windows.envars # Environment variables
# Network Analysis
vol -f memory.dmp windows.netscan # Network connections
vol -f memory.dmp windows.netstat # Active connections
# Registry Analysis
vol -f memory.dmp windows.registry.hivelist # List registry hives
vol -f memory.dmp windows.registry.printkey --key "Software\Microsoft\Windows\CurrentVersion\Run"
# File System
vol -f memory.dmp windows.filescan # Scan for file objects
vol -f memory.dmp windows.dumpfiles --pid 1234 # Dump files from process
# Malware Detection
vol -f memory.dmp windows.malfind # Find injected code
vol -f memory.dmp windows.svcscan # Scan for services
vol -f memory.dmp windows.dlllist --pid 1234 # DLLs loaded by process
# Memory Dump
vol -f memory.dmp windows.memmap --pid 1234 --dump # Dump process memory
vol -f memory.dmp windows.moddump --pid 1234 # Dump executable module
# Suspicious Activity
vol -f memory.dmp windows.handles --pid 1234 # Open handles
vol -f memory.dmp windows.mutantscan # Mutex objects
--download-symbols flag.
Common Investigation Workflow
- Identify OS:
windows.infoorlinux.info - List processes:
pslistandpstree - Find suspicious processes: Unusual names, no parent, wrong path
- Check network:
netscanfor external connections - Analyze malware:
malfindfor code injection - Dump artifacts:
dumpfiles,moddumpfor further analysis
π Splunk - SIEM Platform
Overview
Splunk is a powerful platform for searching, monitoring, and analyzing machine-generated data via a web-style interface.
SPL (Search Processing Language) Essentials
# Basic Search
index=main "error"
index=main sourcetype=access_combined status=404
# Time Range
index=main earliest=-1h
index=main earliest=-7d@d latest=@d
# Boolean Operators
index=main (error OR failed OR exception)
index=main error NOT success
# Wildcards
index=main host=web* sourcetype=*apache*
# Field Extraction
index=main | rex field=_raw "(?\d+\.\d+\.\d+\.\d+)"
# Statistical Commands
index=main | stats count by src_ip
index=main | stats sum(bytes) avg(response_time) by host
index=main | stats dc(user) as unique_users
# Table
index=main | table _time, src_ip, dest_ip, action
# Sort
index=main | stats count by src_ip | sort -count
# Timechart
index=main | timechart span=1h count by status
# Subsearch
index=main [search index=threat | fields malicious_ip | rename malicious_ip as src_ip]
# Transaction (group related events)
index=main | transaction session_id maxspan=30m
# Lookup (enrich with external data)
index=main | lookup threat_intel ip as src_ip OUTPUT threat_level
# Eval (calculate fields)
index=main | eval mb=bytes/1024/1024 | eval risk=if(mb>100, "high", "low")
# Where (filter results)
index=main | stats count by src_ip | where count > 100
# Dedup (remove duplicates)
index=main | dedup src_ip
# Top (most common values)
index=main | top limit=10 src_ip
# Rare (least common values)
index=main | rare limit=10 user_agent
SOC-Specific Queries
# Failed Login Attempts
index=security EventCode=4625 | stats count by src_ip, user | where count > 5
# Successful Login After Failed Attempts
index=security (EventCode=4625 OR EventCode=4624)
| transaction user maxspan=5m
| search EventCode=4624
| where eventcount > 1
# Port Scan Detection
index=firewall action=blocked
| stats dc(dest_port) as port_count by src_ip
| where port_count > 50
# Data Exfiltration (large outbound traffic)
index=firewall action=allowed direction=outbound
| stats sum(bytes_out) as total_bytes by src_ip
| where total_bytes > 1073741824
# Privilege Escalation
index=windows EventCode=4672
| stats count by user
| where user!="*$" AND user!="SYSTEM"
# PowerShell Download Cradle
index=windows EventCode=4104
| search "*DownloadString*" OR "*IEX*" OR "*Invoke-Expression*"
# Suspicious Process Creation
index=windows EventCode=4688
| search (NewProcessName="*powershell.exe*" OR NewProcessName="*cmd.exe*")
AND CommandLine="*-enc*" OR CommandLine="*-Encoded*"
# Lateral Movement (RDP)
index=security EventCode=4624 LogonType=10
| stats dc(ComputerName) as systems_accessed by user
| where systems_accessed > 3
tstats for faster searches on indexed fields.
π VirusTotal - Multi-Engine Malware Scanner
Overview
VirusTotal aggregates many antivirus products and online scan engines to check files, URLs, IP addresses, and domains for malicious content.
API Usage Examples
# Set API key as environment variable
export VT_API_KEY="your_api_key_here"
# File Hash Lookup (SHA256/SHA1/MD5)
curl -X GET "https://www.virustotal.com/api/v3/files/FILE_HASH" \
-H "x-apikey: $VT_API_KEY"
# Domain Report
curl -X GET "https://www.virustotal.com/api/v3/domains/example.com" \
-H "x-apikey: $VT_API_KEY"
# IP Address Report
curl -X GET "https://www.virustotal.com/api/v3/ip_addresses/8.8.8.8" \
-H "x-apikey: $VT_API_KEY"
# URL Scan
curl -X POST "https://www.virustotal.com/api/v3/urls" \
-H "x-apikey: $VT_API_KEY" \
-d "url=https://example.com"
# File Upload (< 32MB)
curl -X POST "https://www.virustotal.com/api/v3/files" \
-H "x-apikey: $VT_API_KEY" \
-F "file=@/path/to/file.exe"
# Parse JSON output with jq
curl -X GET "https://www.virustotal.com/api/v3/files/HASH" \
-H "x-apikey: $VT_API_KEY" | jq '.data.attributes.last_analysis_stats'
Python Script for Batch Analysis
import requests
import time
import json
API_KEY = "your_api_key_here"
BASE_URL = "https://www.virustotal.com/api/v3"
def check_hash(file_hash):
"""Check file hash on VirusTotal"""
url = f"{BASE_URL}/files/{file_hash}"
headers = {"x-apikey": API_KEY}
response = requests.get(url, headers=headers)
if response.status_code == 200:
data = response.json()
stats = data['data']['attributes']['last_analysis_stats']
print(f"Hash: {file_hash}")
print(f"Malicious: {stats['malicious']}")
print(f"Suspicious: {stats['suspicious']}")
print(f"Undetected: {stats['undetected']}")
print("-" * 50)
else:
print(f"Error checking {file_hash}: {response.status_code}")
time.sleep(15) # Rate limit: 4 requests/minute
# Read hashes from file
with open('hashes.txt', 'r') as f:
hashes = f.read().splitlines()
for hash_value in hashes:
check_hash(hash_value)
π³ CyberChef - The Cyber Swiss Army Knife
Overview
CyberChef is a web app for encryption, encoding, compression and data analysis. Perfect for quick transformations without writing code.
Common Recipes for SOC
1. Decode Base64
Use: Decode Base64-encoded PowerShell commands, emails, etc.
Recipe: From Base64 β (optional) Remove null bytes
2. Extract URLs/IPs
Use: Extract all URLs or IPs from log files, memory dumps
Recipe: Extract URLs β Unique β Sort
Recipe: Extract IP addresses β Unique β Sort
3. Decode Multiple Encodings
Use: When you don't know the encoding
Recipe: Magic (auto-detect encoding)
4. Convert Unix Timestamp
Use: Convert epoch timestamps to human-readable dates
Recipe: From Unix Timestamp
5. Parse User-Agent
Use: Decode user-agent strings
Recipe: Parse User Agent
6. Defang URLs/IPs
Use: Make URLs/IPs non-clickable for safe sharing
Recipe: Defang URL β Defang IP Addresses
Example: http://malware.com β hxxp://malware[.]com
7. ROT13/Caesar Cipher
Use: Decode simple obfuscation
Recipe: ROT13 or ROT47
8. Hex to ASCII
Use: Convert hex strings to readable text
Recipe: From Hex β (optional) Remove null bytes
9. Gunzip/Decompress
Use: Decompress gzip data
Recipe: Gunzip β (optional) From Base64 first
10. JWT Decode
Use: Decode JSON Web Tokens
Recipe: JWT Decode
π PowerShell for Incident Response
Overview
PowerShell is essential for Windows incident response, providing deep system access and automation capabilities.
System Information
# System Info
Get-ComputerInfo | Select-Object WindowsVersion, OsHardwareAbstractionLayer
# Installed Software
Get-ItemProperty HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\* |
Select-Object DisplayName, DisplayVersion, Publisher, InstallDate
# Running Processes
Get-Process | Select-Object ProcessName, Id, Path, Company | Sort-Object ProcessName
# Services
Get-Service | Where-Object {$_.Status -eq 'Running'} |
Select-Object Name, DisplayName, StartType
# Scheduled Tasks
Get-ScheduledTask | Where-Object {$_.State -eq 'Ready'} |
Select-Object TaskName, TaskPath, State
# Startup Programs (Registry)
Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Run"
Get-ItemProperty -Path "HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\Run"
Network Analysis
# Active Network Connections
Get-NetTCPConnection | Where-Object {$_.State -eq 'Established'} |
Select-Object LocalAddress, LocalPort, RemoteAddress, RemotePort, OwningProcess
# Get Process Name for Connection
Get-NetTCPConnection | Where-Object {$_.State -eq 'Established'} |
Select-Object @{Name="Process";Expression={(Get-Process -Id $_.OwningProcess).ProcessName}},
LocalAddress, LocalPort, RemoteAddress, RemotePort
# DNS Cache
Get-DnsClientCache | Select-Object Entry, Data
# Network Shares
Get-SmbShare
# Firewall Rules
Get-NetFirewallRule | Where-Object {$_.Enabled -eq 'True'} |
Select-Object DisplayName, Direction, Action
# ARP Table
Get-NetNeighbor | Select-Object IPAddress, LinkLayerAddress, State
Event Log Analysis
# Failed Logins (Last 24 hours)
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4625; StartTime=(Get-Date).AddDays(-1)} |
Select-Object TimeCreated, @{Name='User';Expression={$_.Properties[5].Value}},
@{Name='IP';Expression={$_.Properties[19].Value}}
# Successful Logins
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4624; StartTime=(Get-Date).AddDays(-1)} |
Select-Object TimeCreated, @{Name='User';Expression={$_.Properties[5].Value}},
@{Name='LogonType';Expression={$_.Properties[8].Value}}
# Process Creation (requires Sysmon or audit policy)
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4688; StartTime=(Get-Date).AddHours(-1)} |
Select-Object TimeCreated, @{Name='Process';Expression={$_.Properties[5].Value}},
@{Name='CommandLine';Expression={$_.Properties[8].Value}}
# PowerShell Script Block Logging
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-PowerShell/Operational'; Id=4104} |
Select-Object TimeCreated, Message | Out-GridView
# System Errors
Get-EventLog -LogName System -EntryType Error -Newest 50
File System Analysis
# Recently Modified Files
Get-ChildItem -Path C:\ -Recurse -File -ErrorAction SilentlyContinue |
Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-1)} |
Select-Object FullName, LastWriteTime, Length
# Find Files by Extension
Get-ChildItem -Path C:\Users -Recurse -Include *.exe,*.dll -ErrorAction SilentlyContinue |
Select-Object FullName, CreationTime, LastWriteTime
# Calculate File Hash
Get-FileHash -Path C:\suspicious\file.exe -Algorithm SHA256
# Batch Hash Calculation
Get-ChildItem -Path C:\suspicious -Recurse -File |
ForEach-Object {Get-FileHash $_.FullName -Algorithm SHA256} |
Select-Object Hash, Path | Export-Csv -Path hashes.csv
# Search File Content
Select-String -Path C:\logs\*.log -Pattern "malware.com"
# Alternate Data Streams (ADS)
Get-Item -Path C:\suspicious\file.txt -Stream *
User & Account Analysis
# Local Users
Get-LocalUser | Select-Object Name, Enabled, LastLogon, PasswordLastSet
# Local Administrators
Get-LocalGroupMember -Group "Administrators"
# Currently Logged In Users
quser
# User Login History
Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4624} |
Select-Object TimeCreated, @{Name='User';Expression={$_.Properties[5].Value}} |
Group-Object User | Select-Object Name, Count
| Out-GridView for interactive filtering, or | Export-Csv to save results for later analysis.
π Python for SOC Operations
Overview
Python is the most versatile language for security automation, API integration, and data analysis in SOC environments.
Essential Libraries
# HTTP Requests
pip install requests
# JSON/CSV/Excel Processing
pip install pandas openpyxl
# Network Analysis
pip install scapy
# Threat Intelligence
pip install vt-py # VirusTotal
pip install shodan # Shodan
# SIEM Integration
pip install splunk-sdk
# IP/Domain Parsing
pip install ipaddress dnspython
# Regular Expressions (built-in)
import re
Quick IOC Checker
#!/usr/bin/env python3
"""
Quick IOC checker - VirusTotal & AbuseIPDB
"""
import requests
import sys
VT_API_KEY = "your_vt_api_key"
AIPDB_API_KEY = "your_abuseipdb_key"
def check_ip_abuseipdb(ip):
"""Check IP reputation on AbuseIPDB"""
url = "https://api.abuseipdb.com/api/v2/check"
headers = {
"Key": AIPDB_API_KEY,
"Accept": "application/json"
}
params = {"ipAddress": ip, "maxAgeInDays": "90"}
response = requests.get(url, headers=headers, params=params)
if response.status_code == 200:
data = response.json()['data']
print(f"\n[AbuseIPDB] {ip}")
print(f" Abuse Score: {data['abuseConfidenceScore']}%")
print(f" Total Reports: {data['totalReports']}")
print(f" Country: {data['countryCode']}")
print(f" ISP: {data['isp']}")
else:
print(f"Error checking {ip}: {response.status_code}")
def check_hash_vt(file_hash):
"""Check file hash on VirusTotal"""
url = f"https://www.virustotal.com/api/v3/files/{file_hash}"
headers = {"x-apikey": VT_API_KEY}
response = requests.get(url, headers=headers)
if response.status_code == 200:
data = response.json()['data']['attributes']
stats = data['last_analysis_stats']
print(f"\n[VirusTotal] {file_hash}")
print(f" Malicious: {stats['malicious']}/{sum(stats.values())}")
print(f" First Seen: {data['first_submission_date']}")
print(f" Names: {', '.join(data.get('names', []))[:100]}")
else:
print(f"Hash not found or error: {response.status_code}")
if __name__ == "__main__":
if len(sys.argv) < 2:
print("Usage: python ioc_checker.py ")
sys.exit(1)
ioc = sys.argv[1]
# Determine if IP or hash
if "." in ioc: # Simple IP check
check_ip_abuseipdb(ioc)
else: # Assume hash
check_hash_vt(ioc)
Parse PCAP with Scapy
from scapy.all import rdpcap, IP, TCP, DNS
# Read PCAP file
packets = rdpcap('capture.pcap')
# Extract unique IPs
ips = set()
for pkt in packets:
if IP in pkt:
ips.add(pkt[IP].src)
ips.add(pkt[IP].dst)
print(f"Unique IPs: {len(ips)}")
for ip in sorted(ips):
print(ip)
# Extract DNS queries
dns_queries = []
for pkt in packets:
if DNS in pkt and pkt[DNS].qr == 0: # Query
dns_queries.append(pkt[DNS].qd.qname.decode())
print(f"\nDNS Queries: {len(dns_queries)}")
for query in set(dns_queries):
print(query)
# Find suspicious ports
suspicious_ports = [22, 23, 3389, 4444, 5555, 6666, 8080]
for pkt in packets:
if TCP in pkt:
if pkt[TCP].dport in suspicious_ports:
print(f"Suspicious connection: {pkt[IP].src} -> {pkt[IP].dst}:{pkt[TCP].dport}")
π Shodan - Internet-Connected Device Search Engine
Overview
Shodan is a search engine for Internet-connected devices. Unlike traditional search engines that index websites, Shodan indexes devices, servers, webcams, routers, and other Internet of Things (IoT) devices.
Common Search Operators
# Search by hostname
hostname:example.com
# Search by IP address or CIDR
net:192.168.0.0/24
# Search by port
port:22
port:3389,445
# Search by service/product
product:Apache
product:"Microsoft IIS"
# Search by organization
org:"Google LLC"
# Search by country
country:US
country:ID
# Search by city
city:"Jakarta"
# Search by operating system
os:Windows
os:"Ubuntu Linux"
# Search by vulnerabilities (CVE)
vuln:CVE-2021-44228 # Log4Shell
# Combine filters
apache country:US port:443
# Find open webcams
title:"webcam" country:ID
# Find exposed databases
product:MongoDB port:27017 -authentication
product:MySQL port:3306
# Find IoT devices
product:"Hikvision IP Camera"
product:"Mikrotik RouterOS"
# Find exposed RDP
port:3389 country:US
# Find industrial control systems
tag:ics
tag:scada
Shodan CLI
# Install Shodan CLI
pip install shodan
# Configure API key
shodan init YOUR_API_KEY
# Search from command line
shodan search "apache country:US"
# Get host information
shodan host 8.8.8.8
# Download search results
shodan download results.json.gz "apache country:US"
# Parse downloaded results
shodan parse --fields ip_str,port,org results.json.gz
# Statistics
shodan stats --facets country "apache"
# Monitor your network
shodan alert create "My Network" 192.168.0.0/24
shodan alert list
π¨ AbuseIPDB - IP Reputation Database
Overview
AbuseIPDB is a database of malicious IP addresses. Security professionals report IPs involved in malicious activities like hacking attempts, DDoS, spam, and more.
API Usage
# Set API key
export AIPDB_KEY="your_api_key_here"
# Check single IP
curl -G https://api.abuseipdb.com/api/v2/check \
--data-urlencode "ipAddress=192.168.1.1" \
-d maxAgeInDays=90 \
-H "Key: $AIPDB_KEY" \
-H "Accept: application/json" | jq
# Bulk check (up to 100 IPs)
curl -X POST https://api.abuseipdb.com/api/v2/check-block \
-d "network=192.168.0.0/24" \
-d maxAgeInDays=90 \
-H "Key: $AIPDB_KEY" \
-H "Accept: application/json"
# Report IP abuse
curl -X POST https://api.abuseipdb.com/api/v2/report \
--data-urlencode "ip=192.168.1.100" \
--data-urlencode "categories=18,22" \
--data-urlencode "comment=Port scanning attempt" \
-H "Key: $AIPDB_KEY"
# Category IDs:
# 18 = Brute Force
# 22 = DDoS Attack
# 14 = Port Scan
# 15 = Hacking
# 21 = Malware Distribution
Python Script for Batch Checking
import requests
API_KEY = "your_api_key_here"
URL = "https://api.abuseipdb.com/api/v2/check"
def check_ip(ip):
headers = {
"Key": API_KEY,
"Accept": "application/json"
}
params = {
"ipAddress": ip,
"maxAgeInDays": "90"
}
response = requests.get(URL, headers=headers, params=params)
if response.status_code == 200:
data = response.json()['data']
print(f"\nIP: {ip}")
print(f"Abuse Score: {data['abuseConfidenceScore']}%")
print(f"Total Reports: {data['totalReports']}")
print(f"Country: {data['countryCode']}")
print(f"Usage Type: {data['usageType']}")
print(f"ISP: {data['isp']}")
if data['abuseConfidenceScore'] > 75:
print("β οΈ HIGH RISK - Block this IP!")
else:
print(f"Error: {response.status_code}")
# Read IPs from file
with open('ips.txt', 'r') as f:
ips = f.read().splitlines()
for ip in ips:
check_ip(ip)
πΊοΈ Nmap - Network Mapper
Overview
Nmap is a network scanning tool used for host discovery, port scanning, service detection, and OS fingerprinting. Essential for network security audits.
Installation
# Ubuntu/Debian
sudo apt update && sudo apt install nmap -y
# RHEL/CentOS
sudo yum install nmap -y
# Verify
nmap --version
Installation
# Download installer from nmap.org
# https://nmap.org/download.html
# Install using Chocolatey
choco install nmap -y
# Or use winget
winget install Nmap.Nmap
Installation
# Install via Homebrew
brew install nmap
# Verify
nmap --version
Basic Scans
# Scan single host
nmap 192.168.1.1
# Scan subnet
nmap 192.168.1.0/24
# Scan multiple hosts
nmap 192.168.1.1,10,20
# Scan range
nmap 192.168.1.1-254
# Scan from file
nmap -iL targets.txt
# Fast scan (100 most common ports)
nmap -F 192.168.1.1
# Scan all ports
nmap -p- 192.168.1.1
# Scan specific ports
nmap -p 22,80,443 192.168.1.1
nmap -p 1-1000 192.168.1.1
Discovery & Service Detection
# Ping scan (host discovery only)
nmap -sn 192.168.1.0/24
# TCP SYN scan (default, requires root)
sudo nmap -sS 192.168.1.1
# TCP connect scan (no root required)
nmap -sT 192.168.1.1
# UDP scan
sudo nmap -sU 192.168.1.1
# Service version detection
nmap -sV 192.168.1.1
# OS detection
sudo nmap -O 192.168.1.1
# Aggressive scan (OS detection + version + scripts + traceroute)
sudo nmap -A 192.168.1.1
# Script scan
nmap -sC 192.168.1.1
NSE Scripts (Nmap Scripting Engine)
# List all scripts
ls /usr/share/nmap/scripts/ | grep .nse
# Run default scripts
nmap --script=default 192.168.1.1
# Run specific script
nmap --script=http-title 192.168.1.1
# Multiple scripts
nmap --script=http-enum,http-headers 192.168.1.1
# Vulnerability detection
nmap --script=vuln 192.168.1.1
# SMB enumeration
nmap --script=smb-enum-shares,smb-enum-users 192.168.1.1
# Check for EternalBlue (MS17-010)
nmap --script=smb-vuln-ms17-010 192.168.1.1
# SSL/TLS checks
nmap --script=ssl-enum-ciphers -p 443 example.com
Output Options
# Normal output
nmap -oN scan.txt 192.168.1.1
# XML output
nmap -oX scan.xml 192.168.1.1
# Grepable output
nmap -oG scan.grep 192.168.1.1
# All formats
nmap -oA scan 192.168.1.1
# Verbose output
nmap -v 192.168.1.1
nmap -vv 192.168.1.1 # More verbose
π¦ tcpdump - Packet Capture Tool
Overview
tcpdump is a command-line packet analyzer. It captures network packets and displays them in a human-readable format. Essential for quick packet captures and troubleshooting.
Installation
# Ubuntu/Debian (usually pre-installed)
sudo apt install tcpdump -y
# List interfaces
tcpdump -D
Installation
# tcpdump comes pre-installed on macOS
# List interfaces
tcpdump -D
Basic Capture
# Capture on default interface
sudo tcpdump
# Capture on specific interface
sudo tcpdump -i eth0
# Capture to file (pcap format)
sudo tcpdump -i eth0 -w capture.pcap
# Read from file
tcpdump -r capture.pcap
# Limit packet count
sudo tcpdump -c 100
# Verbose output
sudo tcpdump -v
sudo tcpdump -vv # More verbose
sudo tcpdump -vvv # Very verbose
# Show packet contents (hex + ASCII)
sudo tcpdump -X
# Show only hex
sudo tcpdump -xx
Filters
# Host filters
sudo tcpdump host 192.168.1.100
sudo tcpdump src 192.168.1.100
sudo tcpdump dst 192.168.1.100
# Network filters
sudo tcpdump net 192.168.1.0/24
# Port filters
sudo tcpdump port 80
sudo tcpdump portrange 80-443
sudo tcpdump src port 443
sudo tcpdump dst port 22
# Protocol filters
sudo tcpdump tcp
sudo tcpdump udp
sudo tcpdump icmp
sudo tcpdump arp
# Combine filters (AND)
sudo tcpdump host 192.168.1.100 and port 80
# OR condition
sudo tcpdump host 192.168.1.100 or host 192.168.1.200
# NOT condition
sudo tcpdump not port 22
# TCP flags
sudo tcpdump 'tcp[tcpflags] & tcp-syn != 0' # SYN packets
sudo tcpdump 'tcp[tcpflags] & tcp-rst != 0' # RST packets
# HTTP traffic
sudo tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
# DNS queries
sudo tcpdump -i eth0 udp port 53
# ICMP packets
sudo tcpdump icmp
Practical Examples
# Capture HTTP traffic and show ASCII
sudo tcpdump -A -s 0 port 80
# Capture only SYN packets (port scanning detection)
sudo tcpdump 'tcp[13] & 2 == 2'
# Capture traffic between two hosts
sudo tcpdump host 192.168.1.1 and host 192.168.1.2
# Capture all traffic except SSH
sudo tcpdump not port 22
# Rotate capture files (1GB each)
sudo tcpdump -i eth0 -w capture.pcap -C 1000 -W 5
# Capture with timestamp
sudo tcpdump -tttt
# Filter by packet size
sudo tcpdump greater 1000 # Packets larger than 1000 bytes
sudo tcpdump less 100 # Packets smaller than 100 bytes
-s 0 to capture full packets (default is 262144 bytes). Use -n to skip DNS resolution for faster captures.
π Sysmon - System Monitor
Overview
Sysmon (System Monitor) is a Windows system service that monitors and logs system activity to the Windows event log. It provides detailed information about process creations, network connections, file changes, and more.
Installation
# Download Sysmon from Microsoft Sysinternals
# https://docs.microsoft.com/sysinternals/downloads/sysmon
# Install with default config
Sysmon64.exe -accepteula -i
# Install with SwiftOnSecurity config (recommended)
# Download config: https://github.com/SwiftOnSecurity/sysmon-config
Sysmon64.exe -accepteula -i sysmonconfig-export.xml
# Update configuration
Sysmon64.exe -c updated-config.xml
# Uninstall
Sysmon64.exe -u
Event IDs
| Event ID | Description | Use Case |
|---|---|---|
| 1 | Process Creation | Track malware execution, suspicious processes |
| 3 | Network Connection | Detect C2 communication, data exfiltration |
| 7 | Image/DLL Loaded | Identify DLL injection, malicious libraries |
| 8 | CreateRemoteThread | Detect process injection techniques |
| 10 | Process Access | Credential dumping (LSASS access) |
| 11 | File Created | Track ransomware, dropped files |
| 12/13/14 | Registry Events | Persistence mechanisms, autoruns |
| 22 | DNS Query | Malicious domain lookups |
Query Sysmon Logs
# View recent process creations (Event ID 1)
Get-WinEvent -LogName "Microsoft-Windows-Sysmon/Operational" -MaxEvents 100 |
Where-Object {$_.Id -eq 1} |
Select-Object TimeCreated, Message
# Network connections (Event ID 3)
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Sysmon/Operational'; Id=3} |
Select-Object TimeCreated, @{Name='SourceIP';Expression={$_.Properties[8].Value}},
@{Name='DestIP';Expression={$_.Properties[14].Value}},
@{Name='DestPort';Expression={$_.Properties[16].Value}}
# DNS queries (Event ID 22)
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Sysmon/Operational'; Id=22} |
Select-Object TimeCreated, @{Name='Query';Expression={$_.Properties[5].Value}}
# Process accessing LSASS (credential dumping detection)
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Sysmon/Operational'; Id=10} |
Where-Object {$_.Properties[9].Value -like "*lsass.exe*"}
# PowerShell executions
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Sysmon/Operational'; Id=1} |
Where-Object {$_.Properties[4].Value -like "*powershell*"}
# Suspicious network connections to rare ports
Get-WinEvent -FilterHashtable @{LogName='Microsoft-Windows-Sysmon/Operational'; Id=3} |
Where-Object {$_.Properties[16].Value -in @(4444, 5555, 6666, 8080)}
π» osquery - SQL for Operating Systems
Overview
osquery exposes an operating system as a high-performance relational database. You can write SQL queries to explore system data like processes, users, network connections, and more.
Installation
# Ubuntu/Debian
wget https://pkg.osquery.io/deb/osquery_5.10.2-1.linux_amd64.deb
sudo dpkg -i osquery_5.10.2-1.linux_amd64.deb
# Or add repository
export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys $OSQUERY_KEY
sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb deb main'
sudo apt update && sudo apt install osquery -y
# Start service
sudo systemctl enable osqueryd
sudo systemctl start osqueryd
Installation
# Download MSI installer
# https://osquery.io/downloads/official/5.10.2
# Install using Chocolatey
choco install osquery -y
# Start service
Start-Service osqueryd
Installation
# Install via Homebrew
brew install --cask osquery
# Start service
sudo launchctl load /Library/LaunchDaemons/com.facebook.osqueryd.plist
Interactive Shell Queries
-- Start interactive shell
osqueryi
-- List all tables
.tables
-- Show table schema
.schema processes
-- Running processes
SELECT pid, name, path, cmdline FROM processes;
-- Processes listening on network ports
SELECT DISTINCT pid, name, protocol, local_port
FROM processes p
JOIN listening_ports lp ON p.pid = lp.pid;
-- Established network connections
SELECT pid, local_address, local_port, remote_address, remote_port
FROM process_open_sockets
WHERE state = 'ESTABLISHED';
-- Users with UID 0 (root/admin)
SELECT * FROM users WHERE uid = 0;
-- Logged in users
SELECT * FROM logged_in_users;
-- Installed applications (macOS)
SELECT name, version, path FROM apps;
-- Startup items
SELECT name, path, status FROM startup_items;
-- Chrome extensions
SELECT name, version, author, path FROM chrome_extensions;
-- USB devices
SELECT vendor, model, serial FROM usb_devices;
-- Running kernel modules (Linux)
SELECT name, size, used_by FROM kernel_modules;
-- Suspicious executables in temp folders
SELECT path, size, mode FROM file
WHERE path LIKE '/tmp/%' AND mode LIKE '%x%';
-- Hash files
SELECT path, md5, sha256 FROM hash
WHERE path = '/bin/bash';
-- Cron jobs
SELECT * FROM crontab;
Threat Hunting Queries
-- Processes with network connections
SELECT p.name, p.path, p.cmdline, pos.remote_address, pos.remote_port
FROM processes p
JOIN process_open_sockets pos ON p.pid = pos.pid
WHERE pos.state = 'ESTABLISHED';
-- Unsigned executables (Windows)
SELECT path, name FROM processes
WHERE path NOT IN (SELECT path FROM authenticode WHERE result = 'trusted');
-- Persistence via Registry (Windows)
SELECT * FROM registry
WHERE key LIKE 'HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run%';
-- SSH authorized keys
SELECT * FROM users u
CROSS JOIN authorized_keys ak ON u.directory = ak.uid_dir;
-- Recently modified files
SELECT path, mtime, size FROM file
WHERE path LIKE '/home/%'
AND mtime > (SELECT unix_time FROM time) - 3600;
-- Cleartext passwords in memory (risky!)
SELECT pid, name FROM process_memory
WHERE pattern LIKE '%password%';
π¬ Autopsy - Digital Forensics Platform
Overview
Autopsy is an open-source digital forensics platform with a GUI interface. It's built on top of The Sleuth Kit and provides comprehensive disk imaging, file recovery, timeline analysis, and artifact extraction.
Installation
# Download installer from
# https://www.autopsy.com/download/
# Install MSI package (includes Java runtime)
# autopsy-4.21.0-64bit.msi
# Launch from Start Menu
# Autopsy 4.21.0
Installation
# Install dependencies
sudo apt install testdisk openjdk-11-jdk -y
# Download and extract
wget https://github.com/sleuthkit/autopsy/releases/download/autopsy-4.21.0/autopsy-4.21.0.zip
unzip autopsy-4.21.0.zip
cd autopsy-4.21.0
# Run Autopsy
./bin/autopsy
Key Features
- Timeline Analysis: Visualize file system events chronologically
- Keyword Search: Search across all files and slack space
- Hash Filtering: Filter known good/bad files using NSRL, HashKeeper
- Web Artifacts: Extract browsing history, cookies, downloads
- Email Analysis: Parse PST, MBOX files
- Registry Viewer: Analyze Windows registry hives
- File Type Detection: Identify files by signature, not extension
- Ingest Modules: Automated analysis (Recent Activity, Encryption, etc.)
Typical Forensic Workflow
1. Create New Case
- Case Name: incident_2024_001
- Case Number: IR-2024-001
- Examiner: Your Name
2. Add Data Source
- Disk Image: evidence.E01 / evidence.dd
- Logical Files: folder containing files
- Local Disk: C:\ (live analysis)
3. Configure Ingest Modules
β
Recent Activity (browser, downloads)
β
Hash Lookup (NSRL, known malware)
β
File Type Identification
β
Extension Mismatch Detector
β
Embedded File Extractor
β
Email Parser
β
Encryption Detection
β
Interesting Files Identifier
β
PhotoRec Carver (deleted files)
4. Analysis
- Review "Interesting Items"
- Check "Web History"
- Examine "Recent Documents"
- Look at "Installed Programs"
- Check Timeline for suspicious activity
5. Generate Report
- File β Generate Report
- HTML or Excel format
πΎ FTK Imager - Forensic Imaging Tool
Overview
FTK (Forensic Toolkit) Imager is a free tool for creating forensic disk images and previewing evidence. It supports multiple image formats and provides data preview, mounting, and basic analysis capabilities.
Creating Forensic Images
1. File β Create Disk Image
2. Select Source
- Physical Drive (entire disk)
- Logical Drive (C:, D:, etc.)
- Image File (mount existing image)
- Contents of Folder (logical files)
3. Choose Image Type
- Raw (dd) - Simple, compatible
- SMART (.s01) - AccessData format
- E01 (EnCase) - Industry standard, compressed
- AFF (Advanced Forensic Format)
4. Evidence Item Information
- Case Number
- Evidence Number
- Description
- Examiner Name
- Notes
5. Image Destination
- Image Filename: evidence_2024_001
- Fragment Size: 4GB (for FAT32)
- Compression: 6 (recommended for E01)
- β
Verify images after creation
- β
Create directory listings
6. Start Imaging
- Monitor progress
- Verify MD5/SHA1 hashes match
CLI Usage (ftkimager.exe)
REM Image physical drive
ftkimager.exe \\.\PhysicalDrive0 "D:\Evidence\drive0" --e01 --compress 6 --verify
REM Image logical drive
ftkimager.exe C: "D:\Evidence\C_drive" --e01
REM List drives
ftkimager.exe --list-drives
REM Mount image as read-only drive
ftkimager.exe --mount "D:\Evidence\image.E01" --drive-letter X:
REM Export files from image
ftkimager.exe "image.E01" "output_folder" --export-dir "/Users/victim/Documents"
Key Features
- Evidence Tree: Browse files within images without mounting
- File Preview: View file contents, hex, properties
- Hash Verification: MD5 and SHA1 for integrity
- Memory Capture: Dump RAM from running system
- File Export: Extract specific files from images
- Registry Viewer: View Windows registry hives
- Custom Content Sources: Add evidence from various sources
π€ strings - Extract Text from Binaries
Overview
The strings utility extracts printable character sequences from binary files. Essential for quick malware triage, finding IOCs, identifying packed binaries, and extracting URLs, IPs, and file paths.
Usage (GNU strings - usually pre-installed)
# Basic strings extraction
strings malware.exe
# Show minimum length (default 4)
strings -n 8 malware.exe
# Search for specific patterns
strings malware.exe | grep -i "http"
strings malware.exe | grep -E "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" # IPs
strings malware.exe | grep -i "password"
strings malware.exe | grep -i "cmd.exe"
# Include offset in output
strings -t x malware.exe # Hex offset
strings -t d malware.exe # Decimal offset
# All encodings (ASCII + Unicode)
strings -e l malware.exe # 16-bit little-endian
strings -e b malware.exe # 16-bit big-endian
strings -e s malware.exe # 7-bit
strings -e S malware.exe # 8-bit
# Extract to file
strings malware.exe > malware_strings.txt
# Count unique strings
strings malware.exe | sort -u | wc -l
Usage (Sysinternals Strings)
REM Download from https://live.sysinternals.com/strings.exe
REM Basic extraction
strings.exe malware.exe
REM Unicode strings
strings.exe -u malware.exe
REM Both ASCII and Unicode
strings.exe -a malware.exe
REM Minimum length
strings.exe -n 10 malware.exe
REM Save to file
strings.exe malware.exe > strings.txt
REM Search for patterns
strings.exe malware.exe | findstr /I "http"
strings.exe malware.exe | findstr /I "password"
Usage (pre-installed)
# Same as Linux
strings malware.bin
strings -n 8 malware.bin | grep -i "http"
Malware Analysis Workflow
# Extract all strings
strings -n 6 -a malware.exe > all_strings.txt
# Find URLs
strings malware.exe | grep -Eo "https?://[^\s\"']+"
# Find IP addresses
strings malware.exe | grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}'
# Find file paths
strings malware.exe | grep -E "^[A-Z]:\\\\"
strings malware.exe | grep -E "^/[a-z/]+"
# Find registry keys
strings malware.exe | grep "HKEY_"
# Find email addresses
strings malware.exe | grep -Eo '[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}'
# Find domains
strings malware.exe | grep -Eo "[a-zA-Z0-9.-]+\.(com|net|org|ru|cn|xyz)"
# Find suspicious API calls
strings malware.exe | grep -i "CreateRemoteThread"
strings malware.exe | grep -i "VirtualAllocEx"
strings malware.exe | grep -i "WriteProcessMemory"
strings malware.exe | grep -i "WinExec"
# Find encoding indicators
strings malware.exe | grep -i "base64"
strings malware.exe | grep -i "encode"
strings malware.exe | grep -i "decrypt"
# Compare strings before/after execution
strings malware.exe > before.txt
# Run in sandbox
strings malware_unpacked.exe > after.txt
diff before.txt after.txt
π― YARA - Pattern Matching for Malware
Overview
YARA is a tool for identifying and classifying malware based on textual or binary patterns. Security researchers use YARA rules to describe malware families and detect threats.
Installation
# Ubuntu/Debian
sudo apt install yara -y
# From source (latest version)
sudo apt install automake libtool make gcc pkg-config -y
wget https://github.com/VirusTotal/yara/archive/refs/tags/v4.3.2.tar.gz
tar -xzf v4.3.2.tar.gz && cd yara-4.3.2
./bootstrap.sh
./configure --enable-cuckoo --enable-magic
make && sudo make install
sudo ldconfig
# Python module
pip install yara-python
Installation
# Download pre-built binaries
# https://github.com/VirusTotal/yara/releases
# Or use Chocolatey
choco install yara -y
# Python module
pip install yara-python
Installation
# Install via Homebrew
brew install yara
# Python module
pip install yara-python
Basic YARA Rule Syntax
rule SimpleMalware
{
meta:
description = "Detects simple malware"
author = "SOC Analyst"
date = "2024-01-15"
reference = "https://example.com/analysis"
strings:
$str1 = "malicious_string" ascii wide
$str2 = "C:\\Windows\\Temp\\payload.exe" nocase
$hex1 = { 4D 5A 90 00 } // MZ header
$hex2 = { E8 ?? ?? ?? ?? } // CALL instruction with wildcard
$regex1 = /https?:\/\/[a-z0-9.-]+\.(ru|cn)/ nocase
condition:
($str1 or $str2) and $hex1 and filesize < 500KB
}
rule APT_Ransomware
{
meta:
malware_family = "Ransomware"
threat_level = "high"
strings:
$ransom_note = "Your files are encrypted" nocase
$bitcoin = /[13][a-km-zA-HJ-NP-Z1-9]{25,34}/ // Bitcoin address
$file_ext = ".encrypted" nocase
$api1 = "CryptEncrypt" ascii
$api2 = "CreateMutex" ascii
condition:
3 of them and filesize > 100KB
}
rule Suspicious_PowerShell_Download
{
strings:
$ps1 = "powershell" nocase
$download1 = "Net.WebClient" nocase
$download2 = "DownloadFile" nocase
$download3 = "Invoke-WebRequest" nocase
$download4 = "wget" nocase
$download5 = "curl" nocase
$iex = "IEX" nocase
$invoke = "Invoke-Expression" nocase
condition:
$ps1 and (any of ($download*)) and ($iex or $invoke)
}
Running YARA Scans
# Scan single file
yara rules.yar suspicious.exe
# Scan directory recursively
yara -r rules.yar /path/to/samples/
# Show matched strings
yara -s rules.yar malware.exe
# Print tags
yara -g rules.yar malware.exe
# Scan process memory (Linux, requires root)
sudo yara rules.yar /proc/1234/mem
# Multiple rule files
yara rule1.yar rule2.yar rule3.yar sample.exe
# Timeout for slow scans
yara --timeout=60 rules.yar large_file.bin
# Use compiled rules (faster)
yarac rules.yar compiled_rules.yac
yara compiled_rules.yac sample.exe
Python Integration
import yara
# Compile rule from file
rules = yara.compile(filepath='rules.yar')
# Scan file
matches = rules.match('/path/to/sample.exe')
for match in matches:
print(f"Rule: {match.rule}")
print(f"Tags: {match.tags}")
print(f"Strings:")
for string in match.strings:
print(f" - {string}")
# Scan directory
import os
for root, dirs, files in os.walk('/samples/'):
for file in files:
filepath = os.path.join(root, file)
matches = rules.match(filepath)
if matches:
print(f"[!] {filepath}: {[m.rule for m in matches]}")
Popular YARA Rule Repositories
# Awesome YARA (curated list)
git clone https://github.com/InQuest/awesome-yara.git
# YARA Rules by Florian Roth
git clone https://github.com/Neo23x0/signature-base.git
# ReversingLabs YARA rules
git clone https://github.com/reversinglabs/reversinglabs-yara-rules.git
# ESET YARA rules
git clone https://github.com/eset/malware-ioc.git
π ELK Stack - Elasticsearch, Logstash, Kibana
Overview
The ELK Stack is a collection of three open-source tools: Elasticsearch (search/analytics), Logstash (data processing pipeline), and Kibana (visualization). Used for centralized logging, SIEM, and security analytics.
Installation (Docker Compose - Quick Start)
# docker-compose.yml
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
ports:
- "9200:9200"
volumes:
- esdata:/usr/share/elasticsearch/data
kibana:
image: docker.elastic.co/kibana/kibana:8.11.0
ports:
- "5601:5601"
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
depends_on:
- elasticsearch
logstash:
image: docker.elastic.co/logstash/logstash:8.11.0
ports:
- "5000:5000"
- "9600:9600"
volumes:
- ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
depends_on:
- elasticsearch
volumes:
esdata:
# Start ELK Stack
docker-compose up -d
# Access Kibana
# http://localhost:5601
# Check Elasticsearch
curl http://localhost:9200
Logstash Configuration Example
# logstash.conf
input {
beats {
port => 5044
}
tcp {
port => 5000
codec => json
}
file {
path => "/var/log/auth.log"
start_position => "beginning"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGLINE}" }
}
date {
match => [ "timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
if [source_ip] {
geoip {
source => "source_ip"
target => "geoip"
}
}
# Enrich with threat intelligence
if [ip] {
elasticsearch {
hosts => ["localhost:9200"]
index => "threat_intel"
query_template => "ip_lookup.json"
fields => { "threat_score" => "threat_score" }
}
}
}
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "logs-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
Elasticsearch Queries (DSL)
// Search for failed SSH logins
GET /logs-*/_search
{
"query": {
"bool": {
"must": [
{ "match": { "event_type": "authentication" }},
{ "match": { "status": "failed" }},
{ "match": { "service": "ssh" }}
],
"filter": {
"range": {
"@timestamp": {
"gte": "now-24h"
}
}
}
}
},
"aggs": {
"top_source_ips": {
"terms": {
"field": "source_ip.keyword",
"size": 10
}
}
}
}
// Detect brute force (>10 failures in 5 minutes)
GET /logs-*/_search
{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "now-5m"
}
}
},
"aggs": {
"by_source": {
"terms": {
"field": "source_ip.keyword",
"min_doc_count": 10
}
}
}
}
// Find PowerShell encoded commands
GET /logs-*/_search
{
"query": {
"bool": {
"must": [
{ "match": { "process_name": "powershell.exe" }},
{ "regexp": { "command_line": ".*-enc.*" }}
]
}
}
}
Kibana Query Language (KQL)
# Failed logins
event.action:"authentication_failure"
# Specific user
user.name:"admin" AND event.outcome:"failure"
# Multiple values
event.code:(4625 OR 4624) # Windows Event IDs
# IP range
source.ip >= 192.168.1.0 AND source.ip <= 192.168.1.255
# Wildcards
process.name:powershell*
# NOT operator
NOT user.name:"admin"
# Time range
@timestamp >= "2024-01-01" AND @timestamp < "2024-01-02"
# Port scanning detection
destination.port > 1000 AND destination.port < 65535
AND event.action:"network_connection"
π Burp Suite - Web Application Security Testing
Overview
Burp Suite is an integrated platform for web application security testing. It includes a proxy, scanner, intruder, repeater, and many other tools for finding and exploiting vulnerabilities.
Installation
# Download from https://portswigger.net/burp/communitydownload
# Install (requires Java)
chmod +x burpsuite_community_linux_*.sh
./burpsuite_community_linux_*.sh
# Or use JAR directly
java -jar burpsuite_community.jar
Installation
# Download installer from
# https://portswigger.net/burp/communitydownload
# Run .exe installer
# burpsuite_community_windows-x64_*.exe
Installation
# Download from https://portswigger.net/burp/communitydownload
# Install DMG
# Double-click burpsuite_community_macos_*.dmg
Proxy Configuration
1. Start Burp β Proxy tab β Intercept is on
Default: 127.0.0.1:8080
2. Configure Browser Proxy
Firefox: Settings β Network β Manual Proxy
- HTTP Proxy: 127.0.0.1
- Port: 8080
- β
Use this proxy for HTTPS
3. Install Burp CA Certificate
- Browse to http://burp
- Download cacert.der
- Import to browser/system certificates
4. Intercept Traffic
- Browse target site
- View/modify requests in Proxy β Intercept
- Forward or Drop packets
Key Features
| Tool | Purpose |
|---|---|
| Proxy | Intercept and modify HTTP/S traffic |
| Repeater | Manually modify and resend requests |
| Intruder | Automated attacks (fuzzing, brute force) |
| Decoder | Encode/decode data (Base64, URL, HTML) |
| Comparer | Compare responses to find differences |
| Scanner | Automated vulnerability scanning (Pro only) |
Common Testing Workflow
# 1. Map Application (Spider)
Target β Site map β Right-click domain β Spider this host
# 2. Identify Injection Points
- Look for parameters in GET/POST
- Check cookies, headers, JSON bodies
# 3. Test with Repeater
- Right-click request β Send to Repeater
- Modify parameters, observe responses
- Test: SQLi, XSS, command injection
# 4. Fuzz with Intruder
- Send to Intruder
- Mark payload positions: Β§paramΒ§
- Choose attack type (Sniper, Battering Ram, Pitchfork)
- Load wordlist
- Start attack, analyze results
# 5. Common Payloads to Test
SQL Injection: ' OR '1'='1, 1' UNION SELECT NULL--
XSS: ,
Command Injection: ; whoami, | cat /etc/passwd
Path Traversal: ../../../etc/passwd
SSRF: http://localhost:8080, http://169.254.169.254
# 6. Check for Vulnerabilities
- Missing security headers
- Sensitive data exposure
- Insecure authentication
- CSRF tokens missing
π·οΈ Nikto - Web Server Scanner
Overview
Nikto is an open-source web server scanner that performs comprehensive tests against web servers for dangerous files, outdated versions, and security misconfigurations.
Installation
# Ubuntu/Debian
sudo apt install nikto -y
# From GitHub (latest)
git clone https://github.com/sullo/nikto.git
cd nikto/program
perl nikto.pl -h
Installation
# Install via Homebrew
brew install nikto
# Or from GitHub
git clone https://github.com/sullo/nikto.git
cd nikto/program
perl nikto.pl -h
Basic Scanning
# Basic scan
nikto -h http://example.com
# HTTPS scan
nikto -h https://example.com
# Scan specific port
nikto -h example.com -p 8080
# Multiple ports
nikto -h example.com -p 80,443,8080
# Use proxy
nikto -h example.com -useproxy http://127.0.0.1:8080
# Save output
nikto -h example.com -o report.html -Format html
nikto -h example.com -o report.txt -Format txt
nikto -h example.com -o report.xml -Format xml
# Scan multiple hosts from file
nikto -h targets.txt
# Update plugins
nikto -update
# Tuning (select test types)
nikto -h example.com -Tuning 1234
# 1 = Interesting File / Seen in logs
# 2 = Misconfiguration
# 3 = Information Disclosure
# 4 = Injection (XSS/Script/HTML)
# 5 = Remote File Retrieval
# 6 = Denial of Service
# 7 = Remote File Retrieval (inside web root)
# 8 = Command Execution / Remote shell
# 9 = SQL Injection
# 0 = File Upload
# a = Authentication Bypass
# b = Software Identification
# c = Remote Source Inclusion
# Skip specific tests
nikto -h example.com -Tuning x 9 # Skip SQL injection tests
# Display options
nikto -h example.com -Display V # Verbose
Common Findings
- Server Version Disclosure: Apache/2.4.41, nginx/1.18.0
- Missing Headers: X-Frame-Options, X-Content-Type-Options
- Default Files: phpinfo.php, test.php, /admin/
- Directory Listings: Enabled on /uploads/, /images/
- Outdated Software: Known CVEs for detected versions
- SSL/TLS Issues: Weak ciphers, expired certificates
- HTTP Methods: Dangerous methods enabled (PUT, DELETE)
π jq - JSON Processor
Overview
jq is a lightweight command-line JSON processor. Essential for parsing API responses, log files, and security tool outputs in JSON format.
Installation
# Ubuntu/Debian
sudo apt install jq -y
# RHEL/CentOS
sudo yum install jq -y
Installation
# Download from https://stedolan.github.io/jq/download/
# Or use Chocolatey
choco install jq -y
# Or use Scoop
scoop install jq
Installation
# Install via Homebrew
brew install jq
Basic Usage
# Pretty print JSON
echo '{"name":"John","age":30}' | jq '.'
cat file.json | jq '.'
# Extract single field
echo '{"name":"John","age":30}' | jq '.name'
# Output: "John"
# Extract nested field
echo '{"user":{"name":"John","email":"john@example.com"}}' | jq '.user.email'
# Extract array element
echo '["apple","banana","cherry"]' | jq '.[1]'
# Output: "banana"
# Get array length
echo '[1,2,3,4,5]' | jq 'length'
# Get all keys
echo '{"name":"John","age":30,"city":"NYC"}' | jq 'keys'
# Multiple fields
echo '{"name":"John","age":30,"city":"NYC"}' | jq '.name, .age'
# Create new object
echo '{"name":"John","age":30}' | jq '{username: .name, user_age: .age}'
Array Operations
# Iterate array
echo '[{"name":"John"},{"name":"Jane"}]' | jq '.[]'
# Extract field from each array element
echo '[{"name":"John","age":30},{"name":"Jane","age":25}]' | jq '.[].name'
# Output:
# "John"
# "Jane"
# Filter array
echo '[1,2,3,4,5]' | jq '.[] | select(. > 3)'
# Output: 4, 5
# Map over array
echo '[1,2,3]' | jq 'map(. * 2)'
# Output: [2,4,6]
# Sort array
echo '[3,1,4,1,5]' | jq 'sort'
# Unique values
echo '[1,2,2,3,3,3]' | jq 'unique'
# Group by field
echo '[{"type":"A","value":1},{"type":"B","value":2},{"type":"A","value":3}]' | jq 'group_by(.type)'
Security Use Cases
# Parse Zeek logs (JSON format)
cat conn.log | jq 'select(.id.resp_p == 22) | {src: .id.orig_h, dst: .id.resp_h}'
# Extract IPs from API response
curl -s https://api.shodan.io/shodan/host/8.8.8.8 | jq '.data[].ip_str'
# Filter failed login attempts from JSON logs
cat auth.log.json | jq 'select(.event_type == "authentication" and .result == "failed")'
# Count events by type
cat security.log.json | jq 'group_by(.event_type) | map({event: .[0].event_type, count: length})'
# Extract all URLs from threat intel feed
cat iocs.json | jq '.indicators[] | select(.type == "url") | .value'
# Parse AWS CloudTrail logs
cat cloudtrail.json | jq '.Records[] | select(.eventName == "ConsoleLogin" and .errorCode != null)'
# VirusTotal results - get malicious detections
cat vt_result.json | jq '.data.attributes.last_analysis_stats | {malicious, suspicious}'
# Parse Suricata EVE JSON
cat eve.json | jq 'select(.event_type == "alert") | {timestamp, alert: .alert.signature, src_ip, dest_ip}'
# Elasticsearch query results
cat es_response.json | jq '.hits.hits[]._source | {timestamp, user, action}'
# Convert to CSV (with proper escaping)
cat data.json | jq -r '.[] | [.timestamp, .user, .action] | @csv'
Advanced Filters
# Conditional selection
cat logs.json | jq 'select(.level == "ERROR" or .level == "CRITICAL")'
# Check if field exists
cat data.json | jq 'select(.email != null)'
# Regex matching
cat logs.json | jq 'select(.message | test("failed|error"; "i"))'
# Contains
cat data.json | jq 'select(.tags | contains(["malware"]))'
# Arithmetic
cat data.json | jq '.[] | select(.score > 75 and .score < 100)'
# String concatenation
cat data.json | jq '.full_name = .first_name + " " + .last_name'
# Date parsing (requires date functions in jq 1.6+)
cat logs.json | jq 'select((.timestamp | fromdateiso8601) > now - 3600)'
jq -c for compact output (one JSON object per line), useful for piping to other tools.
π» Bash Scripting for SOC
Overview
Bash scripting is essential for automating repetitive SOC tasks, log analysis, threat hunting, and incident response. Quick scripts can save hours of manual work.
Basic Syntax
#!/bin/bash
# Variables
name="SOC Analyst"
count=10
# Print
echo "Hello, $name"
echo "Count: $count"
# Command substitution
current_date=$(date +%Y-%m-%d)
hostname=$(hostname)
# User input
read -p "Enter IP address: " ip
# If statement
if [ "$count" -gt 5 ]; then
echo "Count is greater than 5"
elif [ "$count" -eq 5 ]; then
echo "Count is exactly 5"
else
echo "Count is less than 5"
fi
# For loop
for i in {1..5}; do
echo "Iteration $i"
done
# While loop
counter=0
while [ $counter -lt 5 ]; do
echo "Counter: $counter"
((counter++))
done
# Functions
function check_port() {
local host=$1
local port=$2
nc -zv $host $port &>/dev/null
if [ $? -eq 0 ]; then
echo "Port $port is open on $host"
else
echo "Port $port is closed on $host"
fi
}
check_port 192.168.1.1 22
Log Analysis Scripts
#!/bin/bash
# Analyze Apache access logs for suspicious activity
LOG_FILE="/var/log/apache2/access.log"
echo "=== Top 10 IPs by request count ==="
awk '{print $1}' $LOG_FILE | sort | uniq -c | sort -rn | head -10
echo -e "\n=== Top 10 requested URLs ==="
awk '{print $7}' $LOG_FILE | sort | uniq -c | sort -rn | head -10
echo -e "\n=== Failed requests (4xx, 5xx) ==="
awk '$9 ~ /^[45]/ {print $9, $7}' $LOG_FILE | sort | uniq -c | sort -rn
echo -e "\n=== Potential SQL injection attempts ==="
grep -i "union.*select\|' or '1'='1\|concat.*0x" $LOG_FILE
echo -e "\n=== Potential XSS attempts ==="
grep -i "
Automated IOC Checker
#!/bin/bash
# Check if IPs from log file are malicious using AbuseIPDB
API_KEY="your_abuseipdb_api_key"
LOG_FILE="$1"
if [ -z "$LOG_FILE" ]; then
echo "Usage: $0 "
exit 1
fi
# Extract unique IPs
echo "Extracting IPs from $LOG_FILE..."
grep -oE '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b' "$LOG_FILE" | sort -u > /tmp/ips.txt
# Check each IP
while IFS= read -r ip; do
echo -n "Checking $ip... "
response=$(curl -s -G https://api.abuseipdb.com/api/v2/check \
--data-urlencode "ipAddress=$ip" \
-d maxAgeInDays=90 \
-H "Key: $API_KEY" \
-H "Accept: application/json")
score=$(echo "$response" | jq -r '.data.abuseConfidenceScore')
country=$(echo "$response" | jq -r '.data.countryCode')
if [ "$score" -gt 75 ]; then
echo "β οΈ MALICIOUS (Score: $score%, Country: $country)"
echo "$ip,$score,$country" >> malicious_ips.csv
elif [ "$score" -gt 25 ]; then
echo "β‘ SUSPICIOUS (Score: $score%, Country: $country)"
else
echo "β
Clean (Score: $score%, Country: $country)"
fi
sleep 1 # Rate limiting
done < /tmp/ips.txt
echo -e "\nResults saved to malicious_ips.csv"
Daily Security Report
#!/bin/bash
# Generate daily security report
REPORT_FILE="security_report_$(date +%Y%m%d).txt"
DATE=$(date +"%Y-%m-%d %H:%M:%S")
{
echo "=========================================="
echo " DAILY SECURITY REPORT"
echo " Generated: $DATE"
echo "=========================================="
echo ""
echo "=== Failed SSH Login Attempts ==="
grep "Failed password" /var/log/auth.log | tail -20
echo ""
echo "=== Successful SSH Logins ==="
grep "Accepted password" /var/log/auth.log | tail -10
echo ""
echo "=== New Users Created ==="
grep "new user" /var/log/auth.log | tail -10
echo ""
echo "=== Sudo Commands ==="
grep "sudo" /var/log/auth.log | tail -20
echo ""
echo "=== Listening Ports ==="
ss -tuln
echo ""
echo "=== Running Processes (Top 10 by CPU) ==="
ps aux --sort=-%cpu | head -11
echo ""
echo "=== Disk Usage ==="
df -h
echo ""
echo "=== Recent Package Updates ==="
grep " install \| upgrade " /var/log/dpkg.log | tail -20
echo ""
echo "=== Firewall Rules ==="
iptables -L -n -v
echo ""
echo "=========================================="
echo " END OF REPORT"
echo "=========================================="
} > "$REPORT_FILE"
echo "Report generated: $REPORT_FILE"
# Email report (requires mailutils)
# mail -s "Daily Security Report - $(date +%Y-%m-%d)" admin@example.com < "$REPORT_FILE"
Useful One-Liners
# Find files modified in last 24 hours
find /home -type f -mtime -1
# Find SUID binaries (potential privilege escalation)
find / -perm -4000 -type f 2>/dev/null
# Find large files (>100MB)
find / -type f -size +100M 2>/dev/null
# Monitor log file in real-time with highlights
tail -f /var/log/syslog | grep --color=auto -E "error|fail|critical"
# Extract and count unique IPs from logs
grep -oE '\b([0-9]{1,3}\.){3}[0-9]{1,3}\b' access.log | sort | uniq -c | sort -rn
# Find processes listening on network
lsof -i -P -n | grep LISTEN
# Check if port is open
timeout 2 bash -c " 100 {print $1, $2}' data.csv
# Monitor failed SSH logins in real-time
tail -f /var/log/auth.log | grep --line-buffered "Failed password"
# Find files owned by specific user
find / -user suspicious_user -ls 2>/dev/null
# Check for listening ports changes
diff <(lsof -i -P -n | grep LISTEN | sort) <(sleep 60; lsof -i -P -n | grep LISTEN | sort)
crontab -e to add scheduled tasks.
π‘οΈ Suricata - IDS/IPS/NSM Engine
Overview
Suricata is a high-performance Network IDS, IPS, and Network Security Monitoring engine. It inspects network traffic using powerful and extensive rules and signature language.
Installation
# Ubuntu/Debian
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt update && sudo apt install suricata -y
# RHEL/CentOS
sudo yum install epel-release
sudo yum install suricata
# Update rules (ET Open)
sudo suricata-update
sudo suricata-update list-sources
sudo suricata-update enable-source et/pro # If you have Pro account
# Start service
sudo systemctl enable suricata
sudo systemctl start suricata
# Check status
sudo systemctl status suricata
Configuration
# Edit /etc/suricata/suricata.yaml
# Set network interface
af-packet:
- interface: eth0
cluster-id: 99
cluster-type: cluster_flow
# Set HOME_NET
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
EXTERNAL_NET: "!$HOME_NET"
# Logging
outputs:
- fast:
enabled: yes
filename: fast.log
- eve-log:
enabled: yes
filetype: regular
filename: eve.json
types:
- alert
- http
- dns
- tls
- files
- ssh
Rule Syntax
# Basic rule structure
# action protocol source_ip source_port -> dest_ip dest_port (options)
# Alert on SSH from external
alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg:"SSH Connection from External"; sid:1000001; rev:1;)
# Detect SQL injection in HTTP
alert http any any -> any any (msg:"Possible SQL Injection"; content:"union"; nocase; content:"select"; nocase; distance:0; sid:1000002;)
# Detect PowerShell download cradle
alert http any any -> $HOME_NET any (msg:"PowerShell Download Cradle"; content:"powershell"; content:"DownloadFile"; nocase; sid:1000003;)
# Detect port scanning
alert tcp any any -> $HOME_NET any (msg:"Possible Port Scan"; flags:S; threshold:type threshold, track by_src, count 20, seconds 60; sid:1000004;)
# DNS tunneling detection
alert dns any any -> any any (msg:"Possible DNS Tunneling"; dns_query; content:"."; pcre:"/^.{50,}/"; sid:1000005;)
# TLS certificate validation
alert tls any any -> any any (msg:"Self-Signed Certificate Detected"; tls.cert_subject; content:"CN=localhost"; sid:1000006;)
# Detect Cobalt Strike beacon
alert tcp any any -> any any (msg:"Cobalt Strike Beacon"; content:"|00 00 00|"; offset:0; depth:3; content:"arch"; distance:0; sid:1000007;)
# File hash matching
alert http any any -> any any (msg:"Known Malware Download"; filestore; filemd5:known_malware_hashes.txt; sid:1000008;)
Monitoring & Analysis
# View fast.log (alerts only)
sudo tail -f /var/log/suricata/fast.log
# Parse EVE JSON logs
sudo tail -f /var/log/suricata/eve.json | jq 'select(.event_type=="alert")'
# Count alerts by signature
sudo cat /var/log/suricata/eve.json | jq -s 'group_by(.alert.signature) | map({sig: .[0].alert.signature, count: length}) | sort_by(.count) | reverse'
# Top source IPs
sudo cat /var/log/suricata/eve.json | jq -s 'group_by(.src_ip) | map({ip: .[0].src_ip, count: length}) | sort_by(.count) | reverse | .[0:10]'
# Extract HTTP traffic
sudo cat /var/log/suricata/eve.json | jq 'select(.event_type=="http") | {timestamp, src_ip, http}'
# DNS queries
sudo cat /var/log/suricata/eve.json | jq 'select(.event_type=="dns") | {timestamp, src_ip, dns}'
# Stats
sudo suricatasc -c "dump-counters"
# Reload rules (no restart)
sudo kill -USR2 $(pidof suricata)
π NetworkMiner - Network Forensics
Overview
NetworkMiner is a Network Forensic Analysis Tool (NFAT) for Windows. It extracts artifacts from PCAP files including files, images, emails, credentials, and more.
Installation
# Download from
# https://www.netresec.com/?page=NetworkMiner
# Extract ZIP
# Run NetworkMiner.exe (no installation required)
# Requires .NET Framework 4.6+
# WinPcap or Npcap for live capture
Key Features
- File Extraction: Automatically extract files transferred over HTTP, FTP, SMB, TFTP
- Image Detection: Extract and display images from network traffic
- Credential Recovery: Extract credentials from protocols (HTTP, FTP, IMAP, SMTP, POP3)
- Host Discovery: Identify hosts with OS fingerprinting
- Session Analysis: Reconstruct TCP sessions
- DNS Analysis: View DNS queries and responses
- Keyword Search: Search for specific strings in traffic
- Geolocation: Map source IPs geographically
Usage Workflow
1. Load PCAP File
File β Open β Select .pcap/.pcapng
2. Automatic Analysis
- Hosts tab: Lists all discovered hosts with OS info
- Files tab: Extracted files (executables, documents, images)
- Images tab: Visual display of extracted images
- Messages tab: Email messages
- Credentials tab: Usernames/passwords found
- Sessions tab: TCP/UDP sessions
- DNS tab: DNS queries/responses
- Parameters tab: HTTP parameters, cookies
- Keywords tab: Search for specific strings
3. Extract Artifacts
- Right-click file β Open folder
- Check AssembledFiles/ directory
4. Analyze Suspicious Activity
- Look for executable downloads
- Check for cleartext credentials
- Identify data exfiltration (large outbound transfers)
- Review DNS for malicious domains
5. Generate Report
- File β Generate Report β HTML/Text
π Ghidra - Reverse Engineering Tool
Overview
Ghidra is a free and open-source reverse engineering tool developed by the NSA. It provides disassembly, decompilation, scripting, and analysis capabilities for binary analysis and malware research.
Installation
# Install Java 11+ (required)
sudo apt install openjdk-17-jdk -y
# Download Ghidra from
# https://github.com/NationalSecurityAgency/ghidra/releases
# Extract
unzip ghidra_10.4_PUBLIC_*.zip
cd ghidra_10.4_PUBLIC
# Run
./ghidraRun
Installation
# Install Java 11+
# Download from https://adoptium.net/
# Download Ghidra
# https://github.com/NationalSecurityAgency/ghidra/releases
# Extract ZIP
# Run ghidraRun.bat
Installation
# Install Java
brew install openjdk@17
# Download Ghidra
# https://github.com/NationalSecurityAgency/ghidra/releases
# Extract and run
unzip ghidra_10.4_PUBLIC_*.zip
cd ghidra_10.4_PUBLIC
./ghidraRun
Basic Workflow
1. Create New Project
File β New Project β Non-Shared Project
Name: malware_analysis_2024
2. Import Binary
File β Import File β Select malware.exe
Ghidra auto-detects format (PE, ELF, Mach-O)
3. Analyze Binary
Analysis β Auto Analyze (default options)
Wait for analysis to complete
4. Key Windows
- Listing: Disassembled code
- Decompiler: C-like pseudocode
- Symbol Tree: Functions, imports, exports
- Data Type Manager: Structures, types
- Function Graph: Visual control flow
5. Navigation
- Press 'G' to go to address
- Double-click to follow jumps
- Right-click β References β Show References to
- Search β For Strings (Ctrl+Shift+S)
- Search β Memory (Alt+M)
6. Analysis Tips
- Look for interesting strings (URLs, IPs, file paths)
- Check imports (what APIs malware uses)
- Find main() or WinMain()
- Analyze suspicious functions
- Look for anti-analysis tricks (IsDebuggerPresent)
- Check for network functions (WSAStartup, socket, connect)
Useful Keyboard Shortcuts
| Shortcut | Action |
|---|---|
| L | Rename label/function |
| ; | Add comment |
| Ctrl+L | Go to address |
| Ctrl+Shift+S | Search for strings |
| Ctrl+Shift+E | Show references to |
| Alt+Left/Right | Navigate back/forward |
| P | Edit function signature |
| T | Change data type |
π― capa - Malware Capability Detector
Overview
capa detects capabilities in executable files by matching them against a library of behavioral rules. It identifies what a program can do: persistence, data theft, anti-analysis, etc.
Installation
# Download standalone binary
wget https://github.com/mandiant/capa/releases/download/v6.1.0/capa-v6.1.0-linux.zip
unzip capa-v6.1.0-linux.zip
chmod +x capa
sudo mv capa /usr/local/bin/
# Or install via pip
pip install flare-capa
# Verify
capa --version
Installation
# Download from
# https://github.com/mandiant/capa/releases
# Extract capa.exe
# Run from PowerShell
.\capa.exe --help
Installation
# Install via pip
pip install flare-capa
# Or download binary
wget https://github.com/mandiant/capa/releases/download/v6.1.0/capa-v6.1.0-macos.zip
unzip capa-v6.1.0-macos.zip
chmod +x capa
Basic Usage
# Analyze malware sample
capa malware.exe
# Verbose output (show all matches)
capa -v malware.exe
# Very verbose (show addresses)
capa -vv malware.exe
# JSON output
capa -j malware.exe > results.json
# Analyze specific function
capa malware.exe --function 0x401000
# Show only high-level capabilities
capa malware.exe | grep -A 5 "CAPABILITY"
Sample Output Interpretation
+------------------------+---------------------------------------------------+
| CAPABILITY | NAMESPACE |
+------------------------+---------------------------------------------------+
| check for sandbox | anti-analysis/anti-sandbox |
| check for debugger | anti-analysis/anti-debugging/debugger-detection |
| encrypt data | data-manipulation/encryption |
| hash data | data-manipulation/hashing/sha256 |
| create file | host-interaction/file-system/create |
| delete file | host-interaction/file-system/delete |
| create registry key | host-interaction/registry/create |
| execute command | host-interaction/process/create |
| send HTTP request | communication/http/client |
| resolve DNS | communication/dns |
| create TCP socket | communication/socket/tcp |
| persist via registry | persistence/registry/run-key |
+------------------------+---------------------------------------------------+
This malware can:
β
Detect analysis environments (sandbox, debugger)
β
Encrypt and hash data
β
Create/delete files
β
Modify registry for persistence
β
Execute commands
β
Establish network connections
Integration with Other Tools
# Batch analyze directory
for file in /malware/samples/*; do
echo "Analyzing $file"
capa "$file" > "results/$(basename $file).txt"
done
# Extract capabilities to CSV
capa -j malware.exe | jq -r '.capabilities | keys[]' | sed 's/$/,malware.exe/' > capabilities.csv
# Check for specific capability
capa malware.exe | grep -i "keylogger" && echo "β οΈ Keylogger detected!"
# Compare capabilities of two samples
diff <(capa sample1.exe | grep CAPABILITY) <(capa sample2.exe | grep CAPABILITY)
π MISP - Threat Intelligence Platform
Overview
MISP (Malware Information Sharing Platform) is an open-source threat intelligence platform for sharing, storing, and correlating Indicators of Compromise (IOCs) and threat intelligence.
Installation (Docker - Quickest Method)
# Clone MISP Docker repository
git clone https://github.com/MISP/misp-docker.git
cd misp-docker
# Build and start
docker-compose up -d
# Access MISP
# https://localhost
# Default credentials: admin@admin.test / admin
# Change password after first login!
Key Features
- Event Management: Create and share threat events with IOCs
- Attribute Types: IP, domain, hash, email, URL, malware-sample, etc.
- Correlation: Auto-correlate IOCs across events
- Taxonomies: Tag events with TLP, PAP, kill-chain phases
- Feeds: Import from external threat feeds
- Sharing Groups: Control who sees what intel
- API: Full REST API for automation
- Modules: Expansion (enrichment), export, import modules
Using MISP API (PyMISP)
# Install PyMISP
pip install pymisp
from pymisp import PyMISP
# Initialize connection
misp_url = 'https://misp.example.com'
misp_key = 'your_api_key_here'
misp_verifycert = False
misp = PyMISP(misp_url, misp_key, misp_verifycert)
# Search for IOC
results = misp.search('attributes', value='192.168.1.100')
for event in results['Attribute']:
print(f"Found in Event: {event['event_id']}")
print(f"Type: {event['type']}")
print(f"Category: {event['category']}")
# Create new event
event = misp.new_event(info="Phishing Campaign 2024", threat_level_id=2, analysis=1)
event_id = event['Event']['id']
# Add attributes (IOCs)
misp.add_attribute(event_id, {'type': 'ip-dst', 'value': '192.168.1.100', 'comment': 'C2 server'})
misp.add_attribute(event_id, {'type': 'domain', 'value': 'malicious.com', 'comment': 'Phishing domain'})
misp.add_attribute(event_id, {'type': 'md5', 'value': 'abc123...', 'comment': 'Malware hash'})
# Add tags
misp.tag(event['Event']['uuid'], 'tlp:amber')
misp.tag(event['Event']['uuid'], 'misp-galaxy:threat-actor="APT29"')
# Publish event
misp.publish(event_id)
# Batch check IPs against MISP
ips = ['1.2.3.4', '5.6.7.8', '9.10.11.12']
for ip in ips:
result = misp.search('attributes', value=ip, type_attribute='ip-dst')
if result['Attribute']:
print(f"β οΈ {ip} found in MISP threat intel!")
else:
print(f"β
{ip} not in MISP")
Common Use Cases
# Incident Response Workflow
1. Create event for incident (e.g., "Ransomware Attack - Jan 2024")
2. Add observed IOCs (IPs, domains, hashes, emails)
3. Tag with TLP level (tlp:red for sensitive)
4. Tag with threat actor (if known)
5. Add relationships between attributes
6. Share with trusted communities/ISACs
7. Subscribe to feeds for automated intel updates
# SOC Integration
- Export IOCs to SIEM (Splunk, ELK) for alerting
- Use MISP API to enrich alerts with context
- Automate firewall/IDS rule updates from MISP
- Correlate internal incidents with global threats
# Threat Hunting
- Search MISP for TTPs seen in environment
- Check if your observed IOCs match known campaigns
- Pivot on correlated attributes
- Download STIX/OpenIOC exports for analysis tools
π‘οΈ Wazuh - Security Monitoring Platform
Overview
Wazuh is a free, open-source security monitoring platform that provides threat detection, integrity monitoring, incident response, and compliance capabilities. Think of it as an open-source SIEM + EDR.
Installation (All-in-One)
# Download installer
curl -sO https://packages.wazuh.com/4.7/wazuh-install.sh
# Install (requires 4GB RAM minimum)
sudo bash wazuh-install.sh -a
# Installation provides:
# - Wazuh Manager
# - Wazuh Indexer (Elasticsearch)
# - Wazuh Dashboard (Kibana fork)
# - Credentials printed at end
# Access Dashboard
# https://your-server-ip
# Username: admin
# Password:
Agent Installation
# Linux Agent
wget https://packages.wazuh.com/4.x/apt/pool/main/w/wazuh-agent/wazuh-agent_4.7.0-1_amd64.deb
sudo WAZUH_MANAGER='manager-ip' dpkg -i wazuh-agent_4.7.0-1_amd64.deb
sudo systemctl start wazuh-agent
# Windows Agent (PowerShell as Admin)
Invoke-WebRequest -Uri https://packages.wazuh.com/4.x/windows/wazuh-agent-4.7.0-1.msi -OutFile wazuh-agent.msi
.\wazuh-agent.msi /q WAZUH_MANAGER='manager-ip'
NET START WazuhSvc
# macOS Agent
curl -so wazuh-agent.pkg https://packages.wazuh.com/4.x/macos/wazuh-agent-4.7.0-1.pkg
sudo installer -pkg wazuh-agent.pkg -target /
sudo /Library/Ossec/bin/agent-auth -m manager-ip
sudo /Library/Ossec/bin/wazuh-control start
Key Detection Capabilities
| Capability | Description |
|---|---|
| File Integrity Monitoring | Detect changes to critical files (syscheck) |
| Log Analysis | Parse and alert on logs (auth.log, Windows Event Log) |
| Rootkit Detection | Detect kernel-level rootkits |
| Vulnerability Detection | Scan for outdated software with known CVEs |
| Compliance | PCI DSS, HIPAA, GDPR, NIST 800-53 checks |
| Active Response | Automated actions (block IP, kill process) |
| Cloud Security | AWS, Azure, GCP monitoring |
Common Rules
5716
Multiple failed SSH login attempts
T1110
60000
-enc|-encodedcommand
Encoded PowerShell command detected
T1059.001
550
/etc/
File modified in /etc/ directory
Useful Queries (Wazuh Dashboard)
# Failed authentication attempts
rule.description:"authentication failed" OR rule.description:"login failed"
# High severity alerts
rule.level>=10
# Specific agent
agent.name:"web-server-01"
# MITRE ATT&CK technique
rule.mitre.id:"T1110"
# FIM alerts (file changes)
rule.groups:"syscheck"
# Vulnerability detection
rule.groups:"vulnerability-detector"
# Windows Event ID 4625 (Failed login)
data.win.system.eventID:"4625"
# Process execution
data.win.eventdata.image:*powershell.exe*
π Recon-ng - Reconnaissance Framework
Overview
Recon-ng is a full-featured reconnaissance framework written in Python. It provides a powerful environment to conduct open-source intelligence (OSINT) gathering quickly and thoroughly.
Installation
# Ubuntu/Debian
sudo apt install recon-ng -y
# Or install via pip
pip install recon-ng
# From source
git clone https://github.com/lanmaster53/recon-ng.git
cd recon-ng
pip install -r REQUIREMENTS
./recon-ng
Installation
# Install via pip
pip install recon-ng
# Or from source
git clone https://github.com/lanmaster53/recon-ng.git
cd recon-ng
pip install -r REQUIREMENTS
./recon-ng
Basic Commands
# Start Recon-ng
recon-ng
# Create workspace
workspaces create example_corp
# List available modules
marketplace search
# Install module
marketplace install recon/domains-hosts/hackertarget
marketplace install recon/hosts-hosts/resolve
marketplace install recon/domains-contacts/whois_pocs
# Load module
modules load recon/domains-hosts/hackertarget
# Show module info
info
# Set options
options set SOURCE example.com
# Run module
run
# View results
show hosts
show contacts
show domains
# Export results
show hosts | grep example.com > hosts.txt
Common Reconnaissance Workflow
# 1. Domain Enumeration
modules load recon/domains-hosts/hackertarget
options set SOURCE example.com
run
# 2. Subdomain Discovery
modules load recon/domains-hosts/threatcrowd
run
modules load recon/domains-hosts/certificate_transparency
run
# 3. Reverse DNS
modules load recon/hosts-hosts/resolve
run
# 4. WHOIS Lookup
modules load recon/domains-contacts/whois_pocs
run
# 5. Email Harvesting
modules load recon/domains-contacts/hunter_io
options set API_KEY your_hunter_io_key
run
# 6. Port Scanning (if authorized)
modules load recon/hosts-ports/shodan_hostname
options set API_KEY your_shodan_key
run
# 7. Generate Report
modules load reporting/html
options set CREATOR "SOC Team"
options set CUSTOMER "Example Corp"
run
Popular Modules
| Module | Description |
|---|---|
recon/domains-hosts/hackertarget |
Find subdomains via HackerTarget |
recon/domains-hosts/threatcrowd |
Subdomain enumeration via ThreatCrowd |
recon/domains-contacts/hunter_io |
Email harvesting via Hunter.io |
recon/hosts-hosts/resolve |
DNS resolution |
recon/domains-contacts/whois_pocs |
WHOIS contact extraction |
recon/hosts-ports/shodan_hostname |
Port info via Shodan |
reporting/html |
Generate HTML report |
keys add module_name api_key
π¬ PEStudio - Windows Executable Analysis
Overview
PEStudio is a free tool for static analysis of Windows executables (PE files). It quickly identifies suspicious indicators, imports, resources, and provides a malware score without executing the file.
Key Features
- Malware Score: 0-100 score based on suspicious indicators
- VirusTotal Integration: Check file hash reputation
- Imports Analysis: Identify suspicious API calls
- Strings Extraction: Find URLs, IPs, file paths
- Resources: Extract embedded files, icons, images
- Certificate Validation: Check digital signatures
- YARA Rules: Built-in YARA scanning
- Indicators: Highlight packing, anti-debugging, obfuscation
Suspicious Indicators to Check
| Section | Red Flags |
|---|---|
| Imports | VirtualAllocEx, WriteProcessMemory, CreateRemoteThread (injection) |
| Imports | IsDebuggerPresent, CheckRemoteDebuggerPresent (anti-debug) |
| Sections | High entropy (.text > 7.0 = likely packed) |
| Strings | URLs to suspicious domains, IP addresses, "cmd.exe" |
| Certificate | No signature, expired, or self-signed |
| Resources | Embedded executables, large data blobs |
| Metadata | Compilation date in future or very old |
Typical Analysis Workflow
1. Open File
Drag & drop suspicious.exe into PEStudio
2. Review Summary Tab
- Check malware score (>70 = high suspicion)
- Note VirusTotal detection ratio
- Review digital signature status
3. Analyze Imports
- Look for process injection APIs
- Check for anti-analysis functions
- Note networking functions (WinSock)
- Registry manipulation (RegSetValue)
4. Check Strings
- Sort by blacklist (shows known bad strings)
- Look for URLs, IPs, file paths
- Search for "cmd", "powershell", "http"
5. Examine Sections
- Check entropy (high = packed/encrypted)
- Look for unusual section names (.upx, .aspack)
- Verify section characteristics
6. Resources Tab
- Extract embedded files
- Check for additional executables
- Review version info discrepancies
7. Generate Report
File β Export β HTML Report
π csvkit - CSV Data Processing
Overview
csvkit is a suite of command-line tools for working with CSV files. Essential for analyzing security logs, threat intelligence feeds, and incident reports in CSV format.
Installation
# Install via pip
pip install csvkit
# Verify
csvstat --version
Installation
# Install via pip
pip install csvkit
Core Tools
| Tool | Purpose |
|---|---|
csvstat |
Generate statistics about CSV data |
csvcut |
Select specific columns |
csvgrep |
Filter rows based on patterns |
csvsort |
Sort rows |
csvjoin |
Join two CSV files |
csvlook |
Pretty-print CSV in terminal |
csvsql |
Query CSV with SQL |
in2csv |
Convert Excel/JSON to CSV |
Common Usage
# View column headers
csvcut -n data.csv
# Select specific columns
csvcut -c "timestamp,src_ip,dst_ip,action" firewall.csv
# Filter rows (grep-like)
csvgrep -c status -m "blocked" firewall.csv
# Filter by regex
csvgrep -c url -r "\.exe$" downloads.csv
# Sort by column
csvsort -c timestamp -r alerts.csv # -r for reverse
# Show statistics
csvstat alerts.csv
# Pretty print (first 10 rows)
head -10 data.csv | csvlook
# Count unique values
csvcut -c src_ip firewall.csv | tail -n +2 | sort | uniq -c | sort -rn
# SQL queries on CSV
csvsql --query "SELECT src_ip, COUNT(*) as count FROM data WHERE action='block' GROUP BY src_ip ORDER BY count DESC LIMIT 10" firewall.csv
# Join two CSVs
csvjoin -c ip_address iocs.csv firewall.csv
# Convert Excel to CSV
in2csv data.xlsx > data.csv
# Convert JSON to CSV
in2csv data.json > data.csv
Security Use Cases
# Analyze firewall logs - top blocked IPs
csvgrep -c action -m "BLOCK" firewall.csv | \
csvcut -c src_ip | tail -n +2 | sort | uniq -c | sort -rn | head -20
# Find high-severity alerts
csvgrep -c severity -m "HIGH" alerts.csv | csvlook
# Threat intel - check if IPs in logs match IOC list
csvjoin -c ip_address threat_intel.csv firewall_logs.csv | \
csvgrep -c threat_score -r "[8-9][0-9]|100" # Score 80-100
# Time-based filtering
csvsql --query "SELECT * FROM logs WHERE timestamp > '2024-01-01' AND severity='CRITICAL'" logs.csv
# Export to JSON
csvjson alerts.csv > alerts.json
# Generate summary report
csvstat -c src_ip,dst_ip,bytes_sent connections.csv > network_summary.txt
# Deduplicate rows
csvsort -c timestamp data.csv | uniq > deduped.csv
# Calculate bandwidth usage
csvsql --query "SELECT dst_ip, SUM(bytes) as total_bytes FROM traffic GROUP BY dst_ip ORDER BY total_bytes DESC" traffic.csv
csvcut | csvgrep | csvsort | csvlook