Cyber Services / All lessons / Cyber Threat Intelligence / Tooling — Utilities
Tooling — Utilities
Module 13 of 13 · 10 min read · Free to read
Beyond the major platforms, effective CTI analysts rely on specialized tools for specific tasks. This lab covers YARA for malware detection, Shodan for internet intelligence, ThreatConnect for intelligence management, Sigma for detection rules, and other essential utilities that enhance threat intelligence operations.
Tools Overview
Lab 1: YARA Rule Development
Mastering Malware Detection with YARA
This lab covers YARA rule creation, testing, and deployment for effective malware detection and threat hunting.
Objectives
Step-by-Step Guide
Verification
1 Learning Objectives
- • Understand YARA rule syntax and structure
- • Create effective malware detection rules
- • Test and validate YARA rules
- • Integrate YARA with security tools
2 Prerequisites
- • YARA installed (Windows/Linux/macOS)
- • Sample malware files (for testing)
- • Text editor or IDE
3 Estimated Time
30-45 minutes
1 Installation & Setup
Install YARA on your system:
# Windows
# Download from: https://github.com/VirusTotal/yara/releases
# Extract and add to PATH
# Linux (Ubuntu/Debian)
sudo apt-get install yara
# Linux (Compile from source)
wget https://github.com/VirusTotal/yara/archive/refs/tags/v4.3.0.tar.gz
tar -xzf v4.3.0.tar.gz
cd yara-4.3.0
./bootstrap.sh
./configure
make
sudo make install
# macOS
brew install yara
# Verify installation
yara --version
2 Basic YARA Rule Structure
Create your first YARA rule:
rule Basic_Malware_Detection {
meta:
description = "Detects basic malware patterns"
author = "CTI Analyst"
date = "2026-01-15"
threat_level = 5
strings:
$malicious_string1 = "malicious payload"
$malicious_string2 = { 6A 40 68 00 30 00 00 6A 14 }
$suspicious_url = "http://evil-domain.com" nocase
$hex_pattern = /get.*password/i
condition:
any of them
}
3 Advanced Rule Development
- • Create rule for specific malware family:
rule Emotet_Malware {
meta:
description = "Detects Emotet malware samples"
author = "CTI Team"
reference = "https://attack.mitre.org/software/S0367/"
tlp = "AMBER"
strings:
$mz_header = "MZ"
$pdb_string = "emotet" nocase wide ascii
$c2_url = /https?:\/\/[a-z0-9]+\.(xyz|top|club)\// nocase
$ip_pattern = /\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b/
condition:
$mz_header at 0 and
all of ($pdb_string, $c2_url) or
($mz_header at 0 and 2 of ($*))
}
- • Create TTP-based detection rule:
rule PowerShell_Execution {
meta:
description = "Detects suspicious PowerShell execution"
author = "Threat Hunting Team"
mitre_attack = "T1059.001"
strings:
$ps1 = "powershell.exe" nocase
$bypass = /-ExecutionPolicy.*bypass/i
$encoded = /-EncodedCommand/i
$download = /DownloadString|DownloadFile/i
condition:
$ps1 and
(1 of ($bypass, $encoded, $download))
}
4 Testing & Validation
Test your YARA rules:
# Test rule against single file
yara my_rule.yar suspicious_file.exe
# Test against directory
yara -r my_rule.yar /path/to/files/
# Test with multiple rules
yara rules.yar malware_sample.exe
# Verbose output with rule metadata
yara -s -m my_rule.yar test_file.exe
# Create test harness in Python
import yara
# Compile rule
rules = yara.compile(filepath='my_rule.yar')
# Test file
matches = rules.match('test_file.exe')
for match in matches:
print(f"Rule: {match.rule}")
print(f"Tags: {match.tags}")
print(f"Meta: {match.meta}")
5 Integration & Automation
- • Integrate with VirusTotal API:
import vt
import yara
client = vt.Client("API_KEY")
# Scan file with YARA
rules = yara.compile('malware_rules.yar')
matches = rules.match('suspicious.exe')
if matches:
# Upload to VirusTotal
with open('suspicious.exe', 'rb') as f:
analysis = client.scan_file(f)
print(f"Submitted to VT: {analysis.id}")
- • Automate YARA scanning with cron/Windows Task Scheduler
- • Integrate with SIEM for automated alerts
1 Lab Completion Checklist
Verify you have completed all required steps:
- ✓ Successfully installed YARA
- ✓ Created at least 3 different YARA rules
- ✓ Tested rules against sample files
- ✓ Created a TTP-based detection rule
- ✓ Integrated YARA with Python script
2 Knowledge Check
Answer these questions to confirm understanding:
- 1. What's the difference between text strings and hex strings in YARA?
- 2. How can you avoid false positives in YARA rules?
- 3. What are some best practices for YARA rule organization?
- 4. How can YARA rules support threat hunting?
Advanced YARA Examples
These examples demonstrate advanced YARA capabilities for CTI operations:
rule APT29_Backdoor {
meta:
threat_actor = "APT29"
malware_family = "Cobalt Strike"
mitre_attack = "T1059, T1105, T1071"
strings:
$beacon_config = { 00 00 00 [4] 00 00 [4] }
$c2_pattern = /https?:\/\/(?:[a-z0-9-]+\.)+[a-z]{2,}\/[:alnum:]+\//
$sleep_time = /sleep.*[0-9]{4,}/
condition:
uint16(0) == 0x5A4D and
filesize < 2MB and
2 of ($*)
}
rule Ransomware_Conti {
meta:
description = "Detects Conti ransomware"
reference = "https://malpedia.caad.fkie.fraunhofer.de/details/win.conti"
strings:
$rsa_key = "-----BEGIN RSA PUBLIC KEY-----"
$extension = ".conti" nocase
$ransom_note = "Your files have been encrypted!"
$mutex = "ContiMutex"
condition:
$rsa_key and
($extension or $ransom_note) and
$mutex
}
rule Malicious_Document {
meta:
description = "Detects malicious Office documents"
tlp = "GREEN"
strings:
$ole_header = { D0 CF 11 E0 A1 B1 1A E1 }
$vba_macro = "Sub AutoOpen()" nocase
$shellcode = /call.*eax|jmp.*esp|pop.*pop.*ret/
$suspicious_url = /(http|ftp|https):\/\/([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:\/~+#-]*[\w@?^=%&\/~+#-])?/
condition:
$ole_header at 0 and
($vba_macro or $shellcode) and
$suspicious_url
}
rule C2_Communication {
meta:
description = "Detects C2 communication patterns"
mitre_attack = "T1071"
strings:
$get_request = "GET /"
$post_request = "POST /"
$user_agent = "User-Agent:"
$beacon = /[0-9A-F]{8,}\/[a-z0-9]{16}/
condition:
($get_request or $post_request) and
$user_agent and
$beacon
}
Lab 2: Shodan for Internet Intelligence
Leveraging Shodan for Threat Intelligence
This lab explores Shodan for discovering exposed devices, vulnerabilities, and threat actor infrastructure.
Objectives
Step-by-Step Guide
Verification
1 Learning Objectives
- • Understand Shodan search syntax and filters
- • Discover exposed services and vulnerabilities
- • Map threat actor infrastructure
- • Use Shodan API for automation
2 Prerequisites
- • Shodan account (free tier available)
- • Basic understanding of networking
- • Python for API automation
3 Estimated Time
30-45 minutes
1 Account Setup & Basic Search
2 Advanced Search Techniques
Combine filters for targeted searches:
# Find vulnerable services
apache 2.4.49 country:US
http.title:"Dashboard" http.component:"django"
# Find exposed databases
port:3306 MySQL
port:5432 PostgreSQL
port:27017 MongoDB
"product:Redis"
# Find industrial control systems
port:502
"Modbus"
"SCADA"
"PLC"
# Find vulnerable webcams
"webcam 7" "200 OK"
"Server: SQ-WEBCAM"
"netcam" "200 OK"
# Find specific vulnerabilities
vuln:CVE-2021-44228
vuln:CVE-2021-41773
"heartbleed" vuln:CVE-2014-0160
3 Threat Intelligence Applications
- • Map threat actor infrastructure:
# Search for known C2 servers
"Metasploit" port:4444
"Cobalt Strike" port:50050
"Empire" port:1337
# Find phishing infrastructure
http.title:"Login" "phpmyadmin"
"Roundcube" port:80
"OWA" "Exchange"
# Discover botnet nodes
"Mirai" port:23
"IoT" default password
"telnet" "root:root"
- • Monitor your organization's exposure:
# Your organization's assets
net:192.0.2.0/24
org:"Your Company Name"
hostname:*.yourcompany.com
# Check for data leaks
"yourcompany.com" "password"
"yourcompany.com" "api_key"
"yourcompany.com" "secret"
4 Shodan API Automation
Automate Shodan searches with Python:
import shodan
import json
# Initialize API
API_KEY = "YOUR_API_KEY"
api = shodan.Shodan(API_KEY)
def search_exposed_services(query):
"""Search for exposed services"""
try:
results = api.search(query)
print(f"Results found: {results['total']}")
for result in results['matches']:
print(f"IP: {result['ip_str']}")
print(f"Port: {result['port']}")
print(f"Organization: {result.get('org', 'N/A')}")
print(f"Data: {result['data'][:100]}...")
print("-" * 50)
return results
except shodan.APIError as e:
print(f"Error: {e}")
return None
def get_host_info(ip):
"""Get detailed information about a host"""
try:
host = api.host(ip)
print(f"IP: {host['ip_str']}")
print(f"Organization: {host.get('org', 'N/A')}")
print(f"Operating System: {host.get('os', 'N/A')}")
print("\nOpen Ports:")
for item in host['data']:
print(f" Port {item['port']}: {item.get('product', 'N/A')}")
return host
except shodan.APIError as e:
print(f"Error: {e}")
return None
def monitor_organization(org_name):
"""Monitor organization's exposed assets"""
query = f'org:"{org_name}"'
results = api.search(query)
exposed_services = {}
for result in results['matches']:
service = result.get('product', 'Unknown')
port = result['port']
key = f"{service}:{port}"
exposed_services[key] = exposed_services.get(key, 0) + 1
print(f"Exposed services for {org_name}:")
for service, count in sorted(exposed_services.items()):
print(f" {service}: {count}")
return exposed_services
# Example usage
if __name__ == "__main__":
# Search for exposed Apache servers
search_exposed_services("apache country:US")
# Get info about specific host
get_host_info("8.8.8.8")
# Monitor organization
monitor_organization("Example Corp")
1 Lab Completion Checklist
Verify you have completed all required steps:
- ✓ Created Shodan account and obtained API key
- ✓ Performed at least 5 different search types
- ✓ Identified vulnerable services in test searches
- ✓ Created Python script for Shodan automation
- ✓ Documented findings for threat intelligence
2 Knowledge Check
Answer these questions to confirm understanding:
- 1. What are the ethical considerations when using Shodan?
- 2. How can Shodan support attack surface management?
- 3. What filters are most useful for threat intelligence?
- 4. How can Shodan API results be integrated with other tools?
Tool Integration Examples
YARA + VirusTotal Integration
Automate malware analysis by combining YARA rules with VirusTotal API for comprehensive threat detection.
# Python script that scans files with YARA
# and submits matches to VirusTotal
import yara, vt, os
rules = yara.compile('rules.yar')
for file in os.listdir('samples/'):
matches = rules.match(f'samples/{file}')
if matches:
upload_to_virustotal(file)
Shodan + Threat Intelligence
Enrich threat intelligence by using Shodan to investigate infrastructure associated with known IOCs.
# Automatically check IPs from threat feeds
# against Shodan for additional context
for malicious_ip in threat_feed:
host_info = shodan.host(malicious_ip)
if 'vulns' in host_info:
alert_security_team(host_info)
ThreatConnect Workflows
Create automated playbooks in ThreatConnect to process, enrich, and distribute threat intelligence.
# ThreatConnect Playbook Example
1. Receive IOC from feed
2. Enrich with VirusTotal/AbuseIPDB
3. Score based on reputation
4. Create alert if score > threshold
5. Distribute to SIEM/SOC teams
Sigma + SIEM Integration
Convert Sigma rules to native SIEM queries (Splunk, Elastic, etc.) for consistent detection across platforms.
# Sigma rule conversion pipeline
sigma convert -t splunk rule.yml
sigma convert -t elastic rule.yml
# Output: Native queries ready for SIEM
Quick Reference: Other Essential Tools
Additional tools every CTI analyst should know:
Cuckoo Sandbox
Automated malware analysis
TheHive
Incident response platform
Cortex
Analyst automation platform
MISP
Threat intelligence sharing
GreyNoise
Internet background noise
AbuseIPDB
IP reputation database
AlienVault OTX
Open threat exchange
Sigma
Generic detection rules
Practical Assignments
Complete these assignments to apply your skills with additional CTI tools:
Assignment 1: Malware Campaign Detection
Research a recent malware campaign and create YARA rules to detect it, then use Shodan to find related infrastructure.
- • Analyze malware sample characteristics
- • Create comprehensive YARA rules
- • Search for C2 infrastructure in Shodan
- • Document findings in threat report
Assignment 2: Attack Surface Assessment
Conduct a Shodan-based assessment of a hypothetical organization to identify exposed services and vulnerabilities.
- • Map organization's digital footprint
- • Identify exposed services and ports
- • Check for known vulnerabilities
- • Create risk assessment report
Assignment 3: Threat Intelligence Pipeline
Build an automated pipeline that ingests threat feeds, enriches IOCs, and creates actionable intelligence.
- • Integrate multiple threat feeds
- • Enrich IOCs with external APIs
- • Score and prioritize threats
- • Export to security tools
Key Takeaways
YARA for Detection
YARA provides powerful pattern matching for malware detection, threat hunting, and IOC-based searching across files and memory.
Shodan for Intelligence
Shodan offers unprecedented visibility into internet-connected devices, enabling attack surface management and infrastructure mapping.
Platform Integration
Tools like ThreatConnect enable orchestration of intelligence workflows, combining multiple tools into cohesive operations.
Specialized Utilities
Effective CTI requires mastery of specialized tools for specific tasks, from malware analysis to infrastructure discovery.
Download Lab Materials
Get all lab instructions, sample rules, scripts, and tool configurations.
Download Additional Tools Lab Kit
Includes: YARA rule templates, Shodan search scripts, ThreatConnect playbooks, Sigma rules, integration examples