Cyber Services / All lessons / CEH — Ethical Hacking / Footprinting Web Infrastructure

Footprinting Web Infrastructure

Module 7 of 130 · 12 min read · Free to read

MODULES ↓ Command blocks have a COPY button
Comprehensive Guide to Web Server Reconnaissance Using Kali Linux Tools
Ethical Hacking Reconnaissance Kali Linux CEH v13
Lab Overview

This lab guides you through the essential steps of footprinting a web server and its associated infrastructure using common tools available in Kali Linux. Footprinting is a critical first step in the ethical hacking process, gathering information to understand the target's attack surface before conducting vulnerability assessment or exploitation.

Lab Objectives
  • Perform Server Discovery to identify live hosts
  • Conduct Service Discovery to find open ports and running services
  • Identify web server software, version, and underlying OS
  • Discover hidden content (directories and files) on web servers
  • Utilize tools like Nmap, Curl, WhatWeb, Dirb, Gobuster, and Nikto
  • Correlate findings for comprehensive security assessment
Ethical Use Mandate

This lab must be performed ONLY on systems you have explicit permission to test. For practice, use dedicated vulnerable virtual machines like Metasploitable 2/3 or OWASP Juice Shop within a controlled virtual network environment. Unauthorized scanning is illegal and unethical.

Lab Setup & Environment Preparation

Environment Preparation

  1. Install Required Tools: Ensure you have Kali Linux (or another Linux distribution with the necessary tools installed) running, preferably as a virtual machine. Ensure tools like Nmap, Curl, WhatWeb, Dirb, Gobuster, and Nikto are installed:
    sudo apt update && sudo apt install nmap curl whatweb dirb gobuster nikto -y
  2. Deploy Target Machine: Set up a target vulnerable machine (e.g., Metasploitable2) on the same virtual network as your Kali machine. Ensure network settings allow communication between the two VMs (e.g., Host-only or NAT Network in VirtualBox/VMware).
  3. Find the Target IP Address: Log in to your target machine (e.g., Metasploitable2 credentials are often msfadmin/msfadmin). Use the command ip addr or ifconfig to find its IP address. Note this address down; we will refer to it as TARGET_IP throughout this lab.
    # Example command on target machine
    ip addr show eth0 | grep "inet "
  4. Find Your Kali IP Address (Optional but helpful): In your Kali terminal, run ip addr to know your own IP and network range.

Phase 1: Server Discovery (Is the Target Alive?)

Before scanning services, confirm the target host is online and reachable using basic connectivity tests.

Step 1.1: Basic Ping Check

Use the ping command to send ICMP echo requests and verify basic network connectivity.

ping -c 4 TARGET_IP

Expected Result: Look for replies (e.g., "64 bytes from ... ttl=...") indicating the host is responding. Note that some systems block ICMP, so a lack of response doesn't always mean the host is down.

Step 1.2: Nmap Host Discovery

Use Nmap's -sn (No Port Scan) flag for more reliable host discovery, which uses various probes (ARP, ICMP, TCP SYN/ACK to common ports).

sudo nmap -sn TARGET_IP

Alternatively, scan the entire local network range (find your range first using ip addr):

sudo nmap -sn YOUR_NETWORK_RANGE/24

Expected Result: Nmap will list hosts that are "up". Confirm your TARGET_IP appears in the list.

Phase 2: Service Discovery (What's Running?)

Identify open ports and the services listening on them, focusing on web-related ports (80, 443, 8080, etc.).

Step 2.1: Basic TCP Port Scan (Common Ports)

Perform a TCP SYN scan (-sS, usually requires sudo) or TCP Connect scan (-sT) on the most common ports.

sudo nmap -sS -F TARGET_IP

(-F scans the 100 most common ports). Look for ports listed as open, especially 80 (HTTP) and 443 (HTTPS).

Step 2.2: Comprehensive Port Scan (All Ports)

Scan all 65535 TCP ports. This takes much longer but is more thorough for comprehensive assessment.

sudo nmap -sS -p- -T4 TARGET_IP

(-p- means all ports, -T4 increases speed - use cautiously in production environments).

Expected Result: A list of all open TCP ports on the target. Identify web-related ports (80, 443, 8000, 8080, 8443, etc.). Note these down for the next steps.

Step 2.3: Service Version Detection

Identify the specific services and their versions running on the open ports found in previous steps.

sudo nmap -sV -p PORT_LIST TARGET_IP

Replace PORT_LIST with the comma-separated list of open web-related ports found previously (e.g., -p 80,443,8080).

Expected Result: Nmap attempts to determine the service (e.g., http, https, http-proxy) and version (e.g., Apache httpd 2.2.8, Nginx 1.18.0) for each specified open port.

Phase 3: Server Identification (Web Server & OS)

Gather more details about the web server software and the underlying operating system using advanced fingerprinting techniques.

Step 3.1: Nmap OS Detection

Attempt to identify the operating system (requires open and closed ports for accuracy).

sudo nmap -O TARGET_IP

Expected Result: Nmap provides its best guess for the OS (e.g., Linux 2.6.x). Accuracy varies based on available information and network conditions.

Step 3.2: Basic Banner Grabbing (Manual/Curl)

Retrieve HTTP headers, which often contain the Server banner revealing web server information.

Using Curl (Recommended):
curl -I http://TARGET_IP:PORT

(Replace PORT with the open HTTP port, usually 80. Use https:// and port 443 for HTTPS, adding -k if the certificate is self-signed).

Expected Result: Look for the Server: header (e.g., Server: Apache/2.2.8 (Ubuntu)).

Tool Focus: WhatWeb

WhatWeb identifies websites and the technologies they use. It has over 900 plugins to recognize CMS, analytics packages, web servers, embedded devices, version numbers, and more. It's particularly effective for comprehensive web technology fingerprinting.

Step 3.3: Basic WhatWeb Scan

Run a default scan against the target web server to identify technologies and frameworks.

whatweb http://TARGET_IP:PORT

(Adjust protocol http/https and PORT based on discovered open web ports).

Expected Result: A concise summary of detected technologies (e.g., Apache, JQuery, PHP, specific CMS platforms).

Step 3.4: Verbose and Aggressive WhatWeb Scan

Increase verbosity (-v) to see plugin details and use aggression level 3 (-a 3) to perform more requests for potentially better version detection.

whatweb -v -a 3 http://TARGET_IP:PORT

Expected Result: More detailed output, including descriptions of detected plugins, specific version numbers if found (like Apache[2.2.22]), OS hints from server headers (HTTPServer[Ubuntu Linux]), detected libraries (JQuery), and potentially other framework details or interesting headers (X-UA-Compatible). Examine the "Summary", "Detected Plugins", and "HTTP Headers" sections carefully.

Phase 4: Hidden Content Discovery

Search for files and directories that are not directly linked from the main web pages but may contain sensitive information or functionality.

Tools Overview

Tools like Dirb, Gobuster, and Nikto automate the process of requesting common filenames and directory paths based on wordlists. Each tool has unique strengths for different reconnaissance scenarios.

Step 4.1: Directory/File Enumeration with Dirb

Dirb is an online directory scanner that searches web servers for hidden files, directories, and pages. It operates by sending HTTP queries to the server and analyzing the answers to identify the existence of a file or directory. Dirb features an internal wordlist file with roughly 4000 words for brute force attacks.

Basic Directory Scanning

Dirb uses a default wordlist to find common directories and files:

dirb http://TARGET_IP:PORT

(Adjust protocol and port as needed). For larger wordlists:

dirb http://TARGET_IP:PORT /usr/share/wordlists/dirb/big.txt

Advanced Dirb Options

Using Different Wordlists: Dirb includes specialized wordlists for different technologies:

dirb http://TARGET_IP:PORT /usr/share/wordlists/dirb/vulns/apache.txt

Saving Output to File:

dirb http://TARGET_IP:PORT -o result.txt

Limiting to Specific File Extensions:

dirb http://TARGET_IP:PORT -X .php

Ignoring Specific HTTP Status Codes:

dirb http://TARGET_IP:PORT -N 302

Adding Request Delays (milliseconds):

dirb http://TARGET_IP:PORT -z 100

Disabling Recursive Scanning:

dirb http://TARGET_IP:PORT -r

Verbose Output (Showing All Requests):

dirb http://TARGET_IP:PORT -v

HTTP Authentication:

dirb http://TARGET_IP:PORT -u USERNAME:PASSWORD

Custom User-Agent:

dirb http://TARGET_IP:PORT -a "USER_AGENT_STRING"

Expected Result: A list of discovered paths with their HTTP status codes (look for 200 OK, 301/302 Redirect, 403 Forbidden - indicates existence but no access).

Note

Dirb is a powerful tool for locating hidden directories and files on web servers. Always use it ethically and with proper authorization. Unauthorized scanning may have legal consequences.

Step 4.2: Directory/File Enumeration with Gobuster

Gobuster is a directory and file brute-forcing tool written in Go that offers significant speed advantages over traditional scanners like Dirb. Its primary benefits include high performance due to Go's concurrency support and efficient resource usage.

Basic Directory Scanning

The most common usage for directory and file enumeration:

gobuster dir -u http://TARGET_IP:PORT -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -x php,txt,html,bak,old

(-u specifies target URL, -w selects wordlist, -x specifies common extensions to test for files).

Gobuster Modes Overview

Gobuster operates in three primary modes:

  • dir - Classic directory/file brute-forcing (most commonly used)
  • dns - DNS subdomain enumeration
  • vhost - Virtual host discovery on web servers

Advanced Gobuster Options

Enhanced Output and Display:

gobuster dir -u http://TARGET_IP:PORT -w WORDLIST -e -q -o results.txt

(-e shows full URLs, -q quiet mode hides banner, -o saves output to file)

Performance Tuning:

gobuster dir -u http://TARGET_IP:PORT -w WORDLIST -t 50 --timeout 10s

(-t sets threads (default: 10), --timeout sets HTTP request timeout)

Handling Special Cases:

gobuster dir -u http://TARGET_IP:PORT -w WORDLIST -f -r -U admin -P password

(-f appends slash to directories, -r follows redirects, -U/-P for HTTP authentication)

Wildcard Handling:

gobuster dir -u http://TARGET_IP:PORT -w WORDLIST --wildcard

(Essential when target uses wildcard DNS - continues scan even with non-existent domains)

Common Wordlists for Gobuster

Kali Linux includes several useful wordlists:

  • /usr/share/wordlists/dirb/common.txt - General purpose (small)
  • /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt - Balanced size/coverage
  • /usr/share/wordlists/dirb/big.txt - Comprehensive but slower
  • /usr/share/seclists/Discovery/Web-Content/ - Various specialized lists

Practical Examples

Quick Scan with Common Extensions:

gobuster dir -u http://TARGET_IP:PORT -w /usr/share/wordlists/dirb/common.txt -x php,html,txt,js,css,bak -t 30

Comprehensive Scan with Output:

gobuster dir -u http://TARGET_IP:PORT -w /usr/share/wordlists/dirb/big.txt -x php,html,txt,bak,old,zip,sql -t 50 -e -o gobuster_scan.txt

Verbose Scan with Authentication:

gobuster dir -u http://TARGET_IP:PORT/admin -w /usr/share/wordlists/dirb/common.txt -U admin -P password -v

Expected Result: Similar to Dirb, lists discovered paths with HTTP status codes. Pay attention to:

  • 200 OK - Accessible content
  • 301/302 Redirect - Page redirection
  • 403 Forbidden - Exists but no access (often significant)
  • 404 Not Found - Standard "not found" response

Note

Gobuster's main limitation is the lack of recursive directory scanning - each directory level requires a separate scan. However, its speed and accuracy make it an essential tool for penetration testing. Always ensure you have proper authorization before scanning.

Step 4.3: Vulnerability & Misconfiguration Scanning with Nikto

Nikto is an open-source web server scanner that performs comprehensive tests against web servers for multiple security items. It checks for server configuration issues, outdated software, dangerous files, and known vulnerabilities across various web servers including Apache, Nginx, IIS, and others.

Basic Vulnerability Scanning

The fundamental Nikto command for comprehensive web server scanning:

nikto -h http://TARGET_IP:PORT

For HTTPS targets, use the -ssl option:

nikto -h https://TARGET_IP:PORT -ssl

Advanced Nikto Options

Output and Reporting:

nikto -h http://TARGET_IP:PORT -o nikto_scan.html -F htm

(-o specifies output file, -F sets format: htm, csv, txt, xml)

Specific Port and Protocol Testing:

nikto -h TARGET_IP -p PORT -ssl

(Explicitly define port and SSL when non-standard configurations are used)

Enhanced Scanning with Authentication:

nikto -h http://TARGET_IP:PORT -id USERNAME:PASSWORD

(-id for basic HTTP authentication when protected areas need scanning)

Targeted Scanning Options:

nikto -h http://TARGET_IP:PORT -Tuning 1,2,3,4,5,6,7,8,9,0,a,b,c,x

(Tuning options: 1=File Upload, 2=Injection, 3=XSS, 4=OS Cmd, 5=File Disclosure, etc.)

Practical Scanning Examples

Comprehensive Scan with HTML Report:

nikto -h http://TARGET_IP:PORT -o /root/scan_results/nikto_scan.html -F htm

Quick Scan with Specific Tests:

nikto -h http://TARGET_IP:PORT -Tuning 1,2,3,4,5

Update Nikto Database:

nikto -update

(Always ensure Nikto's vulnerability database is current before important scans)

Interpreting Nikto Results

Expected Result: Detailed output highlighting:

  • + - Found items (directories, files, configurations)
  • OSVDB/CVE references - Specific vulnerability identifiers
  • Server misconfigurations - Dangerous settings and exposures
  • Outdated software versions - With known vulnerabilities
  • Missing security headers - Security best practices not implemented
  • Default files and scripts - Often contain sensitive information
Note

Nikto is included by default in Kali Linux. For other distributions, install via package manager (apt-get install nikto on Debian/Ubuntu, pacman -S nikto on Arch) or from the official GitHub repository. Nikto scans can be intrusive and generate significant log activity - use responsibly and with proper authorization.

Phase 5: Analysis and Reporting

Correlation is Key

The real value in footprinting comes from correlating the findings from different phases and tools:

  • Did Nmap identify Apache 2.2.8? Research known CVEs for that version.
  • Did WhatWeb identify PHP 5.2.4 and JQuery? Look for vulnerabilities specific to those versions or related components.
  • Did Dirb/Gobuster find an /admin or /phpmyadmin directory? Try accessing it in a browser.
  • Did Nikto report a specific configuration weakness? Investigate how it might be exploited.
  • Combine OS information with service versions to build a comprehensive target profile.

Document your findings methodically in your report, noting the tool used, the command run, the key results, and potential security implications or next steps for vulnerability analysis.

Conclusion

You have successfully performed fundamental web infrastructure footprinting using common Kali Linux tools, following a methodology aligned with CEH objectives. You discovered the target server, identified running services, fingerprinted the web server software and OS, identified underlying technologies using WhatWeb, and searched for hidden content. These steps provide a foundational understanding of the target's web presence, crucial for subsequent vulnerability analysis and penetration testing phases.

Legal and Ethical Disclaimer

This lab guide is for educational purposes within authorized environments only. Unauthorized scanning or testing of systems is illegal and unethical. Always obtain explicit permission before conducting any security assessments and comply with all applicable laws and regulations. The techniques shown here should only be used in environments you own or have written authorization to test.

Modules (130) ← CEH — Ethical Hacking