Testing firewall configuration is a critical step in securing any network infrastructure that’s often overlooked until something goes wrong. Whether you’re a sysadmin deploying new rules or a developer trying to figure out why your application can’t reach a service, having the right tools and methodology to verify firewall behavior can save hours of debugging headaches. This guide walks you through using two essential network diagnostic tools – Nmap for active scanning and Tcpdump for passive packet analysis – to effectively test, validate, and troubleshoot firewall configurations in real-world scenarios.

How Network Scanning and Packet Analysis Work

Before diving into implementation, it’s worth understanding how these tools interact with firewalls at the network level. Nmap operates by sending various types of packets to target hosts and analyzing the responses (or lack thereof) to determine port states, service versions, and network topology. Firewalls sitting between Nmap and the target can drop, reject, or modify these packets, creating different response patterns that reveal firewall behavior.

Tcpdump works at a lower level, capturing raw network packets as they traverse network interfaces. When combined with Nmap testing, Tcpdump provides the “ground truth” of what’s actually happening on the wire – showing exactly which packets are being sent, received, dropped, or modified by network devices including firewalls.

The key insight is that firewalls don’t just block traffic; they exhibit specific behaviors that create identifiable signatures. A dropped packet results in no response and eventual timeout, while a rejected packet generates an ICMP unreachable or TCP reset. Understanding these nuances helps distinguish between different firewall rules and configurations.

Setting Up Your Testing Environment

Effective firewall testing requires proper positioning of your testing tools. You’ll typically want to run Nmap from outside the firewall (simulating external attackers or legitimate external services) while running Tcpdump on multiple points to see packet flow.

First, ensure you have the necessary tools installed and sufficient privileges:

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install nmap tcpdump

# CentOS/RHEL
sudo yum install nmap tcpdump

# Verify installation and permissions
sudo nmap --version
sudo tcpdump --version

For comprehensive testing, you’ll want to capture packets on multiple interfaces. Here’s a basic Tcpdump setup that logs to rotating files:

# Capture on external interface with rotation
sudo tcpdump -i eth0 -s 65535 -w /tmp/firewall-test-%Y%m%d-%H%M%S.pcap -G 3600 -W 24

# Capture specific traffic related to your test
sudo tcpdump -i any -n 'host 192.168.1.100 or port 22 or port 80' -w test-capture.pcap

When testing production firewalls, coordinate with your network team and consider the impact of scan traffic. Some intrusion detection systems will flag aggressive Nmap scans as malicious activity.

Basic Firewall Probing Techniques

Start with basic connectivity tests before moving to more sophisticated techniques. The simplest approach combines a basic Nmap scan with simultaneous packet capture:

# Terminal 1: Start packet capture
sudo tcpdump -i any -n 'host 192.168.1.100' -w basic-test.pcap

# Terminal 2: Basic port scan
nmap -p 22,80,443,8080 192.168.1.100

# More detailed scan with timing info
nmap -p 1-1000 -T4 -v 192.168.1.100

The output reveals different port states that indicate firewall behavior:

• Open: Port accepts connections – firewall allows traffic
• Closed: Port actively rejects connections – host reachable but service not running
• Filtered: No response received – likely blocked by firewall
• Open|Filtered: Ambiguous response – could be open or filtered

Correlate Nmap results with Tcpdump output to understand the underlying network behavior:

# Analyze captured packets
sudo tcpdump -r basic-test.pcap -n -A

# Look for specific patterns
sudo tcpdump -r basic-test.pcap -n 'tcp[tcpflags] & (tcp-rst) != 0'  # RST packets
sudo tcpdump -r basic-test.pcap -n 'icmp[icmptype] = icmp-unreach'   # ICMP unreachable

Advanced Scanning Techniques

Different scan types reveal different aspects of firewall configuration. SYN scans are stealthier and often bypass basic logging, while connect scans provide more definitive results:

# SYN scan (stealthy, requires root)
sudo nmap -sS -p 1-65535 192.168.1.100

# Connect scan (more thorough, works as normal user)
nmap -sT -p 1-65535 192.168.1.100

# UDP scan (often reveals different firewall rules)
sudo nmap -sU -p 53,67,68,123,161 192.168.1.100

# ACK scan (maps firewall rules)
sudo nmap -sA -p 1-1000 192.168.1.100

ACK scans deserve special attention for firewall testing because they reveal stateful vs stateless filtering. A stateful firewall will drop unsolicited ACK packets, while a stateless firewall might allow them through if there’s a corresponding allow rule.

Fragment scans can reveal firewall evasion possibilities:

# Fragment packets to evade simple packet filters
sudo nmap -f -sS -p 22,80,443 192.168.1.100

# Decoy scan to mask source
sudo nmap -D RND:10 -sS -p 1-1000 192.168.1.100

Real-World Testing Scenarios

Here are practical scenarios you’ll encounter and how to approach them systematically:

Scenario 1: Web Server Behind Firewall

Testing a typical web server setup with ports 80 and 443 exposed:

# Terminal 1: Capture web traffic
sudo tcpdump -i any -n 'port 80 or port 443' -s 0 -w web-test.pcap

# Terminal 2: Comprehensive web server test
nmap -p 80,443,8080,8443 -sV --script http-enum 192.168.1.100

# Test for common web vulnerabilities and misconfigurations
nmap -p 80,443 --script http-methods,http-headers 192.168.1.100

Scenario 2: SSH Access Through Firewall

SSH often has complex firewall rules including rate limiting and source IP restrictions:

# Test SSH connectivity and gather version info
nmap -p 22 -sV --script ssh-auth-methods,ssh-hostkey 192.168.1.100

# Test for SSH rate limiting
for i in {1..10}; do 
  nmap -p 22 -sS 192.168.1.100 & 
done
wait

Scenario 3: Database Server Security

Database servers typically have strict firewall rules limiting access to specific source IPs:

# Test common database ports
nmap -p 3306,5432,1433,1521,27017 -sV 192.168.1.100

# Test from different source IPs (if available)
sudo nmap -S 10.0.1.100 -p 3306 192.168.1.100

Analyzing Packet Captures for Firewall Behavior

The real insights come from correlating Nmap results with packet-level analysis. Here’s how to extract meaningful information from your captures:

# Basic packet analysis
tcpdump -r capture.pcap -n -c 100

# Look for firewall-generated responses
tcpdump -r capture.pcap -n 'icmp or (tcp[tcpflags] & (tcp-rst) != 0)'

# Analyze timing patterns (may indicate rate limiting)
tcpdump -r capture.pcap -n -ttt

# Extract unique IP conversations
tcpdump -r capture.pcap -n | awk '{print $3 " -> " $5}' | sort -u

Create a simple script to correlate scan results with packet timing:

#!/bin/bash
# firewall-analysis.sh

echo "Starting firewall analysis..."
TARGET=$1

# Start packet capture in background
sudo tcpdump -i any -n "host $TARGET" -w analysis-$TARGET.pcap &
TCPDUMP_PID=$!

sleep 2

# Run comprehensive scan
nmap -p 1-1000 -sS -T4 -v $TARGET > scan-results-$TARGET.txt

# Stop packet capture
sleep 5
sudo kill $TCPDUMP_PID

echo "Analysis complete. Check scan-results-$TARGET.txt and analysis-$TARGET.pcap"

Comparison of Scanning Techniques

Different scan types provide varying levels of information and stealth. Here’s a practical comparison:

Scan Type	Stealth Level	Accuracy	Firewall Detection	Best Use Case
SYN Scan (-sS)	High	High	Excellent	General firewall testing
Connect Scan (-sT)	Low	Very High	Good	Definitive connectivity testing
ACK Scan (-sA)	Medium	Medium	Excellent	Mapping firewall rules
UDP Scan (-sU)	High	Medium	Good	Testing UDP services
FIN Scan (-sF)	Very High	Medium	Excellent	Evading simple filters

Common Issues and Troubleshooting

Several issues frequently arise when testing firewall configurations. Here’s how to identify and resolve them:

Issue: Nmap Shows “Filtered” for All Ports

This usually indicates aggressive firewall dropping or network connectivity issues:

# Test basic connectivity first
ping -c 4 192.168.1.100

# Try different scan techniques
nmap -sn 192.168.1.100  # Ping scan only
nmap -Pn -p 80 192.168.1.100  # Skip ping, test specific port

# Check for ICMP blocking
sudo nmap -PE -sn 192.168.1.100  # ICMP echo
sudo nmap -PP -sn 192.168.1.100  # ICMP timestamp

Issue: Inconsistent Results Between Scans

This often indicates stateful firewall rules or rate limiting:

# Test with delays between scans
nmap -p 80 --scan-delay 1000ms 192.168.1.100

# Use slower timing template
nmap -p 1-1000 -T1 192.168.1.100

Issue: UDP Scans Return No Results

UDP scanning is inherently unreliable. Combine with service-specific probes:

# Standard UDP scan with version detection
sudo nmap -sUV -p 53,67,123,161 192.168.1.100

# Use specific service probes
nmap -sU -p 161 --script snmp-info 192.168.1.100
dig @192.168.1.100 example.com  # Test DNS directly

Best Practices and Security Considerations

When testing firewall configurations, follow these guidelines to avoid disrupting operations or triggering security alerts:

• Coordinate with security teams: Inform IDS/IPS operators about planned testing to avoid false alarms
• Use appropriate timing: Aggressive scans can overwhelm network devices or trigger rate limiting
• Test from multiple sources: Firewall rules often vary based on source IP or network segment
• Document everything: Keep detailed logs of scan parameters and results for comparison
• Verify permissions: Ensure you have authorization to scan target networks

Create standardized testing procedures for consistency:

#!/bin/bash
# Standard firewall test procedure

TARGET=$1
DATE=$(date +%Y%m%d-%H%M%S)
LOGDIR="/var/log/firewall-tests"

mkdir -p $LOGDIR

# Phase 1: Basic connectivity
echo "Phase 1: Basic connectivity test"
ping -c 4 $TARGET > $LOGDIR/ping-$TARGET-$DATE.log

# Phase 2: Port discovery
echo "Phase 2: Port discovery"
nmap -sS -p 1-65535 -T3 $TARGET > $LOGDIR/portscan-$TARGET-$DATE.log

# Phase 3: Service identification
echo "Phase 3: Service identification"  
nmap -sV -p $(grep open $LOGDIR/portscan-$TARGET-$DATE.log | cut -d/ -f1 | tr '\n' ',') $TARGET > $LOGDIR/services-$TARGET-$DATE.log

echo "Testing complete. Results in $LOGDIR/"

Integration with Monitoring and Automation

For ongoing firewall validation, integrate these techniques into monitoring workflows. Many organizations use automated scanning to detect configuration drift:

# Simple monitoring script for critical services
#!/bin/bash
CRITICAL_PORTS="22,80,443"
TARGET="production-server.example.com"

RESULT=$(nmap -p $CRITICAL_PORTS $TARGET | grep -c "open")
EXPECTED=3

if [ $RESULT -ne $EXPECTED ]; then
    echo "ALERT: Expected $EXPECTED open ports, found $RESULT"
    # Send alert via your monitoring system
fi

Consider using configuration management tools to maintain consistency between Nmap test results and documented firewall policies. Tools like Nmap Scripting Engine can automate complex testing scenarios and generate structured output for analysis.

For enterprise environments, explore integration with security orchestration platforms that can automatically correlate scan results with firewall logs, network flow data, and vulnerability assessments to provide comprehensive security posture visibility.

---

---

This article incorporates information and material from various online sources. We acknowledge and appreciate the work of all original authors, publishers, and websites. While every effort has been made to appropriately credit the source material, any unintentional oversight or omission does not constitute a copyright infringement. All trademarks, logos, and images mentioned are the property of their respective owners. If you believe that any content used in this article infringes upon your copyright, please contact us immediately for review and prompt action.

This article is intended for informational and educational purposes only and does not infringe on the rights of the copyright owners. If any copyrighted material has been used without proper credit or in violation of copyright laws, it is unintentional and we will rectify it promptly upon notification. Please note that the republishing, redistribution, or reproduction of part or all of the contents in any form is prohibited without express written permission from the author and website owner. For permissions or further inquiries, please contact us.