Packet Capture and Traffic Analysis
Packet capture records network frames or packets as they pass an interface. It answers questions strace cannot: what actually left or arrived on the wire, whether TLS hides application bytes, and whether a drop happened in the kernel, on the network path, or past the first hop.
This page covers tcpdump on Linux, when to open captures in Wireshark, and how to capture safely in production.
For syscall-level debugging (including connect, sendto, recvfrom), see System Calls. For cloud-scale connection metadata without full payloads, see VPC Flow Logs and Network RCA.
When to Use Pcaps vs strace
Section titled “When to Use Pcaps vs strace”| Question | Better tool |
|---|---|
Did my process attempt connect() and what errno did the kernel return? | strace — see syscalls and return values. |
| Did a SYN leave the NIC? Was there a TCP handshake? Retransmits? | tcpdump / Wireshark — see packets. |
| Is the peer sending RST or ICMP errors? | tcpdump — visible on the wire. |
| TLS: what certificate or cipher? (metadata only unless you have keys) | Wireshark — decode TLS handshakes; payload stays encrypted without session keys. |
| Container egress: which interface and bridge? | Often both — strace for app behavior, capture on veth or host bridge for actual frames. |
strace shows the boundary between your process and the kernel. tcpdump shows what crossed a network interface (after the kernel stack, encapsulation, and sometimes offload). They complement each other: a failed connect() with ECONNREFUSED matches a TCP RST in a capture; a successful connect() with hangs may show retransmits or no reply in the capture.
tcpdump basics
Section titled “tcpdump basics”tcpdump reads live traffic or writes pcap files for later analysis. It usually needs root or CAP_NET_RAW (and CAP_NET_ADMIN for some operations).
# List interfacestcpdump -D
# Capture first 100 packets on eth0, print to terminalsudo tcpdump -i eth0 -c 100
# Write to file (rotate by size with -C / -W in production scripts)sudo tcpdump -i eth0 -w /tmp/capture.pcapCommon BPF-style filters (tcpdump’s expression language):
# Host A talking to host B (either direction)sudo tcpdump host 198.51.100.10 and host 198.51.100.20
# TCP port 443 onlysudo tcpdump tcp port 443
# Subnetsudo tcpdump net 10.0.0.0/24
# Combine: HTTP or HTTPS to one serversudo tcpdump host web.example.com and \( tcp port 80 or tcp port 443 \)Tips:
-n— Do not resolve hostnames (faster, clearer in incidents).-nn— Also skip port name resolution.-s snaplen— Snapshot length;0or large values capture full payloads (privacy and disk impact). Default is often enough for headers-only troubleshooting.-c N— Stop after N packets (good for quick samples).
What to Look for in the Capture
Section titled “What to Look for in the Capture”Once you have output (terminal lines or a pcap), you are looking for TCP behavior, ICMP, and application-adjacent clues. Correlate with routes, security groups / firewalls, NAT, load balancer idle timeouts, and process logs — the capture shows the network path, not why an app returned 500.
| Pattern | What it usually means | Where to look / fix |
|---|---|---|
| SYN repeats, no SYN-ACK | Remote not answering on that port, ACL/firewall drop, wrong IP/port, asymmetric routing, or target down. | telnet/nc test; check SG/NACL; verify listener on server; traceroute/path MTU if intermittent. |
| SYN → SYN-ACK → RST (quick reset after handshake) | Nothing listening on port, proxy/LB rejecting, or policy RST. | Confirm service bind address (0.0.0.0 vs 127.0.0.1); backend health on LB; middlebox rules. |
| Mid-connection RST | Idle timeout at LB/proxy, pod killed, admin reset, or explicit RST from stack. | Align TCP/LB idle timers with HTTP for Operators; check deploys/restarts. |
| Retransmissions (same seq repeats, duplicate ACKs in Wireshark) | Loss, congestion, asymmetric path, or severe latency. | Path MTU / fragmentation; Wi-Fi or VPN paths; oversubscribed links; capture both ends if possible. |
| ICMP “unreachable” / “fragmentation needed” | No route to host/net, port closed, or MTU black hole. | Routing tables; PMTUD / MSS; UDP vs TCP path. |
| TLS alert after Client Hello (best seen in Wireshark) | Cert/name mismatch, cipher/protocol mismatch, expired cert, or middlebox interference. | SNI, server chain, minimum TLS version; compare with openssl s_client. |
DNS failures (query without response, SERVFAIL in decode) | Resolver, VPC DNS, or upstream issues. | Resolver config; Route 53 / private zones if on AWS. |
tcpdump one-line TCP flags (varies by verbosity; often Flags field):
S— SYN (start of handshake)..orA— ACK (often mid-flow).F— FIN (orderly close).R/R.— RST (abort or refuse).P— PSH (often piggybacks data).
Many “errors” are not literal ERROR strings: they are missing packets (expected reply never arrives), wrong endpoints, or ICMP lines mixed into the trace. In Wireshark, use Expert Information, TCP retransmission coloring, and Follow TCP stream to turn noise into a story.
Reading captures in Wireshark
Section titled “Reading captures in Wireshark”Wireshark (GUI) or tshark (CLI) decodes protocols and offers display filters (different syntax from tcpdump).
Useful workflows:
- Follow TCP stream — Right-click a packet → Follow → TCP Stream. Reconstructs application-layer bytes when not encrypted.
- Display filters — e.g.
tcp.port == 443,ip.addr == 10.0.1.5,dns.qry.name contains "example". - Statistics → Conversations — Who talks to whom, bytes per flow.
- TLS — Inspect Client Hello (SNI, cipher suites) without decrypting application data.
Export from tcpdump as pcap; Wireshark opens it directly.
Network-related syscalls and pcaps together
Section titled “Network-related syscalls and pcaps together”When an app fails to reach a remote host:
- strace may show
connect()returning-1 EHOSTUNREACH,ETIMEDOUT,ECONNREFUSED, or blocking inpoll/selecton the socket. - A capture on the correct interface shows whether SYNs leave the box, whether SYN-ACK returns, or whether ICMP (e.g. “fragmentation needed”) appears.
For UDP, strace shows sendto/recvfrom; the capture confirms whether packets leave and whether responses arrive (DNS is a common case).
Safe capture in production
Section titled “Safe capture in production”Capturing on busy hosts can hurt performance and expose secrets (HTTP bodies, tokens in URLs, database payloads if unencrypted).
- Narrow the filter — Interface + host + port, not
tcp port any. - Limit duration and size —
-c, rotate files, stop when you have the handshake or error pattern. - Avoid full payloads unless policy allows — smaller snaplen for metadata-only triage.
- Use staging — Reproduce with synthetic traffic when possible.
- Encrypt and restrict pcap files — Treat them like credentials; delete when done.
- In cloud VPCs, prefer flow logs for aggregate allow/deny and volume; use host capture when you need TCP flags, latency, or payload-adjacent debugging.
Permission and containers
Section titled “Permission and containers”- On the host, attach to the veth or bridge that carries container traffic if you need to see container egress; capturing inside the container network namespace may require
nsenteror running tcpdump in that namespace. - Kubernetes — Ephemeral debug pods or node-level capture are common patterns; coordinate with platform policy.
Summary
Section titled “Summary”| Tool | Sees |
|---|---|
| strace | Syscalls, errno, blocking on sockets. |
| tcpdump | Packets on an interface; write pcap. |
| Wireshark / tshark | Decode, streams, TLS metadata, statistics. |
| Flow logs | Cloud connection records (no full payload). |
Use strace for “what did the app ask the kernel?” and packet capture for “what crossed the network?”