Opening Framing: From Theory to Reality
Last week covered the OSI model—a theoretical framework. This week we dive into TCP/IP—the practical protocol stack that actually runs the Internet. While OSI has seven layers, TCP/IP has four, mapping roughly to OSI but designed for real-world implementation.
Understanding TCP/IP at the packet level is essential for security work. When you analyze traffic, write firewall rules, or investigate incidents, you're working with TCP/IP. Every header field matters; attackers manipulate them, and defenders must recognize abnormalities.
This week goes deep: IP headers, TCP headers, the three-way handshake, connection states, and how these protocols can be abused.
Key insight: TCP/IP wasn't designed with security in mind. Many attacks exploit fundamental protocol behaviors that were designed for reliability, not defense against malicious actors.
1) TCP/IP Model vs. OSI
The TCP/IP model has four layers that map to OSI:
TCP/IP Model OSI Model
───────────── ─────────────────────
Application ←───→ Application (7)
Presentation (6)
Session (5)
Transport ←───→ Transport (4)
Internet ←───→ Network (3)
Network Access ←───→ Data Link (2)
Physical (1)
TCP/IP Layer Functions:
- Application: HTTP, DNS, SMTP, SSH, FTP—user-facing protocols
- Transport: TCP and UDP—end-to-end delivery
- Internet: IP, ICMP—routing across networks
- Network Access: Ethernet, Wi-Fi—local delivery
Why TCP/IP Won:
TCP/IP was practical and implemented first. While OSI remained a reference model, TCP/IP powered ARPANET and became the Internet. Security professionals must know both: OSI for conceptual analysis, TCP/IP for practical work.
Key insight: When working with real traffic, think TCP/IP. When categorizing attacks or explaining concepts, OSI often helps.
2) IP Protocol: Headers and Addressing
IP (Internet Protocol) handles addressing and routing. Every packet crossing the Internet has an IP header:
IPv4 Header (20 bytes minimum):
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│Version│ IHL │ ToS │ Total Length │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Identification │Flags│ Fragment Offset │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ TTL │ Protocol │ Header Checksum │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Source IP Address │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Destination IP Address │
└─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘
Security-Relevant Fields:
- Source IP: Can be spoofed! No built-in authentication
- TTL (Time To Live): Decrements at each hop; prevents infinite loops. Also used for OS fingerprinting (Windows=128, Linux=64)
- Protocol: Identifies upper layer (6=TCP, 17=UDP, 1=ICMP)
- Flags/Fragment Offset: Used in fragmentation attacks
IP Fragmentation:
# Large packet gets fragmented:
Original: [IP Header][1500 bytes data]
Fragmented:
Fragment 1: [IP Header][500 bytes] Offset=0, MF=1
Fragment 2: [IP Header][500 bytes] Offset=500, MF=1
Fragment 3: [IP Header][500 bytes] Offset=1000, MF=0
# Attackers abuse fragmentation:
- Tiny fragments hide TCP port info
- Overlapping fragments confuse reassembly
- Fragmentation used to bypass firewalls
Key insight: IP has no security. Source addresses aren't verified, and fragmentation creates complexity attackers exploit.
3) TCP Protocol: Reliable Delivery
TCP provides reliable, ordered delivery over unreliable IP. The TCP header adds connection management:
TCP Header (20 bytes minimum):
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Source Port │ Destination Port │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Sequence Number │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Acknowledgment Number │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Offset│ Res │U│A│P│R│S│F│ Window Size │
├─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┼─┤
│ Checksum │ Urgent Pointer │
└─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┘
TCP Flags (Critical for Security):
- SYN: Synchronize—initiates connection
- ACK: Acknowledge—confirms receipt
- FIN: Finish—graceful close
- RST: Reset—abrupt termination
- PSH: Push—deliver immediately
- URG: Urgent—priority data
Sequence and Acknowledgment:
# TCP tracks every byte:
Client sends: SEQ=1000, 100 bytes of data
Server acks: ACK=1100 (expecting byte 1100 next)
Client sends: SEQ=1100, 200 bytes
Server acks: ACK=1300
# This enables:
- Ordered delivery (reassemble by sequence)
- Reliable delivery (retransmit if no ACK)
- Flow control (window size limits in-flight data)
Key insight: Sequence numbers were originally designed for reliability, not security. Predictable sequences enabled session hijacking until randomization was implemented.
4) TCP Connection Lifecycle
Understanding TCP states is essential for analyzing traffic and attacks:
Three-Way Handshake (Connection Establishment):
Client Server
│ │
│──────── SYN (SEQ=100) ───────>│ Client: SYN_SENT
│ │ Server: SYN_RECEIVED
│<─── SYN-ACK (SEQ=300,ACK=101)─│
│ │
│──────── ACK (ACK=301) ───────>│ Both: ESTABLISHED
│ │
│<═══════ DATA TRANSFER ═══════>│
Four-Way Termination (Connection Close):
Client Server
│ │
│──────── FIN ─────────────────>│ Client: FIN_WAIT_1
│ │ Server: CLOSE_WAIT
│<─────── ACK ─────────────────│ Client: FIN_WAIT_2
│ │
│<─────── FIN ─────────────────│ Server: LAST_ACK
│ │
│──────── ACK ─────────────────>│ Client: TIME_WAIT
│ │ Server: CLOSED
TCP States (from netstat/ss):
LISTEN - Server waiting for connections
SYN_SENT - Client sent SYN, waiting for SYN-ACK
SYN_RECV - Server received SYN, sent SYN-ACK
ESTABLISHED - Connection active
FIN_WAIT_1 - Sent FIN, waiting for ACK
FIN_WAIT_2 - Received ACK for FIN, waiting for FIN
TIME_WAIT - Waiting to ensure remote received final ACK
CLOSE_WAIT - Received FIN, waiting for application to close
LAST_ACK - Sent FIN, waiting for final ACK
CLOSED - Connection terminated
Key insight: Many attacks target specific states. SYN floods exhaust SYN_RECV slots. TIME_WAIT accumulation can cause resource exhaustion.
5) TCP/IP Attacks and Defenses
SYN Flood Attack:
# Attacker sends many SYNs with spoofed source IPs
Attacker ──SYN──> Server (allocates resources, sends SYN-ACK)
──SYN──> Server (allocates more resources)
──SYN──> Server (connection table filling up)
...
# Server's SYN queue fills, can't accept legitimate connections
# Defense: SYN cookies (stateless SYN handling)
TCP Reset Attack:
# Attacker injects RST packet with correct sequence number
Client <══════> Server (established connection)
Attacker ──RST (spoofed)──> Server
# Connection terminated if sequence number is in window
# Defense: TCP sequence number randomization, encrypted connections
Session Hijacking:
# Attacker predicts or captures sequence numbers
1. Sniff traffic to learn sequence numbers
2. Inject packets with valid SEQ/ACK
3. Take over session
# Modern defense: Randomized initial sequence numbers (ISN)
Port Scanning Techniques:
# TCP Connect Scan (full handshake)
nmap -sT target
# Completes 3-way handshake - logged by target
# SYN Scan (half-open)
nmap -sS target
# Only sends SYN, analyzes response - stealthier
# Response analysis:
SYN-ACK = Port open
RST = Port closed
Nothing = Filtered (firewall)
Defense Summary:
- SYN cookies for SYN flood protection
- Randomized ISNs prevent prediction
- Stateful firewalls track connections
- Rate limiting prevents scanning
- Encryption (TLS) prevents injection/hijacking
Key insight: Most TCP attacks require being on the network path (MitM) or require spoofing. Encryption defeats most injection attacks.
Real-World Context: TCP/IP in Security Operations
TCP/IP knowledge powers security operations:
Firewall Rules: Firewall rules specify source/destination IPs and ports. Understanding TCP flags lets you write rules like "allow established connections" (packets with ACK set) while blocking new inbound connections (SYN only).
Traffic Analysis: Unusual TCP flags indicate attacks or reconnaissance. A packet with SYN+FIN is invalid and indicates scanning or evasion. NULL packets (no flags) are used in stealth scans.
Incident Investigation: Packet captures reveal attack timelines. You can trace connection attempts, identify data exfiltration by volume, and spot C2 beacons by connection patterns.
MITRE ATT&CK Reference:
- T1046 - Network Service Discovery: TCP port scanning
- T1095 - Non-Application Layer Protocol: Raw TCP/IP manipulation
- T1571 - Non-Standard Port: Services on unusual ports
Key insight: Deep TCP/IP knowledge separates senior analysts from juniors. When you can read packets like text, investigations go much faster.
Guided Lab: TCP Handshake Analysis
Let's capture and dissect a TCP three-way handshake in detail.