How it works
TCP requires both endpoints to track every connection. An attacker creates connection state that is never completed or never used, filling tables on servers, load balancers and firewalls until new sessions are refused.
State is a smaller resource than bandwidth
Volumetric attacks aim at the size of your circuit. State-exhaustion attacks aim at a resource that is usually far smaller and much cheaper to consume: the tables in which network devices record connections.
A server tracks every TCP connection from the first packet of the handshake until it is closed or timed out. So does the load balancer in front of it, and so does any stateful firewall in the path. Each of those tables has a finite limit, and it is frequently reached at traffic volumes that barely register against available bandwidth. An attack that would be trivially absorbed as raw bits can still take a service offline by filling a table.
This is why the two categories demand different responses. Bandwidth problems must be solved upstream. State problems are usually solved by changing what tracks state, and where — which is often within your own control.
SYN floods and the half-open handshake
A TCP connection opens in three steps: the client sends SYN, the server replies SYN-ACK and records the pending connection, and the client completes with ACK. The asymmetry an attacker exploits sits in the second step. The server must allocate and hold state after receiving a single packet, before the client has proved anything at all.
A SYN flood sends handshake openings and never completes them. Each one occupies a slot in the backlog queue until it times out, which is typically measured in tens of seconds. Because the attacker discards the SYN-ACK rather than answering it, the source address can be forged, which removes any possibility of blocking by source and makes the traffic appear to come from everywhere.
The arithmetic favours the attacker heavily. A modest packet rate sustained for the duration of the timeout is sufficient to keep a backlog permanently full, and the attacker spends almost nothing per connection while the server spends memory and a timer.
SYN cookies, and what they cost
SYN cookies remove the asymmetry. Instead of allocating state on the SYN, the server encodes the connection parameters into the sequence number of its SYN-ACK and keeps nothing. If the client returns a valid ACK, the parameters are recovered from the acknowledgement number and the connection is built then — at a point where the client has demonstrably received the reply and therefore cannot have forged its address.
The trade-off is real and worth understanding before enabling it reflexively. A cookie has limited space, so some TCP options negotiated during the handshake cannot be preserved. Depending on the implementation this can mean losing window scaling or selective acknowledgement on cookie-built connections, which degrades throughput for high-latency or high-bandwidth clients.
Most modern stacks handle this well by activating cookies only when the backlog is actually under pressure, so normal connections negotiate options properly and cookies apply solely during an attack. That behaviour is the sensible default, and it is worth confirming rather than assuming.
ACK and RST floods
Not every state attack targets the handshake. Sending large volumes of ACK or RST packets that belong to no existing session imposes a different cost: every stateful device in the path must look each packet up in its connection table to determine that it does not match anything.
A stateless router discards such traffic cheaply. A stateful firewall or load balancer cannot — the lookup is the work, and it happens whether or not the packet is legitimate. The result is a device saturating its processing capacity while its throughput graph looks unremarkable, which frequently misleads a first-line diagnosis.
These floods are also used deliberately against inspection devices rather than servers. A firewall that fails closed under load takes the service down as effectively as any attack on the origin, and a firewall that fails open removes the protection everything behind it assumed was present.
Detection signals
State exhaustion is one of the more legible attack categories, because the ratios it distorts are stable under normal conditions.
- Handshake completion ratio — the proportion of SYNs that become established connections. Normal traffic sits high and steady; a SYN flood collapses it immediately.
- Half-open connection count as a proportion of the configured backlog, alarmed well before saturation.
- TCP flag distribution — ACK or RST volume that greatly exceeds established sessions has no benign explanation.
- Connection table occupancy on every stateful device in the path, not only on the servers. The firewall usually fills first.
- CPU against throughput on inspection devices. Rising CPU with flat throughput is the signature of per-packet lookup cost.
- Connection setup latency, which degrades before outright failure and provides earlier warning than error rates.
Where to place the defence
The governing principle is that state should be created as late as possible, by infrastructure designed to absorb it.
Handshake completion belongs at the edge — on a load balancer, reverse proxy or CDN built for connection volume — so incomplete handshakes are resolved there and origin servers only ever see connections a real client finished. This single architectural choice removes most of the exposure.
Invalid-state traffic should be dropped as far upstream as possible, ideally by stateless equipment that discards it without a lookup. Pushing that decision to a stateful device is precisely the cost the attack is trying to impose.
Connection-rate limits by source network add useful protection against non-spoofed attacks, but they are not a defence against SYN floods specifically, because forged sources defeat any per-source accounting. They are worth having for other reasons; they should not be relied on here.
Finally, review the timeouts. A long half-open timeout multiplies the effect of every attack packet. Shortening it reduces how long each forged handshake occupies a slot, at some cost to clients on genuinely poor connections.
Response priorities
Establish first whether the constraint is state or bandwidth, because the two lead in opposite directions. If circuits have headroom while connections fail, the problem is state, and escalating to a transit provider will not resolve it.
Then identify which device is saturating. Servers, load balancers and firewalls all exhibit similar symptoms from the perspective of a failing client, and the fix differs entirely depending on which one has run out. Connection table occupancy per device answers this in seconds and is worth having on a dashboard before it is needed.
If SYN cookies are available and not enabled, enable them. If a stateful firewall sits in front of infrastructure that does not require it, consider whether it belongs there — a firewall inspecting traffic destined for a public web service often contributes more exposure than protection during this category of attack.
Also known as
What defenders may observe
- SYN-to-ACK completion ratio collapses
- Half-open connection tables grow toward their limit
- ACK or RST packets greatly exceed the number of established sessions
- Firewall or load balancer CPU rises while throughput stays flat
- New connections time out although the circuit is not saturated
Prevention and mitigation
- Enable SYN cookies or equivalent handshake proxying at the edge
- Terminate connections on stateless or high-capacity infrastructure before they reach origin servers
- Drop invalid-state TCP traffic as early in the path as possible
- Apply connection-rate limits by source network and reputation
- Baseline TCP flag ratios so anomalies are visible without a signature
Initial response priorities
- Shift mitigation upstream before the circuit saturates.
- Compare packet rates, flags and source distribution with the baseline.
- Rate-limit or filter the affected service without blocking known-good traffic.
Use your approved incident-response plan and adapt decisions to your environment. This is educational guidance, not a substitute for live incident leadership.