Module 16 — Availability and denial of service¶

Three requests to /login in secure mode. The shape of the timings (your numbers will differ):

correct password for alice   ~ a few hundred ms   (bcrypt runs)
wrong password for alice     ~ a few hundred ms   (bcrypt runs)
malformed JSON body          ~ a few ms           (rejected before hashing)

The slow hash from Module 6 is doing its job: every guess is expensive for an attacker with a stolen database. It is also expensive for your server on every attempt, valid or not, and /login has no rate limit. The control that protects confidentiality has created an availability problem.

Availability is the letter of CIA that courses usually skip. This module puts a number on it.

Why it matters to a software engineer¶

Module 1's asset question asks what would hurt if something is "disclosed, changed, or unavailable" — the third leg of the CIA triad gets one line in most courses and then disappears. This module gives it a proper pass: resource exhaustion, volumetric abuse, and asymmetric-cost attacks, using the same lab you already ran for Module 9's brute-force scenario, viewed through a different lens.

Visual overview¶

flowchart LR
  A[Attacker: cheap request] --> EP[Endpoint]
  EP --> WORK[Expensive work: slow hash / query / render]
  WORK --> R[Response]
  EP -. no rate limit .-> WORK

Intuition

Module 1's asset question already named "unavailable" as a first-class harm, next to "disclosed" and "changed" — this module finally spends time on it. The interesting failure isn't always a flood of traffic; it's one request that costs the attacker a cent and costs your server a dollar. Scaling out doesn't fix a bad cost ratio — it just makes losing money at a bigger scale.

Volumetric:         many cheap requests -> exhaust bandwidth/connections
Asymmetric-cost:     few expensive requests -> exhaust CPU/memory per request
                      (e.g. a login endpoint's slow password hash runs on
                       every attempt, valid or not)

Hint

A rate limit placed after the expensive work already ran only stops the next request, not the cost of this one. Ask "what runs before I can even say no" — that's where the limit has to sit.

The same telemetry can mean two different incidents — see "Detection overlaps but the response differs" below.

Learning objectives¶

Distinguish volumetric denial of service from asymmetric-cost (algorithmic complexity) denial of service.
Distinguish one source (DoS) from many sources (DDoS), and say which hop each one fills.
Explain why a login endpoint using a deliberately slow KDF (Module 6) is also, unavoidably, a resource-exhaustion target.
Identify the missing control in the lab app and name the standard controls that would add it back.
Read the same telemetry event stream through an availability lens instead of a credential-access lens.

Key concepts¶

Availability is a property of the whole path, not just the server. DNS, gateway, application process, and database can each be the bottleneck. Module 2's "follow one request" diagram is the map: an attacker only needs to exhaust the narrowest point on that path, not every point.

Volumetric vs algorithmic-complexity attacks. Volumetric: overwhelm capacity with sheer request or packet volume. Algorithmic- complexity / asymmetric-cost: a small number of expensive requests that cost the caller little and the server a lot — a regex with catastrophic backtracking, an uncapped file upload, or, concretely in this lab, a login attempt that forces a full slow-KDF computation (Module 6) for every guess, correct or not.

One source or many (idea 19). Denial of service (DoS) is one caller filling a resource. Distributed denial of service (DDoS) is many callers filling one. Those two words are about who. Volumetric and asymmetric are about what fills up. You can have all four combinations. The lab login is one source buying bcrypt: DoS, asymmetric. A link full of packets from many addresses is DDoS, volumetric. Many sources each triggering bcrypt is distributed and asymmetric: blocking a single address does not stop the CPU spend.

flowchart TB
  one["one client"] --> cpu["notes API CPU"]
  many["many clients"] --> edge["bandwidth or the connection table"]
  edge --> api["the API, if the edge still has room"]

Intuition

Count the sources, then name the hop that saturated. One address and a hot CPU is a request you can refuse in front of the hash. Many addresses and a full link are upstream of your process: a blocklist of one IP restores nothing.

def availability_case(sources: int, saturated: str) -> str:
    if sources < 1 or saturated not in {"cpu", "bandwidth"}:
        return "name the hop that filled, then count the sources"
    distributed = sources > 1
    who = "DDoS" if distributed else "DoS"
    if saturated == "cpu":
        return f"{who}, asymmetric: each request buys expensive work"
    return f"{who}, volumetric: the path is full of cheap traffic"

The function classifies a case you already observed. It does not generate load. In this lab you time /login from one client. You do not have a second source, and you do not have a saturated link. The distributed rows are how you recognize a case the lab cannot stage.

Asymmetric cost is the interesting case for engineers. A volumetric flood is an infrastructure/capacity problem (rate limiting, autoscaling, upstream scrubbing) — someone else's team usually owns the fix. Asymmetric cost is an application design problem you own: it lives in your own request-handling logic, so it is scaling, not spending, that fails to fix it.

Missing controls compound. The lab's /login route has no rate limit, no lockout, and no CAPTCHA-equivalent challenge — the same gap Module 9 exploited to demonstrate credential access (DET-001) is, independently, an availability gap: nothing stops an attacker from sending attempt 7, 700, or 7000.

Detection overlaps but the response differs. DET-001 (password guessing) and an availability incident can share the exact same telemetry signature — a burst of login_failure events from one source — but the correct response differs: credential-access response is "was any password correct, rotate if so"; availability response is "can the service still serve legitimate users, and where do we throttle."

Disaster recovery is a different property than surviving an attack. Everything above defends uptime while under load. DR defends the case where the service goes down anyway — attack, operator error, or plain hardware failure — and asks whether you can come back. Two numbers make this concrete instead of aspirational:

RPO (recovery point objective): how much data you can afford to lose, measured in time since the last good backup. "We back up nightly" means an RPO of up to 24 hours, whether or not anyone said so on purpose.
RTO (recovery time objective): how long you can afford to be down while restoring.

Restore objectives (idea 19). Incident recovery in Module 11 puts the API back after you remove the cause. These two numbers say how much of the sqlite file you already accepted losing, and how long users wait.

flowchart LR
  backup["last good backup"] --> rpo["notes written after that: the RPO window"]
  outage["API stops answering"] --> rto["API answers again: the RTO window"]

from datetime import datetime, timedelta

def rpo_hours(last_backup: datetime, incident_at: datetime) -> float:
    return (incident_at - last_backup) / timedelta(hours=1)

A nightly backup and an incident at 18:00 is an RPO of about eighteen hours, whether or not a policy document says "four."

A backup you have never restored is a belief, not a control — the same "controls fail, plan for that" idea from Module 1 applies to backups themselves: the failure mode isn't "we forgot to back up," it's "we backed up for two years and the restore script silently broke in month three."

Worked scene — putting a number on it¶

Illustrative figures; use your own measurements.

Measured. A failed login for alice in secure mode takes a few hundred milliseconds of CPU, about the same as a successful one.
Measured. A malformed body is rejected in a few milliseconds.
Calculate. At ~250 ms of CPU per attempt, one core handles about four logins a second. Six attempts from simulate.py barely register. Sixty a second from a script uses fifteen cores.
Check the code. /login has no rate limit, lockout, or per-source throttle.

What that implies. Bcrypt protects a stolen database, and it also makes every attempt expensive for your server. The fix is to make attempts cheap to refuse (per-source and per-account limits before the hash runs), not to make the hash fast again.

Architecture connection¶

Rate limiting, backoff, and circuit breakers belong at the same trust boundary as authentication (Module 3) — the gateway or the endpoint itself — because by the time a request reaches business logic, the expensive work (password hashing, database query, downstream call) has already happened. "Authenticate, then rate-limit" is backwards; the limit has to apply to attempts, not just successes.

Prerequisites¶

Docker/Compose and curl. This experiment requires secure mode because the default teaching mode deliberately uses fast SHA-256. Resetting below deletes lab-only alerts, cases, and data; preserve anything you need first.

Before you run this¶

Write down three answers before you time anything:

Rank these by time: a correct login, a wrong password for alice, a wrong password for a user who doesn’t exist, and a malformed body.
Which of those four timings leaks whether a username exists?
Roughly how many login attempts per second could this API take before legitimate users notice?

Then run the steps. If a result surprises you, which assumption was wrong?

Steps¶

Start from a clean database and run the stack with bcrypt enabled:

make lab-reset
cd labs
LAB_MODE=false docker compose up -d --build
cd ..
curl -s http://127.0.0.1:8080/health

Confirm the health response contains "lab_mode":false. Do not continue if it reports true; otherwise this experiment measures fast SHA-256 and cannot demonstrate the intended cost.

Time a single successful login:

time curl -s -X POST http://127.0.0.1:8080/login \
  -H 'Content-Type: application/json' \
  -d '{"username":"alice","password":"alice-lab-password"}' >/dev/null

Time a single failed login for the known user alice with an obviously wrong password, using the same command with a bad password. Compare the wall-clock time to a trivial rejection (a malformed JSON body, which fails before hashing runs). The failed-but-well-formed attempt should cost close to the same as the successful one — the slow KDF from Module 6 runs either way.

Time a failed login for a non-existent user. In this app, missing users skip bcrypt (row is None short-circuit) — cheaper, and a user-enumeration oracle. That is why step 3 specifies alice. A production login should dummy-hash on unknown users. Confirm "lab_mode" is JSON false (spacing may vary; look at the field). 4. Run the existing six-request brute-force scenario and calculate its approximate requests per second from the elapsed time:

time python3 labs/attack-sim/simulate.py --scenario brute_force

Ingest, then query soc-lite for the resulting login_failure events:

curl -s -X POST http://127.0.0.1:8090/ingest | python3 -m json.tool
curl -s 'http://127.0.0.1:8090/events?event=login_failure' | python3 -m json.tool

Note the timestamps. Ask: at what request rate would this endpoint's CPU become the bottleneck for legitimate logins, given the per-attempt cost you measured in step 3? 6. Read labs/notes-api/app.py's /login route. Confirm there is no rate limit, lockout counter, or per-source throttle — every attempt, valid or not, pays the full authentication cost.

Expected observations¶

In confirmed secure mode, failed and successful logins for the known user cost roughly the same wall-clock time because both run bcrypt; a malformed request is rejected far faster. Exact timings vary by laptop and one sample is noisy, so repeat each measurement several times before comparing it. DET-001 fires on the same burst that would, at higher volume, start starving legitimate logins of CPU.

Security lessons¶

A slow password hash is the correct control against offline cracking (Module 6) and simultaneously an availability liability against online attempts — there is no single control that solves both; you need the slow hash and a rate limit, not one instead of the other.

Common mistakes¶

Concluding "rate limiting is a DevOps/infra problem" and stopping there — see "asymmetric cost is the interesting case for engineers" above.
Testing only the volumetric case and missing that a handful of well- placed expensive requests can be worse than a large number of cheap ones.
Adding a rate limit only on /login success, which does nothing for the cost already paid on every failure.
Treating "we take backups" as a finished control without ever running a restore drill — see "disaster recovery is a different property" above.

Cleanup¶

Restore the default course state:

make lab-reset
make lab-up

Confirm /health reports "lab_mode":true. This removes the secure-mode database whose bcrypt hashes are incompatible with the default teaching mode.

Knowledge check¶

What is the difference between a volumetric and an algorithmic- complexity denial-of-service attack?
Why does a deliberately slow password hash create an availability trade-off?
Where should a rate limit sit relative to the expensive work it's meant to prevent?
Why can DET-001's telemetry signature mean two different incidents?
Why doesn't horizontal autoscaling fix an asymmetric-cost design flaw?
One client is driving /login to 100% CPU. A thousand addresses have filled the link, and the API is idle. Which is DoS, which is DDoS, and what does blocking one address change in each?

Answers: (1) Volumetric overwhelms capacity with sheer volume; algorithmic- complexity makes each individual request disproportionately expensive to process. (2) The same slow KDF that defeats offline cracking (Module 6) also means every online attempt, valid or not, costs real server CPU. (3) Before the expensive work runs, not after — otherwise the limit itself arrives too late to save the cost. (4) A burst of login_failure events can be credential-access reconnaissance or an availability attack; the response differs even though the detection is identical. (5) More servers still each do more expensive work per attacker-controlled request than per legitimate one — the cost ratio, not total capacity, is the problem. (6) The hot CPU from one client is DoS, and refusing that source (or putting the limit before bcrypt) can help. The full link from many addresses is DDoS; one blocked address leaves the rest of the sources.

Engineering assignment¶

Design (do not implement) a rate-limit and lockout policy for /login: what is counted, what the window and threshold are, what happens on breach, and how a legitimate user recovers. State one trade-off your policy accepts (e.g. a shared-IP office network briefly locking out real users).

State an RPO and RTO for notes-api's sqlite file, and one sentence on how you would actually verify the restore works, not just that the backup file exists.

Self-check¶

Answer before expanding. These are the assessment moves on this module's Acme Notes lab, not trivia.

Explain: Why is unthrottled /login an application design problem, not just a capacity problem?

Each guess forces expensive password-verify work (worse under bcrypt) while the attacker sends tiny JSON. Autoscaling copies that cost ratio; it does not fix it.

Predict: Where must a rate limit sit relative to hashing, and what telemetry still looks like DET-001?

Before the expensive work, on attempts not only successes. A burst of login_failure is still the signature — Module 11's rule does not tell you whether this is credential access or an availability attack.

Diagnose: DET-001 fires during a load test of /login. What two incident types share that evidence?

Password guessing (rotate if any success) vs denial of service (can legitimate users still log in, where do we throttle). Responding to the alert name without that split wastes the window.

Design: One tradeoff your lockout policy must accept.

Example: shared-IP office NAT briefly locking real users, vs never locking and letting CPU melt. Count by what (IP, username, both), say how a legitimate user recovers.

Defend: A backup file of notes.db exists. Why is that not DR yet?

RPO/RTO are claims about restore. A backup you have never restored is a belief. Containment of a DoS is throttle/isolate; residual is data you lose if sqlite dies during the incident.

Before you leave¶

Predict — write expected latency/error evidence before you time /login.
Diagnose — name the asymmetric-cost step (hash, query, downstream) from the measurement.
Build — complete the login cost-asymmetry lab and the rate-limit / RPO-RTO assignment (design, do not implement a new attack).

How these are graded: assessment.