Artica Wiki

Possible Exfiltration

Noticing when far more data leaves a site than normally does — in one answer, or added up at one address. What each threshold does, what it measures, and what it cannot see.

Possible exfiltration notices when far more data leaves a site than normally does — either in one answer, or added up at one address. It never blocks. At the byte level a theft and the quarterly export are the same event, so it raises a finding for a person to judge and refuses nothing.

The question it answers, and the one it does not

Response Scanning reads response bodies for recognisable secrets: card numbers, social security numbers, API keys, private keys. It is good at that and blind to the ordinary breach. It skips anything compressed, reads four content types, stops at a size limit, and matches patterns — so a customer table of names, addresses, order history and password hashes matches nothing at all. Email detection is off by default, and the rest of those columns have no pattern to match.

Counting bytes has none of those limits. It works on gzip, because a byte count needs no decompression. It works on data with no pattern, because it never looks inside. And it adds up across requests, so a thousand pulls of one row each stop being invisible.

It cannot see data taken another way. A mysqldump piped to curl, a reverse shell, exfiltration over DNS, a stolen backup — none of those pass through nginx. EdgeGuard sits in front of inbound HTTP and never sees what your application sends outwards on its own. This measures what leaves through the website, and nothing else.

The two findings

Finding What it means
Possible exfiltration — one large response A single answer was far bigger than this endpoint has ever sent
Possible exfiltration — one IP over time Many perfectly ordinary answers, adding up at one address far beyond what other addresses pull

They are deliberately separate, because they need different responses. The first points at one request — look at what it returned. The second points at a client — look at what it has been doing for the last ten minutes.

console-exfil-02-events.png

The action column reads Observed for both: the request was served and a finding recorded. Nothing was refused, and the column that says what was done says so honestly. Which of the two findings it is stands in the Rule column.

Reading a finding

console-exfil-03-detail.png

Every finding carries the figures that justify it, because "unusual volume" with no numbers is not something anybody can act on or dismiss:

{
  "response_volume": {
    "kind": "client_volume",
    "bytes": 69000000,
    "baseline": 21657,
    "ratio": 3185,
    "window": "10m0s",
    "human": "this address pulled 65.8 MB in 10m0s, against 21 KB for a typical address on this site",
    "user_agent_source": "last request seen from this address"
  }
}

Severity comes from the score. An address pulling far above what the site's others pull scores 10 (high). A single answer a thousand times its baseline scores 10. Anything else that passed the thresholds scores 5 (medium). Nothing is ever critical — critical should mean an attack confirmed, and this signal cannot tell a theft from a legitimate export.

Where the numbers come from

The nginx module counts the bytes of every response from its body filter and reports them to the engine over the local socket. It is counted above gzip, which matters: a 50 MB table compresses to about 5 MB, and the compressed figure would put a 600 MB dump under every threshold.

nginx module  →  unix socket  →  engine  →  per-endpoint baseline  →  finding

Reporting is on by default and costs one short socket write per response, made after the client has been served, with a connect that gives up immediately if the engine is busy. To switch it off for a location:

security_response_feedback off;

That directive is the emergency brake: it removes the measurement from the response path with a reload and no rebuild. It also blinds three other things that read the same signal — the origin load measure, the circuit breaker, and 404-enumeration detection — so it is not the same as switching the finding off in the console.

Paths are grouped by endpoint, not by identifier. /api/orders/48213 and /api/orders/91002 are both /api/orders/:id. Without that, every order id would be its own endpoint with one observation, nothing would ever earn a baseline, and the tracker would be a memory leak that detects nothing.

Learning before judging

An endpoint is watched and never reported until it has served 50 responses. Until then there is nothing to compare against: a first request is infinitely above a baseline that does not exist.

The per-address signal needs more: 50 closed windows of ten minutes, which is over eight hours of traffic. Both counts survive a restart — they are written to the agent's database every five minutes and read back at startup. Before that they lived only in memory, and a node restarted daily could never reach the eight hours the address signal needs.

You can see where a site stands:

curl -s http://127.0.0.1:8080/status | python3 -m json.tool | grep -A 30 response_volume

"observations": 2850,            how much history this endpoint has
"observations_until_judged": 0,  0 means it can be judged
"mean_bytes": 244,               what normal looks like here
"mean_deviation_bytes": 88,      and how much it varies
"requests_decided": 53,          this process only, comparable with the next
"counted_since_start": 60

An empty findings list usually means "nothing can be judged yet", not "nothing is happening". That is why the figures above say which. client_windows_until_judged answers the same question for the address signal.

The baseline adapts. It is an exponentially weighted average, so a sustained change is absorbed: seventy-five identical 3 MB responses produce one finding, not seventy-five, because after about thirty the endpoint's normal is 3 MB. That is right for spotting one anomalous dump and it is a real limit for slow, sustained extraction — which is what the per-address window exists to catch instead. Neither signal covers the other.

Configuration

Global Settings → Data Protection → Response Volume, and the same card on a group, a tenant or a path rule. Everything inherits unless you set it, and leaving a field empty means "use the default" — setting one threshold never zeroes another, so an installation that never opens this page behaves exactly as it did.

console-exfil-01-settings.png

One response, against that endpoint's own history

console-exfil-05-tiers.png

There are two pairs of settings, and a response is reported when it clears either pair. One pair alone cannot serve both a site whose endpoints answer a few hundred bytes and one whose endpoints answer megabytes.

Large response floor — default 1048576 (1 MiB)

How big an answer must be before the modest multiple below applies. Nothing under this size is ever reported through this pair, whatever the multiple.

It exists so that a tiny endpoint growing a little is not news: 200 bytes becoming 4 KB is twenty times normal and still nothing worth a person's attention.

Multiple of normal — default 20

How many times the endpoint's usual size, for answers above that floor. With the defaults, an endpoint that usually sends 2 MB is reported at 40 MB.

Moderate response floor — default 65536 (64 KiB)

A second, lower floor — for answers that are an extreme multiple of normal rather than a modest one.

This is the pair that makes the signal work at the scale most sites actually live at. On a real appliance the API endpoints answer 36 to 290 bytes: a thousand records pulled through a 290-byte endpoint is 290 KB, a thousandfold, and the 1 MiB floor says nothing at all about it.

Extreme multiple — default 200

How many times the usual size, for answers above the moderate floor. The pair is deliberately strict: a modest multiple of a small number is still a small number.

Outside its own spread by — default 8

How far outside the endpoint's normal variation, measured in average deviations rather than a fixed size. This applies to both pairs.

It is what keeps an endpoint whose responses genuinely vary from reporting its own ordinary range. One real endpoint reads a mean of 536 bytes with a deviation of 801 — wider than the mean, and entirely legitimate. Without this test, ratio alone would report its ordinary high days.

An endpoint whose answers are always identical has a deviation of zero, so this test passes trivially and the floors and multiples do the work. That is correct: a perfectly constant endpoint suddenly sending something different is the signal.

One address, over a window

console-exfil-06-window.png

This is the signal for data leaving in pieces that each look entirely ordinary — which is what per-response thresholds can never catch, because no single response is unusual.

Address total floor — default 67108864 (64 MiB)

How much one address must pull within the window before it is reported. An absolute amount, so a quiet site does not report its busiest ordinary visitor.

Multiple of a typical address — default 10

How many times what the site's other addresses pull in the same period. This is a comparison with the site itself, not a fixed number, so it adapts to how the site is actually used.

Both this and the floor must be cleared. On a real appliance a typical address pulls about 21 KB in ten minutes, so the floor is almost always the binding constraint — but on a media site where everyone pulls hundreds of megabytes, this is what stops every visitor being reported.

Window (seconds) — default 600

The period one address's total is measured over. Reported once per window, not once per response — once the threshold is crossed, every later response in that window crosses it too, and reporting each would turn a single pull into hundreds of findings.

The window in flight is lost when the engine restarts, deliberately: restoring a half-finished window would make the first comparison after a restart wrong in a way nobody could reconstruct. The learned baseline survives; the ten minutes in progress do not.

Learning

Both signals compare against what a site normally does, so both need a period of watching before they can judge anything. This setting governs how long that is, and it is the difference between a quiet console and a console that alarms on every new endpoint.

console-exfil-07-learning.png

Observations before judging — default 50

How many responses an endpoint must have served before it can be compared with anything. Until then it is watched and never reported.

Without it, the first request to any new endpoint is infinitely above a baseline that does not exist, and every new path would alarm once.

This same number governs the address signal, where it counts closed windows rather than responses: fifty ten-minute windows is over eight hours of traffic before the first address can be judged. Both counts survive a restart.

Seeing them together

console-exfil-04-analytics.png

Analytics carries a panel counting the findings for the period, with a link into the filtered list. In Security Events, the Attack Type filter offers them together and separately, and the global Action filter offers Observed.

Zero findings on an ordinary day is the correct result. This is a rare signal by design: thresholds that cry wolf get muted, and a muted signal is worse than none because the console still says it is watching.

Proving it works

On a site you operate. Serve two files under one path so they share a baseline — numeric names group to :id:

location /eg2/ {
    alias /var/www/eg2/;
    default_type text/plain;
}

head -c 120     /dev/urandom | base64 | head -c 120     > /var/www/eg2/1
head -c 3000000 /dev/urandom | base64 | head -c 3000000 > /var/www/eg2/2

UA='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120 Safari/537.36'

# 60 small answers build the baseline at 120 bytes
for i in $(seq 60); do curl -ks -A "$UA" https://example.com/eg2/1 >/dev/null; done

# one large answer: 25,000 times that baseline
curl -ks -A "$UA" https://example.com/eg2/2 >/dev/null

A Possible exfiltration — one large response finding appears within seconds.

Send a browser user agent. Bot detection answers curl with a block page, and the baseline would then be built from block pages rather than your files — the test would measure the wrong thing and look like it had failed.

For the address finding, use only the large file so the baseline settles at its size and the per-response tier stays quiet:

for i in $(seq 25); do curl -ks -A "$UA" https://example.com/eg2/2 >/dev/null; done

Twenty-five times three megabytes crosses the 64 MiB floor, and one finding appears — not twenty-five.

Choose sizes against both tiers. "1 MB" written as 1,000,000 bytes is below 1 MiB (1,048,576) and clears neither floor; three megabytes clears both comfortably.

When nothing arrives

curl -s http://127.0.0.1:8080/status | grep -o '"responses_reported":[0-9]*'

If that number is not climbing, nginx is not reporting. The module says why, once per worker, in the error log:

grep "security response feedback" /var/log/nginx/error.log | tail -4
It says What to do
security_response_feedback is off for this location It defaults to on, so it was turned off somewhere in this configuration
no security_tenant_id is in effect The location serving the response has no tenant; an internal redirect wipes the request context, so the tenant has to come from configuration that location can see
could not reach the engine without waiting The engine's socket backlog was full and the report was dropped rather than made the visitor wait

The module logs these at error level on purpose. nginx's default error_log level is error, so a line written at warn — which is less severe — is discarded by a plain error_log /var/log/nginx/error.log;, and the log stays empty however loudly the code speaks.

If responses are arriving but an endpoint is never judged, compare the columns on /status: requests_decided beside counted_since_start. They should move together, with counted a little higher — some requests are answered from the module's own decision cache without asking the engine, and the response is counted anyway. Decided much larger than counted is the fault worth chasing.

What it is not