DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Our Export button asks for 1,000 audit rows, the route caps it at 100, and the page says all

CogniPrep keeps a GDPR audit log: one append-only Postgres table that records exports, deletions, consent and profile changes, so an account can answer "what has been done with my data". The settings page offers to download it, and the copy reads:

Download a record of all privacy-related actions

The button behind that sentence does this:

// Fetch all activity logs (no pagination limit for export)
const response = await fetch('/api/user/audit-log?limit=1000');
Enter fullscreen mode Exit fullscreen mode

And the route it calls does this:

const limit = Math.min(parseInt(searchParams.get('limit') || '20', 10), 100); // Max 100 per request
const offset = Math.max(parseInt(searchParams.get('offset') || '0', 10), 0);
Enter fullscreen mode Exit fullscreen mode

Math.min(1000, 100) is 100. The request is honoured, the response is a valid 200, and the downloaded file is the most recent 100 rows of a table that grows on every data access. Nothing in the JSON says it was truncated: the endpoint returns total and hasMore, and the export reads only data.logs. An account with 400 rows gets a file called activity-log-2026-10-26.json containing a quarter of its history and the word "all" in the UI above it.

The cap is right. A 100 row page is a sensible ceiling for an endpoint anyone can call in a loop. The bug is that the one caller which genuinely wants everything never learned to paginate, and the comment above it ("no pagination limit for export") records an intention rather than a behaviour.

Nine actions, eight of them ever written

The action set is a TypeScript union, which is the part of this that looked healthiest:

export type AuditAction =
  | 'data_export'
  | 'data_deletion'
  | 'session_deletion'
  | 'account_deletion'
  | 'consent_change'
  | 'data_access'
  | 'preference_change'
  | 'admin_entitlement_change'
  | 'admin_device_reset';
Enter fullscreen mode Exit fullscreen mode

Nine members. There are 13 write sites across 8 files, and they use 8 distinct actions:

action writes
preference_change 4
data_deletion 3
data_export 1
data_access 1
session_deletion 1
account_deletion 1
admin_entitlement_change 1
admin_device_reset 1
consent_change 0

The one action named for consent is the one never recorded. It is not that consent goes unlogged. The endpoint that records a policy acceptance writes this:

await logAuditAction({
  userId: user!.id,
  action: 'preference_change',
  resourceType: 'account',
  details: {
    field: 'policy_acceptance',
    tos_version: CURRENT_TOS_VERSION,
    privacy_policy_version: CURRENT_PRIVACY_VERSION,
  },
  ...
});
Enter fullscreen mode Exit fullscreen mode

So the fact is in the table, as a string inside a jsonb column, reachable only by details->>'field' = 'policy_acceptance'. The typed column that exists to make exactly this query cheap is set to the wrong value, and the union member that would have made the mistake obvious was added and then not used.

The resource types went the same way. Nine declared, seven written, and the two that never appear are 'consent' and 'preferences', which is to say the same event twice over.

This is what a union buys you and what it does not. It stops a typo: 'consnet_change' is a compile error. It cannot tell you that a correct member is the wrong choice at a call site, and it will never report that a member has no call sites at all. Three lines of grep do the second job, and until I ran them the table looked fully exercised because every row in it was valid.

The index comment that a query disagrees with

The table has one index, and a long note explaining why:

// Every read of this table has the same shape:
//   WHERE user_id = $1 ORDER BY created_at DESC LIMIT n
// ...
// Dropped alongside this: audit_logs_action_idx and audit_logs_created_at_idx.
// No query filters on `action` or on `created_at` alone ...
index('audit_logs_user_created_idx').on(table.user_id, table.created_at),
Enter fullscreen mode Exit fullscreen mode

"No query filters on action" is false. One does, in the module that decides whether to ask a customer for a public review:

const rows = await db
  .select({ id: auditLogsTable.id })
  .from(auditLogsTable)
  .where(
    and(
      eq(auditLogsTable.user_id, userId),
      eq(auditLogsTable.action, 'data_deletion'),
      gte(auditLogsTable.created_at, cutoff)
    )
  )
  .limit(1);
Enter fullscreen mode Exit fullscreen mode

Somebody who has just asked us to erase their data should not then be asked to rate us, and the audit log is the only thing we are still allowed to know about them, so the suppression reads the log rather than keeping a tombstone of its own.

The decision the comment defends is still correct. That predicate is served by the composite index through its leading user_id column, the row count per user is tiny, and filtering a handful of rows on action in memory costs nothing. The index that was dropped would not have helped. What is wrong is only the sentence, and a wrong sentence in a schema is worse than no sentence, because the next person to add an action filter will read it and believe the field is unindexed on purpose.

The ascending column order in the same note, on the other hand, is worth stealing. Declaring created_at descending in the index produces DESC NULLS LAST, which does not match ORDER BY created_at DESC (Postgres defaults that to NULLS FIRST), so the planner stops using the index for the sort. A btree scans backwards perfectly well. Leave it ascending.

What the log keeps and the download does not

Every write captures the request's IP and user agent:

ipAddress: request.headers.get('x-forwarded-for') || request.headers.get('x-real-ip') || undefined,
userAgent: request.headers.get('user-agent') || undefined,
Enter fullscreen mode Exit fullscreen mode

The response maps five fields and neither of those is among them: id, action, resourceType, details, createdAt. That is the right default for a file a user may forward to anyone, and it is also a gap in a transparency feature, because the IP we hold is itself personal data. Our privacy policy already tells readers we keep it. Load cogniprep.app/privacy and search for "audit" and you get one hit:

IP address (used for security and audit logging, and for rate limiting on sensitive endpoints such as sign-in, checkout and AI processing)

So the policy names the field, the table holds it, and the export of "all privacy-related actions" omits it. One of those three has to move.

Two smaller things in the same file. logAuditAction swallows its own failure and only prints in development, which is deliberate (a failed log must not fail a deletion) and means a broken insert is invisible in production. And the route is declared { rateLimit: 'write' } while being a GET, because the preset is about cost rather than about HTTP verbs: it is a pagination query anyone can hammer.

See it

The endpoint is authenticated, and you can watch it say so:

curl -s -o /dev/null -w "%{http_code} %{size_download}\n" \
  "https://cogniprep.app/api/user/audit-log?limit=1000"
Enter fullscreen mode Exit fullscreen mode

That is 401 24, the 24 bytes being {"error":"Unauthorized"}. Note the cache-control on that response: public, max-age=0, must-revalidate, not the private, max-age=20, stale-while-revalidate=40 the handler sets for an authorised read, because the wrapper's error path never reaches the handler's headers.

Then read the policy the table exists to satisfy at cogniprep.app/privacy, which lists what we collect field by field, including the one the download leaves out. The product the log belongs to is at cogniprep.app: practice for 56 assessment providers, where the settings page now owes its users a paginated loop and one honest word instead of "all".

Top comments (2)

Collapse
 
mercerk17 profile image
Kai Mercer •

The cap is fine, and your point that the response already carries total and hasMore is the useful part. A test that would have caught this: seed 101 rows, run the export, and assert the file's row count equals total. The sizes worth covering are 100, 101 and an empty log, since off-by-one is where paging code usually breaks. A second cheap guard is to make the export loop on hasMore and fail loudly if it stops with hasMore still true, so a future cap change cannot silently shorten the file again.

Collapse
 
williamsj04 profile image
Jessica Williams •

The grep for union members with no call sites is a neat check. On the export fix, plain offset paging has one more trap on a table that grows with every data access: if new rows are written while the loop runs, an offset-based walk can repeat or skip rows between pages. Keyset paging on (created_at, id) avoids that, and it lines up with the composite index you describe because the planner can read it in order. If the file needs to be a consistent picture, running the whole export inside one REPEATABLE READ transaction gives every page the same snapshot, since PostgreSQL's docs say that level takes its snapshot at the first statement. It is worth stamping the file with the snapshot time as well, so a reader knows what it covers.