Run a Focused Security Review for an AI Content Application

Define the application and its assets

The example application lets learners upload documents, ask questions and save summaries. Its assets include original files, extracted text, search indexes, generated answers and account records. Start by drawing the actual data path and naming the services that receive each type of information. Keep the scope small enough to examine concrete operations.

Identify actors separately: an unauthenticated visitor, a learner, another learner, an administrator and background workers. A worker may run without a browser session, but it still needs a verified data scope. Record which boundaries are enforced by the application and which depend on an external service.

Check identity and resource authorization

Authentication establishes who is acting. Authorization decides whether that actor can read or change a particular record. Test every protected operation directly, including download, search, export and job status. A hidden button does not protect an endpoint, and possession of a file identifier does not establish ownership.

For multi-tenant systems, derive membership from trusted records and carry it through background jobs and caches. Use two test accounts with similar document names to expose accidental cross-user or cross-organisation access. Inspect snippets and citations as well as complete document bodies.

Treat uploads and retrieved text as untrusted

Validate upload size and supported types on the server, use application-controlled storage names and bound parsing resources. Inspect how files are served back to the browser. A familiar extension does not prove content is safe, and a successful upload does not mean processing is complete.

Retrieved text can include instructions that conflict with the user’s task. Keep it as evidence, not authority. Restrict available tools and enforce permissions outside the model. A summarization feature should not gain the ability to send private files merely because a document asks it to do so.

Review every action boundary

Minimize sensitive diagnostics

Logs should connect a user-visible reference to the relevant request, tool call or job without storing everything. Prefer structured identifiers, error categories and timings. Exclude credentials and unnecessary private content before they reach logging destinations. Review exception paths and external monitoring integrations, which may capture more than the normal application code.

Assign access and retention for logs. A redacted payload may still contain sensitive metadata, such as a private document title or account relationship. Collect what serves a named diagnostic purpose and keep a process for investigating incidents without broadly sharing the underlying private records.

Verify deletion and recovery

Map deletion from the original file to extracted text, chunks, indexes, cached answers and exports. Track cleanup as a durable operation when several stores are involved. Test with a synthetic document containing a distinctive phrase and confirm that user-facing retrieval no longer exposes it after deletion.

Document the different handling of active data, backups and required audit records according to the product’s actual commitments. Do not promise immediate erasure from every copy unless the system can provide it. For operational recovery, practice restoring the application in an isolated environment and verify that access boundaries remain correct.

Turn findings into owned work

Record each finding with the affected boundary, reproducible safe case, impact and proposed correction. Prioritize by consequence and reachability rather than the length of the report. After a fix, rerun the specific failing case and a representative normal journey. Preserve the evidence without retaining unnecessary sensitive material.

• Protected reads and writes enforce resource-specific authorization.
• Uploads and parsers have explicit limits.
• Untrusted text cannot grant tool authority.
• Caches, jobs and exports preserve user scope.
• Logs and deletion cover derived data paths.
• Findings have owners and verifiable closure criteria.