Threat Tribunal

Threat Tribunal · Demonstration Review

Application and AI Security Review

Attackers can manipulate AI, reach another customer's records or personally identifiable information (PII), trigger system actions, misuse uploaded files or exploit unsafe third-party software. This demonstration shows how one security review traces those risks across the whole application.

Audience Technology and security leaders Evidence Code, project files and test output Coverage AI manipulation, safety retesting, access controls, file handling and third-party software Companion Azure deployment review Formats Chaptered · single page for reading or print

Informed by OWASP LLM Top 10 OWASP Top 10 / ASVS NIST AI RMF (GenAI Profile) MITRE ATLAS / CWE Microsoft Zero Trust

Executive Summary

One manipulated AI response can expose trusted data or trigger a system action. Reviewing the AI, web pages, connected systems, background work, cloud setup and third-party software separately can miss the attack paths where they connect.

This review checks how attackers could manipulate the AI, whether safeguards still work after changes, what the source code allows, and whether third-party software introduces risk.

It treats every user message, uploaded document, piece of information pulled from another system, and AI response as potentially unsafe.

Trusted application code must check that content and decide whether the next action is allowed.

Assessment Principle

  • AI provides semantic reasoning. It interprets meaning, classifies information and supports human judgment.
  • Deterministic code keeps control. The application, not the AI, controls identity, permissions, data formats, allowed actions and high-impact approvals.

Scope and Reviewed Technology

The Example Flow

Click or tap a zone or checkpoint on the diagram to jump to the related illustrative security examples.

Sequence diagram of the illustrative document-processing flow: upload and intake, conversion within processing limits, AI analysis of potentially unsafe documents, checks before AI output is used, human review and assembly, and logging, with a security check at each step.
The illustrative document-processing flow with a security check at every step. Fictional and representative; the diagram columns follow the review's four parts of the system. See also the reference architecture in the Azure review.

What We Check in Each Component

Component What we check What could go wrong
Application code
React + TypeScript frontend How users sign in, where access tokens (the digital proof of sign-in) are stored and whether server-only settings reach the browser
  • Browser scripts can read an access token
  • Private server settings are sent to the browser
FastAPI API Who can perform each system action, which fields a request may change and what errors reveal
  • A signed-in user reaches another customer's records
  • Callers change protected fields
  • Errors expose internal details
Background Python worker Whether queued work is genuine, what the worker identity can access and how jobs are isolated
  • Forged or repeated work items
  • One worker identity can reach too much data
Document and AI processing
File uploads and conversion Whether the file is really an allowed type and how much processing one file may consume
  • A hostile file exhausts processing capacity before paid AI services begin
Azure AI Document Intelligence + Azure OpenAI Instructions sent to AI, checks on AI responses and which service addresses the application follows
  • A document manipulates the AI
  • An unchecked AI response becomes a trusted record
  • The worker follows an unsafe address
Software supply chain and delivery
Dependencies (pip-audit, npm audit) Third-party packages with known vulnerabilities, ranked by whether untrusted data can reach them
  • An unsafe package processes customer data
  • Indirect packages change without a locked version
Containers (Trivy) Packages inside each image, its base image and the account it runs under
  • An unsafe base image
  • A container runs with administrator rights
  • A container image name can change
Delivery (Azure DevOps) [planned] How code is scanned, approved, released and rolled back
  • Secrets remain in source history
  • Image references can change
  • Releases lack approval checks

Review Areas

Each review area has its own page with worked, explicitly fictional severity examples.

Relationship to the Azure Review

This review leads with application behaviour and object authorization: what the code allows, which records a caller may use and how untrusted AI or file content is handled. The Azure review proves the companion boundary in deployed configuration, cloud identity, networking and monitoring.

See how the Azure review verifies the deployed cloud boundary.

What You Are Reading

This demonstration shows what a Threat Tribunal engagement delivers and how each chapter uses evidence.

This site contains no client names, identifiers, resource names, source paths, or any combination of details that could reconstruct a client environment.

Cross-reference links lead only to fictional examples created for this demonstration. They never lead to client code, Azure resources or engagement test output.

Limitations

This report explains what the review covers, how it works and what it checks. It does not include client findings, ratings, evidence, fix status or current cloud settings.

It is not a penetration test, certification or assurance opinion. Inclusion of a topic means it was assessed, not that a finding existed.

Public References