🧨 Improper Output Handling in LLMs (LLM05): XSS, SSRF, SQL Injection, and RCE Through the Model's Response

Audience: Teams whose applications pass an LLM response onward through the system — into a browser, a SQL query, a system shell, a file path, or an external API — that is, anywhere the model's output becomes input for another component and can cross a trust boundary.

Improper output handling in LLMs is the passing of a model's response to a downstream component (SQL, shell, HTML/browser, file system, API) without validation, sanitization, and escaping, as if that output were trusted. The problem is not that the model "made a mistake," but that its response — essentially untrusted text — flows directly into an execution sink and is interpreted as code or a command. This class is captured in the OWASP Top 10 for LLM 2025 as LLM05:2025 Improper Output Handling. The result: XSS and CSRF in the browser, SSRF and privilege escalation on the backend, SQL injection when a generated query runs without parameterization, path traversal when building file paths, and RCE when the output is executed in a shell. On this page we break down the vectors of improper output handling and show how they are closed by SYNTREX — the defense layer of the Spectorn platform (a set of detection and blocking engines that runs within Spectorn and deploys standalone inside the customer's internal perimeter).

Part of the threat series: OWASP LLM Top 10 · Prompt Injection · Data Leakage · Excessive Agency.

Why LLM Output Cannot Be Treated as Trusted

The core principle of LLM05 can be stated in one sentence: treat any model output as untrusted input — apply the same controls to it as you would to an arbitrary string from an external user. The reason is that a language model is neither a source of truth nor a trusted service: its response is formed from the system prompt, the user's query, RAG documents, and tool output — that is, from a stream into which an attacker could have smuggled an instruction via prompt injection. "The model said so" is not validation and not authorization to execute.

The danger is realized the moment the response crosses a trust boundary and flows into an execution sink. The typical downstream sinks:

Browser / DOM. The model's response is rendered as HTML or Markdown and executed as JS — reflected or stored XSS, CSRF.
SQL engine. The LLM generates a query (Text2SQL) or a fragment of one, and the application runs the string without parameterization — SQL injection, destructive operations such as DELETE without a WHERE.
System shell. The model's output is passed to exec/eval/os.system or executed as generated code in CI/CD — remote code execution (RCE).
File system. The response participates in building a file path — path traversal via ../, reading and writing outside the permitted directory.
Outbound HTTP / external API. A URL is assembled from the model's output, which the backend then requests — SSRF, access to internal services and cloud metadata.

It is important to distinguish LLM05 from overreliance and misinformation: LLM05 is not about the factual accuracy of the response, but about the output crossing a trust boundary and flowing into an execution sink unprocessed. Even a semantically perfect response becomes a vulnerability if the application executes it as trusted code. That is why the control is built not on "fact-checking" but on inspecting the stream for dangerous constructs and on securely encoding the sink itself.

🛑 Vectors of Improper Output Handling and How SYNTREX Closes Them

1. XSS via rendering the model's response in the browser

Risk: The application inserts the model's response into the page as HTML or Markdown and serves it to the browser without escaping. The model (through its own generation or under the influence of an injection) returns <script>, an onerror= event handler, a javascript: link, or a Markdown construct that expands into executable HTML. The browser executes this as code in the context of the user's session — reflected or stored XSS, cookie/token theft, CSRF actions on behalf of the victim. Chat interfaces that render Markdown "as is" are especially dangerous.

OWASP LLM05:2025 Improper Output Handling · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, injection, Shield DMZ.
output_scanner inspects the model's response for executable HTML/JS constructs (the <script> tag, inline event handlers, javascript: schemes, dangerous Markdown) before the response reaches the client, and blocks it or replaces it with a safe placeholder.
injection catches the source instruction in the incoming content that forces the model to generate an XSS payload; the Shield DMZ inspects every model response at the frontier before delivery to the browser.

What SYNTREX honestly does NOT replace: context-aware escaping on the sink side (HTML-encode when outputting to the DOM), Content-Security-Policy, and safe Markdown rendering with an allowlist of tags. SYNTREX intercepts dangerous constructs in the response stream, but the final escaping for the specific output context is the frontend's responsibility. This is defense-in-depth, not a replacement for sink-side encoding.

2. SQL injection via a model-generated query (Text2SQL)

Risk: The LLM translates a natural-language user query into SQL (Text2SQL) or completes a query fragment, and the application executes the resulting string directly, without parameterization. Through an injection in the input, an attacker forces the model to generate a destructive or extractive query: DELETE FROM users without a WHERE, UNION SELECT against other tables, ; DROP TABLE. Because the query is "legitimately" generated by your own assistant, classic WAF filters of user input do not see it — it is born already inside the perimeter.

OWASP LLM05:2025 Improper Output Handling · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, injection.
output_scanner recognizes SQL-injection patterns and dangerous DML/DDL constructs in the model's output (DELETE/UPDATE without a condition, stacked queries, UNION injections, DROP) before the string goes to the DB driver, and blocks the dangerous query.
injection intercepts the instruction in the incoming content that aims Text2SQL at a destructive or extractive query.

What SYNTREX honestly does NOT replace: parameterized queries (prepared statements) and database access under an account with minimal privileges. A correct Text2SQL architecture separates the query structure from the data and never concatenates the model's output into a SQL string. SYNTREX backstops this frontier on the stream, but safe database access must be coded on the sink side.

3. Command injection and RCE via executing the output in a shell or CI/CD

Risk: The model's response is passed to a system call — exec, eval, subprocess, os.system — or executed as generated code. Typical scenarios: a "copilot" assistant generates a shell command that the wrapper runs automatically; an LLM agent writes code that the CI/CD pipeline executes without review; a plugin substitutes the model's output into a command template. Any injection that reaches this output turns into remote arbitrary-code execution on the host — the most severe outcome of LLM05.

OWASP LLM05:2025 Improper Output Handling · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, injection, Shield DMZ.
output_scanner inspects the stream for executable shell constructs and dangerous code patterns (pipes into sh, curl … | bash, command separators ;/&&/$(...), backticks) and blocks the call before it executes on the host.
injection recognizes a command payload injected through input content or tool output; the Shield DMZ blocks or replaces the dangerous response with a safe placeholder before passing it to the executor.

What SYNTREX honestly does NOT replace: a sandbox for code execution (an isolated container with no access to secrets or the network), mandatory human review before running generated code in CI/CD, and the principle that "model output is not executed directly." SYNTREX catches dangerous constructs in transit, but sandboxing and a review gate must exist independently of it.

4. SSRF via a URL assembled from the model's output

Risk: The application builds a URL for an outbound request from the model's response — for example, an agent "go to this link and summarize it," where the address is dictated by the LLM's output, or the backend substitutes a domain from the response into a request template. Through an injection, an attacker substitutes the target with an internal address: http://169.254.169.254/ (cloud metadata), http://localhost:6379 (internal Redis), the address of a service on the internal network. The backend, having network access from inside the perimeter, makes the request — Server-Side Request Forgery, access to services unreachable from outside, and theft of cloud credentials.

OWASP LLM05:2025 Improper Output Handling · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, injection.
output_scanner recognizes outbound URLs with suspicious targets in the model's output (internal and loopback addresses, cloud-metadata endpoints, nonstandard ports of internal services) before the backend makes the request.
output_scanner blocks or redacts such a response in the content, and injection intercepts the instruction that substitutes the request target with an internal address. This is content inspection at the gateway, not a network egress filter — strict egress control is implemented at the network and server level (see below).

What SYNTREX honestly does NOT replace: a server-side allow-list of permitted destination domains, network segmentation, and blocking access to the metadata endpoint at the infrastructure level. SYNTREX recognizes a suspicious target in the stream, but strict control over where the backend can even reach is implemented at the network and server level.

5. Path traversal via building a file path from the response

Risk: The model's output participates in forming a file path — a report name, a template path, a key in a file cache, an attachment name. If the application concatenates the response into a path without normalization, an injection smuggles in ../../../etc/passwd or an absolute path, and the application reads or overwrites files outside the permitted directory. This is both a leak of sensitive files and potential data corruption if the output is used for writing.

OWASP LLM05:2025 Improper Output Handling · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, injection.
output_scanner inspects the output for directory-traversal constructs (../, the encoded variants %2e%2e, absolute and UNC paths) before the string enters a file operation, and blocks the dangerous value.
injection recognizes the instruction in the input content that aims the path construction at stepping outside the working directory.

What SYNTREX honestly does NOT replace: path canonicalization and validation on the sink side (resolving realpath with a check that the result is inside the permitted root) and running under an FS account with minimal privileges. SYNTREX backstops the stream, but safe path handling is coded in the application's own file layer.

6. Leakage of secrets and PII via output into downstream logs and sinks

Risk: The model's response carries sensitive values — a token, an API key, a connection string, PII from the context — and the application passes it onward: writes it to a downstream service's log, sends it to an external API, places it in a queue, renders it in a report. A secret born in the output (for example, quoted from the system prompt or a RAG document), without masking, leaks into a sink whose access is broader than that of the LLM session itself. This is the intersection of LLM05 with data leakage (LLM02): the danger is not the text itself but the fact that it goes into a downstream sink unprocessed.

OWASP LLM05:2025 Improper Output Handling, LLM02:2025 Sensitive Information Disclosure · MITRE ATLAS AML.T0054 (Indirect Prompt Injection).

SYNTREX defense:

Engines/components: output_scanner, secret_scanner, pii, false_completion.
secret_scanner — an always-on invariant: tokens, keys, passwords, and connection strings are masked in the output before it goes to a downstream sink or log; pii masks personal data in redact mode without blocking the entire response.
output_scanner inspects the outbound stream for sensitive constructs before passing it onward; false_completion recognizes an injection-imposed "confirming" completion with which the model legitimizes disclosing a secret.

🛠️ Recommended Configuration

The output-inspection profile prioritizes checking the model's response before passing it to any downstream sink (shell, SQL, browser, FS, outbound URL) plus masking of secrets and PII:

YAML

# syntrex.yaml — model-output inspection profile (LLM05)
version: "1.0"
mode: output_inspection

engines:
  output_scanner:
    action: block                  # inspect the response before passing it to a downstream sink
    detect_command_injection: true # shell / exec / eval / RCE in the output
    detect_sqli: true              # SQL injection, DELETE/UPDATE without WHERE, stacked queries
    detect_xss: true               # <script>, inline handlers, javascript: schemes
    sanitize_html: true            # escape/clean HTML and Markdown before rendering
    detect_ssrf: true              # internal/loopback URLs, metadata endpoints
    detect_path_traversal: true    # ../ and encoded directory-traversal variants
  injection:
    action: block                  # source instruction aiming the dangerous output
    inspect_tool_output: true
    confidence_threshold: 0.80
  secret_scanner: always_on        # tokens/keys/connection strings do not go to the sink
  pii:
    action: redact                 # mask PII without blocking the whole response
    mask_character: "*"
    entities: [card, passport, phone, email, iban, ssn]
  false_completion:
    action: alert                  # imposed "confirming" completion

shield:
  dmz: true                        # Shield DMZ inspects every response, block/placeholder

audit:
  decision_logger: true            # immutable chain (SHA-256/HMAC) of output decisions

🚨 Correlation Rules (SOC)

Insecure output is almost always visible as the pairing "injection in input → a dangerous construct in the response" or as an output_scanner firing right before a call to an execution sink:

JSON

{
  "name": "INJECTION_TO_DANGEROUS_OUTPUT",
  "description": "An injection in incoming content, after which a dangerous construct (SQLi/XSS/shell) appears in the model's response",
  "condition": "sequence(injection[source='external_content' OR source='tool_output', confidence>0.7], output_scanner[match_type IN ('sqli','xss','command_injection'), confidence>0.7], 15s)",
  "severity": "CRITICAL",
  "playbook": "block_response_and_alert_soc"
}

JSON

{
  "name": "OUTPUT_SCANNER_MATCH_BEFORE_EXEC",
  "description": "An output-inspection hit immediately before a call to an execution sink (shell/SQL/HTTP)",
  "condition": "sequence(output_scanner[match=true, match_type IN ('command_injection','sqli','ssrf','path_traversal')], downstream_sink[action='execute'], 5s)",
  "severity": "HIGH",
  "playbook": "quarantine_response_and_review_sink"
}

❓ Frequently Asked Questions (FAQ)

What is improper output handling in LLMs (LLM05)? Improper output handling (OWASP LLM05:2025 Improper Output Handling) is the passing of a model's response to a downstream component (SQL, shell, browser, file system, API) without validation, sanitization, and escaping, as if the output were trusted. Because an LLM response is essentially untrusted text, its flow into an execution sink without processing leads to XSS, SQL injection, SSRF, path traversal, and RCE. SYNTREX closes this with stream inspection: output_scanner recognizes dangerous constructs in the response before passing it to the sink, and injection catches the source instruction.

Can an LLM cause XSS or SQL injection? Yes — not the model itself, but the application that executes its output without processing. If the response is rendered in the browser as HTML/Markdown without escaping, a <script> or javascript: link from the output yields XSS. If model-generated SQL (Text2SQL) is executed without parameterization, an injection such as DELETE FROM users without a WHERE can be smuggled in through the input. output_scanner with the detect_xss and detect_sqli flags recognizes these constructs in the response before it reaches the browser or the DB driver.

How do you defend against SQL injection via a model's response? With two layers. The primary, irreplaceable control is parameterized queries on the sink side: the SQL structure is separated from the data, and the model's output is never concatenated into the query string. On top of this, SYNTREX backstops the stream: output_scanner with detect_sqli recognizes dangerous DML/DDL constructs and injection patterns in the generated query and blocks it before it is sent to the DB driver, while injection catches the instruction that aims Text2SQL at a destructive query. SYNTREX does not replace parameterization — that is the responsibility of the database-handling code.

Why must LLM output be treated as untrusted? Because the model is neither a source of truth nor a trusted service: its response is assembled from the system prompt, user input, RAG documents, and tool output — a stream into which an attacker could have smuggled an instruction via prompt injection. "The model said so" is not validation. The base principle of LLM05: apply the same controls to the output as you would to an arbitrary string from an external user, especially when it crosses a trust boundary and flows into an execution sink.

How is LLM05 different from misinformation and hallucinations? Misinformation and overreliance are about the response's factual inaccuracy: the model confidently states something false. LLM05 is about something else: the output (even if factually correct) crosses a trust boundary and flows into an execution sink unprocessed, becoming code or a command. Here the danger is not in the meaning of the text but in the sink executing it as trusted. That is why defense against LLM05 is stream inspection for dangerous constructs and secure sink encoding, not "fact-checking."

Does SYNTREX replace parameterized queries and escaping? No, and this is an important boundary. SYNTREX is a detection-and-blocking layer on the response stream: it intercepts dangerous constructs (SQLi, XSS, shell, SSRF, path traversal) in transit before they reach the sink. But secure encoding of the sink itself — parameterized queries, context-aware escaping (HTML-encode for the browser, bind parameters for SQL), Content-Security-Policy, a sandbox for code execution — remains mandatory. This is defense-in-depth: SYNTREX catches the dangerous content in the stream, and the sink must still be encoded securely.

📚 Sources

OWASP LLM05:2025 — Improper Output Handling — the primary source for the improper-output-handling class.
OWASP Top 10 for LLM Applications (2025) — the catalog of risks LLM01–LLM10.
MITRE ATLAS — AML.T0054 (Indirect Prompt Injection): poisoned input as the driver of dangerous output.
OWASP — Cross-Site Scripting (XSS) — a reference on XSS as a realization of insecure output in the browser.
OWASP — Injection (SQL and command) — a reference on injection vulnerabilities in execution sinks.
NIST AI Risk Management Framework (AI RMF 1.0) — managing the output-handling risks of AI systems.
NIST AI 600-1 — Generative AI Profile — the generative AI risk profile.