Free CCAR-F Practice Test Questions 2026

Explanation:
The pipeline crashed mid-processing after completing work on 12 of 28 documents, with partial progress from multiple agents. The solution must preserve the fidelity of completed work while efficiently loading only the necessary state for resumption. A structured, coordinated state management approach ensures completeness without redundant processing.

Correct Option:

A. Have each agent persist a structured export to a known location. On resumption, the coordinator loads the manifest and injects relevant state into agent prompts.
This is the most effective approach because it combines fidelity with efficiency. Each agent exports a structured, complete representation of its work (findings, extractions, patterns identified) to a known location. The coordinator maintains a manifest tracking which documents and tasks are complete. On resumption, the coordinator loads the manifest, determines what work remains, and injects only the relevant completed state into agent prompts. This avoids repeating work (the 12 processed documents) while ensuring complete fidelity through structured data rather than lossy summaries.

Incorrect Options:

B. Have each agent maintain its own persistent state file and reload it independently at the beginning of each session.
This approach lacks coordination and could lead to inconsistency or duplication. Without a central manifest, agents may not know which documents were already processed by other agents, potentially causing redundant work or gaps. Independent state management also makes it difficult to ensure cross-agent consistency, as each agent's view of "completed" may differ.

C. Persist the coordinator's conversation log containing all task delegations and responses, and provide this log to the agents when resuming.
The conversation log is a lossy, narrative representation that may not capture all structured details needed to resume accurately. Agents would need to parse the log to reconstruct state, which is error-prone and inefficient. This approach also lacks the structured data needed for precise resumption—the log may contain instructions and summaries but not the complete findings needed to avoid reprocessing.

D. Index all agent outputs in a shared vector store. When resuming, have each agent query the store using semantic search to retrieve relevant prior findings.
Vector stores are designed for semantic similarity retrieval, not precise state restoration. Semantic search may retrieve incomplete or irrelevant results, potentially missing critical prior findings. This approach also introduces retrieval uncertainty and complexity, as agents would need to formulate effective queries to locate their own prior work. Structured exports with clear manifests are far more reliable for deterministic state restoration.

Reference:

Anthropic Claude Agent SDK Documentation – State Management – Recommends structured state persistence with manifests to enable efficient resumption after failures.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of data persistence, audit trails, and recovery mechanisms in automated systems to ensure reliability and completeness.

Distributed Systems Best Practices – Checkpointing – Highlights that structured checkpoints with manifests are more reliable than log-based or retrieval-based state restoration for resuming work after failures.

Explanation:
The managed Code Review through the Claude GitHub App automatically runs on every pull request, producing 18 findings per review, of which only ~4 (22%) are genuine logic bugs. The noise comes from three specific categories: style/linting (already covered by CI), generated code (src/gen/*), and intentional rendering patterns. The solution must reduce noise from these categories while preserving detection of real issues, using mechanisms supported by the Claude GitHub App.

Correct Option:

A. Create a REVIEW.md file at the repository root containing skip rules for CI-enforced checks and generated files, together with a verification requirement that rendering-related findings cite a specific line demonstrating incorrect behavior.
The Claude GitHub App supports REVIEW.md (or similar configuration files) for customizing review behavior on a per-repository basis. This approach is the most effective because it leverages the app's native configuration mechanisms to address all three noise categories directly. Skipping CI-enforced checks prevents duplicate findings; ignoring src/gen/* prevents noise from auto-generated code; and requiring verification evidence for rendering-related findings ensures that only legitimate issues (where the pattern actually causes incorrect behavior) are reported, not merely flagged for resembling anti-patterns.

Incorrect Options:

B. Configure separate GitHub Actions workflow files for each code area: one for generated code with findings suppressed, one for rendering code with custom instructions, and one general workflow for everything else.
This adds unnecessary complexity and does not align with how the Claude GitHub App operates. The managed Code Review runs as a single GitHub App integration, not as separate Actions workflows per code area. This approach also fragments the review process, making it harder to manage and risking inconsistent coverage.

C. Add custom review instructions to a GitHub Actions workflow file, using the action's prompt parameter to suppress duplicate lint findings, ignore generated template code, and impose stricter evidence requirements on rendering-related issues.
This option misidentifies the integration mechanism. The Claude GitHub App is a managed GitHub App, not a GitHub Action. It does not have a "prompt parameter" in a workflow file. While GitHub Actions can be used to invoke Claude's API, the managed Code Review provided through the GitHub App is configured via repository files (like REVIEW.md), not via workflows.

D. Add detailed explanations to the project's CLAUDE.md describing intentional patterns, stating that CI handles linting, and identifying src/gen/ as automatically generated code.
CLAUDE.md provides persistent context for Claude Code CLI and general agent interactions, but the Claude GitHub App's managed Code Review uses a separate configuration mechanism (REVIEW.md). While CLAUDE.md may influence some Claude interactions, the GitHub App's behavior is controlled through its dedicated configuration. This option would not affect the managed Code Review's findings.

Reference:

Anthropic Claude GitHub App Documentation – REVIEW.md Configuration – Specifies that REVIEW.md (or similar repository configuration files) control the managed Code Review's behavior, including skip rules and custom instructions.

Anthropic Claude GitHub App – Managed Code Review – Explains that the GitHub App runs automatically on PRs and can be configured via repository files to reduce noise and focus on actionable findings.

CI/CD Best Practices – Emphasizes that automated code review tools should complement, not duplicate, existing linting and formatting checks, and should use configuration files to suppress known noise sources.

Explanation:
The conservative prompt instructions ("only report high-confidence," "err on the side of not commenting") are suppressing genuine bug detection to achieve low noise. The solution must separate the detection (comprehensive coverage) and filtering (thresholding) stages so that the model is not forced to self-censor during discovery, while still producing a manageable set of findings.

Correct Option:

C. Split the review into a finding stage whose objective is comprehensive coverage—reporting every potential issue with confidence and severity metadata—and a separate stage that verifies and thresholds those findings.
This is the most effective approach because it resolves the core tension between recall and precision by decoupling detection from filtering. The finding stage operates without conservative constraints, enabling Claude to report all potential issues including the race condition that was previously suppressed. Each finding includes confidence and severity metadata. The separate verification stage then applies thresholds, deduplication, and historical noise filtering to produce a manageable set of high-quality findings. This two-stage design substantially improves recall while keeping false positives manageable through programmatic control.

Incorrect Options:

A. Add detailed few-shot examples demonstrating bug categories Claude should flag—race conditions, null dereferences, and error-handling gaps—while retaining the high-confidence filtering instruction.
The high-confidence filtering instruction is the root cause of the low recall. Adding examples of bug categories will not override the instruction to suppress uncertain findings—the model will still avoid reporting anything it deems below the confidence threshold. The race condition that caused the outage may have been subtle enough that Claude was not "certain" about it, leading to suppression.

B. Remove the conservative instructions and have Claude report every potential issue, then apply a programmatic filter that deduplicates findings and suppresses historically noisy categories.
While removing the conservative instructions is necessary, reporting every potential issue would likely produce overwhelming noise. Programmatic filtering alone cannot effectively separate genuine bugs from trivial concerns without the structured metadata (confidence, severity) that a dedicated verification stage can provide. This approach also misses the opportunity to use a second model pass for intelligent verification, which is more nuanced than simple filtering.

D. Expand the context to include related tests, recent Git history, and the module's dependency graph so Claude has richer evidence for judging severity.
While richer context may help the model make better judgments, it does not address the core issue—the model is being instructed to suppress all but the most certain findings. Even with perfect context, the high-confidence filter would still suppress the race condition if the model wasn't "certain" about it. Context expansion is a complementary improvement, not a solution to the suppression problem.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends separating detection and filtering stages to avoid self-censorship during discovery while maintaining output quality.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of comprehensive detection and subsequent validation rather than conservative self-censorship in automated systems.

Software Engineering – Code Review Best Practices – Highlights that effective review processes separate the detection of potential issues from the triage/filtering phase to ensure bugs are not missed due to premature judgment.

Explanation:
The engineer has a specific, distinctive error message string ("SYNC_CONFLICT: entity version mismatch") that uniquely identifies the error. The most efficient approach is to search the codebase for this exact text to locate where it is defined or raised, then read the relevant files to understand the context. This targeted approach minimizes scanning and reading.

Correct Option:

A. Use Grep to search for distinctive text from the error message, such as SYNC_CONFLICT or entity version mismatch, and then read the matching files to understand the context.
This is the most efficient approach because the error message contains distinctive, unique text that is unlikely to appear elsewhere in the codebase. Grep can quickly scan all files and return the exact locations where this error is defined or raised. This directly answers the engineer's question ("which service generates it?") in minimal time. Reading the matching files then provides the necessary context about the error's origin and conditions.

Incorrect Options:

B. Use Glob to find files in directories commonly associated with error handling, such as errors, exceptions, or handlers, and then read every matching file. This is inefficient because it relies on naming conventions that may not be consistent across 12 services. The error definition could be in a file named something unexpected. Reading every file in error-related directories across all services would waste significant time and tokens, especially if many directories are empty or contain unrelated errors.

C. Read the project's README and service-configuration files, and then systematically read source files in every service directory.
This is extremely inefficient. The engineer would need to read through large amounts of unrelated documentation and configuration before even starting to search source code. Systematically reading every service directory would be time-consuming and would include many files that do not contain the error definition. This approach does not leverage the unique identifier provided by the error message.

D. Use Grep to locate every file importing the project's error-handling module, and then read those files to find custom error definitions.
This is more focused than options B or C, but it still makes an assumption that the error is defined using a central error-handling module. The error could be defined inline, use a different module, or be raised by a third-party library. The direct Grep for the error message text is more reliable because it targets the exact error string rather than indirect patterns.

Reference:

Anthropic Claude Agent SDK Documentation – Tool Usage Best Practices – Recommends starting with Grep for specific, distinctive text patterns to locate code efficiently.

Software Engineering – Codebase Navigation – Highlights that searching for unique strings (error messages, function names, constants) is the most efficient way to locate code responsible for specific behaviors.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of efficient and accurate root-cause analysis in automated systems.

Explanation:
The most reliable way to ensure structured output that strictly conforms to a JSON schema is to use Claude's native tool_use capability with a defined tool input schema. This approach leverages the API's built-in validation to enforce the schema at the structural level, eliminating manual parsing and reducing the risk of malformed or inconsistent outputs.

Correct Option:

D. Define a tool with an input schema matching your required JSON structure and extract the data from Claude's tool_use response.
This is the most reliable approach because it uses Claude's native structured output mechanism. By defining a tool with an input schema that exactly matches your required JSON structure (name, contact details, skills, work experience, education), you offload schema enforcement to the API. The model is forced to produce valid JSON matching the schema to successfully call the tool. This eliminates the need for manual parsing, regex extraction, or additional formatting steps, ensuring consistency and reducing the risk of downstream validation failures.

Incorrect Options:

A. Parse Claude's text response with regex patterns to extract JSON objects, using retry logic for malformed responses.
Regex parsing is brittle and error-prone, as it may fail if the response format deviates slightly. Retry logic increases latency and costs without guaranteeing a valid response. This approach does not leverage any built-in validation mechanism and relies on post-hoc fixes for malformed outputs, which is unreliable for production systems.

B. Include detailed JSON formatting instructions and a template example in the system prompt, asking Claude to output only valid JSON.
While this may work in many cases, it relies entirely on the model following instructions without any structural enforcement. The model may occasionally omit required fields, include extra fields, or produce malformed JSON, especially for complex schemas. This approach provides guidance but no guarantee, making it less reliable than tool-based schema enforcement.

C. Make two separate API calls—first extracting information as text, then asking Claude to format that text as JSON.
This approach doubles API costs and latency while still relying on instructions rather than structural enforcement. The second call could introduce formatting errors not present in the first extraction, and each call carries its own risk of inconsistency. This is inefficient and less reliable than a single tool-based call.

Reference:

Anthropic API Documentation – Tool Use – Recommends using tools with defined input schemas for reliable structured output extraction, as the API enforces schema compliance.

Anthropic Prompt Engineering Best Practices – Structured Outputs – Highlights that tool_use is the most reliable method for ensuring JSON schema compliance compared to prompting for JSON in text responses.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of validation and structured outputs in automated systems to ensure reliability and downstream integration.

Explanation:
With limited reviewer capacity (only 5% of total volume), the review system must prioritize extractions that are most likely to contain errors or require human judgment. Routing based on model confidence and document ambiguity targets the highest-risk cases, ensuring that limited reviewer capacity is used where it adds the most value.

Correct Option:

A. Route extractions where the model indicates low confidence or where source documents contain ambiguous or contradictory information.
This is the most effective basis for selection because it targets the cases that are most likely to contain errors. Model confidence scores (especially when calibrated) correlate with extraction accuracy—low-confidence outputs are more likely to be incorrect. Similarly, ambiguous or contradictory source documents are inherently harder for automated extraction and more likely to benefit from human judgment. This approach maximizes the value of limited reviewer capacity by focusing on the highest-risk cases, while allowing high-confidence, unambiguous extractions to proceed automatically.

Incorrect Options:

B. Route extractions containing specific high-priority entity types (e.g., financial figures, dates) for human review, regardless of extraction confidence.
While high-priority entities may deserve scrutiny, routing them all for review would likely exceed the 5% reviewer capacity, as financial figures and dates appear in many documents. This approach also wastes reviewer attention on high-confidence, straightforward extractions of these entities, reducing capacity for truly ambiguous cases. A confidence-based approach is more efficient.

C. Route extractions for review only when downstream systems report data quality issues or processing failures.
This is reactive rather than proactive—issues are only caught after they've already caused downstream failures. By the time a downstream system reports a problem, the incorrect data may have already affected decisions, reports, or integrations. This approach also provides no opportunity to catch issues before they propagate, increasing operational risk.

D. Randomly sample 5% of extractions for review.
Random sampling provides broad but shallow coverage. It will review many correctly extracted, high-confidence cases while missing most of the truly problematic extractions. This is inefficient use of limited reviewer capacity, as it does not target the cases most likely to contain errors. This approach has low return on review effort.

Reference:

Anthropic Prompt Engineering Best Practices – Confidence Scores – Recommends using confidence scores and ambiguity indicators to triage outputs for human review, focusing capacity on highest-risk cases.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes that human review should be targeted based on model uncertainty, data ambiguity, and materiality to maximize oversight value.

NIST AI Risk Management Framework – Highlights that risk-based triage of model outputs (focusing on high-risk, low-confidence cases) is more effective than random sampling or reactive monitoring.

Explanation:
The agent is losing accuracy due to context pressure—8 source files plus associated test files and flow tracing exceed the available context window, causing the model to forget earlier patterns. The solution must distribute the investigative work across specialized subagents, each with a narrower focus, while the main agent coordinates and preserves high-level understanding.

Correct Option:

A. Spawn subagents to investigate specific questions (e.g., "find all test files for payment processing," "trace refund flow dependencies") while the main agent coordinates findings and preserves high-level understanding.
This is the most effective approach because it distributes the investigative workload across multiple specialized subagents, each with a narrower context requirement. The main agent can coordinate efforts, synthesize findings, and maintain the overall investigation strategy without being overwhelmed by 45 files worth of code. Subagents can each read and analyze specific portions (test files, refund flow, etc.) without context overload, and return structured findings to the coordinator. This approach preserves accuracy and scalability while completing the investigation efficiently.

Incorrect Options:

B. Clear context with /clear, then selectively re-read only the most critical files discovered so far, writing key findings to a scratchpad file that persists between context resets.
Clearing context discards valuable findings and understanding already built, forcing the agent to redo work. Relying on a scratchpad file for state introduces complexity and still requires re-reading files. This approach loses the cumulative understanding the agent has developed and risks omitting important patterns that were captured earlier.

C. Switch to using Grep to search for specific function names instead of reading full files, reducing the content loaded into context for remaining exploration.
While Grep can reduce context usage, it sacrifices depth of understanding—the agent would need to read the actual code to understand logic, dependencies, and potential untested paths. Grep searches are complementary to reading, not a replacement. This approach would likely yield incomplete or inaccurate analysis of untested code paths.

D. Document all current findings in a summary report, clear context completely, then use that report as the sole reference for continuing the investigation.
Summarizing all findings risks losing critical details and patterns discovered earlier. The summary would be lossy, and the agent would need to work from incomplete information. This approach also prevents the agent from recalling specific code patterns, variable names, or nuanced relationships that are essential for a thorough investigation.

Reference:

Anthropic Claude Agent SDK Documentation – Subagent Coordination – Recommends spawning subagents for large-scale investigations to distribute workload and manage context constraints.

Anthropic Claude Code – Context Management – Highlights that subagents are more effective than context clearing or scratchpads for maintaining accuracy in large codebase explorations.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of robust investigation methods that maintain accuracy and completeness in complex systems.

Explanation:
The root cause is that a single static prompt with only changed files cannot identify cross-file impacts because it lacks visibility into calling code. The solution must enable the review process to dynamically explore the codebase, following references to verify how changes affect other parts of the system. This transforms the review from static analysis into adaptive exploration.

Correct Option:

A. Redesign the review as a turn-limited agentic task in which the model can read files and search the codebase through tools, following references to verify cross-file findings.
This is the most effective approach because it enables dynamic, adaptive exploration of the codebase. The agentic task can use Grep to find callers of changed functions, Read to inspect their usage, and trace dependencies to verify compatibility. This directly addresses the 35% cross-file bug rate by allowing the model to follow the dependency graph naturally, reading only files that are actually relevant to the specific changes. Unlike static prompts, the agent can adapt its search based on what it discovers, ensuring comprehensive cross-file analysis without overwhelming the context window.

Incorrect Options:

B. Add chain-of-thought instructions asking the model to list all external references in the diff and then reason step by step about how each change might affect callers in other files.
Instructions alone cannot overcome the fundamental limitation of not having the actual code to analyze. The model can only guess about potential callers based on naming patterns or conventions, which is unreliable and prone to false positives or missed issues. Without the ability to read actual caller code, the model cannot verify compatibility.

C. Use static analysis to build a dependency graph of changed code, and then expand the prompt to include every file within two dependency hops of any changed file.
This risks including too many files (exponential growth), overwhelming the context window and increasing costs. A two-hop dependency graph for a core utility function could include hundreds of files. It also lacks precision—many included files may be irrelevant, while some critical callers might be more than two hops away. Static inclusion cannot adapt to the specific nature of each change.

D. Run parallel review passes for each changed file with its direct dependents included, and then aggregate and deduplicate the findings through a final summarization call.
This is inefficient and still incomplete—including only direct dependents may miss transitive callers or indirect usages. Running multiple passes increases token usage and latency significantly. The aggregation step also adds complexity and potential for lost or duplicated findings. An agentic approach is more elegant and efficient because it reads only what's needed based on the specific changes.

Reference:

Anthropic Claude Agent SDK Documentation – Agentic Code Review Patterns – Recommends using agentic tasks with tool access for reviews that require cross-file analysis, enabling dynamic exploration of the codebase.

Software Engineering Code Review Best Practices – Highlights that cross-file impacts require understanding of call graphs and usage patterns, which is best achieved through interactive exploration rather than static analysis.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of comprehensive testing and validation that accounts for interactions across system components.

Explanation:
The agent occasionally violates the $500 refund escalation policy despite clear prompt instructions, indicating that relying solely on model instructions is insufficient for regulatory compliance. The solution must enforce the rule at the system/backend level rather than depending on the model's discretion. A pre-execution hook provides guaranteed enforcement.

Correct Option:

**D. Implement a hook to intercept tool calls, when the refund process amount exceeds $500, block it and invoke human escalation.**
This is the most effective approach because it enforces the compliance rule at the infrastructure level, not through model instructions. By intercepting tool calls to process_refund, you can inspect the amount parameter before the tool executes. If the amount exceeds $500, the hook blocks the call and triggers human escalation. This eliminates the 3% failure rate entirely because the enforcement is deterministic and not subject to model judgment or prompt adherence issues. This approach provides guaranteed compliance.

Incorrect Options:

A. Add few-shot examples to the prompt showing correct escalation behavior at various refund amounts ($400, $500, $600).
Few-shot examples may improve instruction following but cannot guarantee compliance. The model can still fail to apply the rule consistently, especially in ambiguous contexts or edge cases. The 3% failure rate shows that even with clear instructions, model discretion is not reliable for regulatory compliance.

B. Strengthen the system prompt with emphatic language: "CRITICAL POLICY: Refunds over $500 MUST trigger human escalation. NEVER process these directly."
While stronger language may help, it still relies on model judgment. The 3% failure rate demonstrates that even explicit system prompt instructions can be violated. Without structural enforcement, there is no guarantee of compliance, regardless of how emphatic the prompt language is.

C. Modify the refund tool to return an error with message "Amount exceeds policy limit—please escalate" when the threshold is exceeded.
This would prevent the refund from being processed but would not automatically trigger escalation. The agent would need to handle the error and then call escalate_to_human. This still relies on the model's behavior to correctly respond to the error message, leaving room for compliance gaps. A hook that both blocks the call and triggers escalation provides guaranteed compliance.

Reference:

Anthropic Claude Agent SDK Documentation – Tool Call Interception – Recommends using pre-execution hooks for policy enforcement that must be guaranteed, such as compliance rules and authorization checks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes that compliance and regulatory rules should be enforced through deterministic controls rather than relying solely on model judgment.

Financial Services Compliance Best Practices – Highlights that monetary thresholds and regulatory rules require structural enforcement through system-level controls, not model instructions.

Explanation:
The agent sometimes exhausts its turn budget after gathering data but before completing resolution or escalating. The goal is to guarantee every interaction ends with either resolution or human handoff. The solution must handle the termination case from outside the agent loop to ensure reliable escalation.

Correct Option:

C. Add orchestration-layer code that checks the agent's outcome after each loop termination—if the loop ended without a completed resolution or escalation, programmatically call escalate_to_human with the accumulated conversation context and tool results.
This approach guarantees that every interaction ends with either a completed resolution or a human handoff. It works from outside the agent loop, catching all termination cases—including budget exhaustion, errors, and unexpected loop exits. The orchestration code has access to the accumulated conversation context and tool results, enabling a rich handoff to the human agent. This is the most reliable approach because it does not rely on the model to correctly handle the termination case, providing a deterministic safety net.

Incorrect Options:

A. Implement a pre-tool-use hook that counts tool invocations and terminates the loop with an automatic escalation once the agent reaches 80% of its max_turns limit.
This approach escalates before the limit is reached, but it doesn't guarantee resolution completion if the agent legitimately needs more turns. A hard escalation at 80% may interrupt the agent just before it would have reached a resolution, degrading first-contact resolution rates. More importantly, it does not guarantee that the final outcome is always resolution or escalation—just that the loop is terminated early.

B. Split the workflow into two sequential agent invocations—a first agent gathers information via get_customer and lookup_order, then a second agent receives that data and handles process_refund or escalate_to_human, each with separate turn budgets.
This avoids the turn limit issue for each agent but does not guarantee the final outcome. The second agent could also exhaust its turn budget without resolution or escalation. Additionally, splitting the workflow increases complexity and latency, and may reduce the agent's flexibility to adapt based on information discovered during resolution.

D. Add system prompt instructions telling the agent to call escalate_to_human with a summary of its findings whenever it determines it cannot complete resolution within its remaining actions.
This still relies on the model's judgment and ability to correctly handle the situation. If the agent unexpectedly exhausts its turns before realizing it cannot complete resolution, it may terminate without escalating. Prompt instructions are not a guarantee—the 3% failure rate in the previous question showed that even explicit instructions can be violated.

Reference:

Anthropic Claude Agent SDK Documentation – Orchestration Patterns – Recommends using orchestration-layer code to handle termination cases and ensure consistent outcomes, rather than relying solely on model behavior.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of deterministic controls and safety nets in automated systems to guarantee compliance and outcome completeness.

Customer Support Automation Best Practices – Highlights that automated support systems should have escalation guarantees to ensure no customer interaction ends without resolution or human handoff.

Explanation:
The extraction system faces conflicting values in 15% of documents, but historical analysis shows the detailed specifications table is correct 90% of the time. The solution must handle this common edge case efficiently by giving the model clear precedence rules, enabling it to extract the correct single value and preserve the existing schema.

Correct Option:

C. Add extraction instructions specifying that values from the detailed specifications table take precedence when conflicting values exist, while retaining the single-value schema.
This is the most effective approach because it addresses the inconsistency at the extraction stage using the known accuracy pattern (detailed table is correct 90% of the time). The model can apply the precedence rule during extraction, selecting the correct value without changing the schema or requiring downstream handling. This preserves the single-value field, avoids manual reviews for most cases, and solves the problem at the source without additional downstream complexity.

Incorrect Options:

A. Change the field to an array that captures every discovered value and its source location, leaving downstream systems to apply precedence rules.
This shifts the burden of handling conflicts to downstream systems, which may not be designed to handle multiple values or apply precedence rules. This creates complexity and risk, especially if multiple downstream systems need to implement the same logic consistently. The schema change would also break existing downstream integrations.

B. Reject every extraction containing conflicting values and require the source document to be corrected before processing continues.
This is impractical and significantly increases operational friction. A 15% rejection rate would block a large volume of documents and require manual corrections for each. This approach is inefficient and delays downstream processing unnecessarily, especially when historical analysis shows a clear pattern about which source is usually correct.

D. Add a conflict_detected Boolean field and route every affected document for manual review.
This would route 15% of documents to human review, exceeding the 5% reviewer capacity established earlier. Manual review is an appropriate fallback but should be used when ambiguity is unresolvable by rules. Here, there is a clear rule (detailed table takes precedence) that can be applied automatically, making manual review unnecessary for most conflicting cases.

Reference:

Anthropic Prompt Engineering Best Practices – Handling Ambiguity – Recommends providing explicit precedence rules in extraction instructions when sources are known to have varying reliability or consistency.

Data Quality Management Best Practices – Highlights that when document sources have known reliability patterns, extraction should apply those rules automatically rather than passing the burden to downstream systems or human reviewers.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of using historical analysis and known data quality patterns to inform automated decision-making in extraction systems.

Explanation:
The agentic loop dynamically determines the next action based on the conversation context, including the order details returned from lookup_order. The model reasons about the situation (purchase date, product type, customer context, and applicable policies) and decides whether the refund can be processed or requires human escalation.

Correct Option:

A. The order details are added to the conversation and the model reasons about which action to take.
This is the core of the agentic loop. When lookup_order returns data, the tool result is added to the conversation history, and the model uses its reasoning capabilities to determine the next appropriate action based on the available information. The model evaluates factors like the 45-day purchase date, return policy, customer history, and other relevant context, then selects either process_refund (if eligible) or escalate_to_human (if the case requires it). This dynamic reasoning is what makes the agent adaptable to high-ambiguity requests.

Incorrect Options:

B. The orchestration layer automatically routes to the next tool based on the order's status field.
This would be a deterministic, rule-based approach, not an agentic one. The agentic SDK is designed for model-driven reasoning, not hardcoded routing rules. Relying on the orchestration layer to route based on a single field would make the system brittle and unable to handle the nuance and complexity of high-ambiguity requests.

C. The agent follows a pre-configured decision tree mapping order attributes to specific tool calls.
A decision tree is a deterministic, non-agentic approach. It would require predefining every possible path based on order attributes, which is inflexible and cannot handle the varied, ambiguous nature of real-world customer support requests. This approach would defeat the purpose of using an agentic system designed for high-ambiguity scenarios.

D. The agent executes the remaining steps in a tool sequence planned at the start of the request.
This would be a fixed, pre-planned workflow—again, not agentic. The agentic loop does not pre-plan a sequence of tools; it adapts based on the information it receives at each step. Planning all steps upfront would make the system inflexible and unable to respond to unexpected information or changing circumstances during the interaction.

Reference:

Anthropic Claude Agent SDK Documentation – Agentic Loop – Explains that the agent iteratively reasons about the current context and available tools to determine the next action, rather than following pre-planned sequences or decision trees.

Anthropic Claude Agent SDK – Tool Calling – Highlights that tool results are added to the conversation history, and the model uses this context to make informed decisions about subsequent actions.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of adaptive reasoning and dynamic decision-making in automated systems handling complex, high-ambiguity scenarios.