Free CCAR-F Practice Test Questions 2026

Explanation:
The agent prefers Grep over analyze_dependencies because the tool descriptions are ambiguous and overlapping. Grep's description explicitly mentions "search" and "patterns," which strongly matches the user's query about "dependencies." The MCP tool's sparse description ("Analyzes dependency graph") does not clearly communicate its superior capability for dependency analysis, leading the agent to default to the more familiar Grep tool.

Correct Option:

B. Expand MCP tool descriptions to detail capabilities and outputs—e.g., "Builds dependency graph showing direct imports, transitive dependencies, and cycles."
This is the most effective approach because it directly addresses the root cause: inadequate tool descriptions. By expanding the description to explicitly detail what the tool does (direct imports, transitive dependencies, cycles) and what it outputs, the agent can better understand when this tool is more appropriate than Grep. The enhanced description makes the tool's specialized value clear, enabling the agent to select the right tool for the right task without requiring complex routing logic or removing useful tools.

Incorrect Options:

A. Add routing instructions to the system prompt specifying that dependency-related questions should use MCP tools rather than Grep.
While routing instructions could help, they are brittle and require explicit mapping of every possible query pattern to tool selections. This approach doesn't scale as the number of tools grows and fails when users phrase queries differently than anticipated. Better tool descriptions are more robust because they enable the model to make intelligent selections based on understanding rather than hardcoded rules.

C. Remove Grep from available tools when the MCP server is connected to eliminate functional overlap.
This is overly drastic and removes a genuinely useful tool. Grep serves many purposes beyond dependency analysis (searching for error messages, finding usage patterns, etc.). Removing it would degrade the agent's overall capability for unrelated tasks. The goal is to improve tool selection, not to eliminate tools that have legitimate uses in other contexts.

D. Split analyze_dependencies into granular tools (list_imports, resolve_transitive_deps, detect_circular_deps) so each has a focused purpose less likely to overlap with Grep.
While more granular tools could help, this approach is overkill for the problem and creates additional complexity. Each granular tool requires its own description, testing, and maintenance. The simpler and more elegant solution is to provide a richer description for the existing tool, which requires minimal changes and no additional tool management overhead.

Reference:

Anthropic Claude Agent SDK Documentation – Tool Description Best Practices – Recommends detailed, capability-focused tool descriptions that clearly articulate what the tool does, what inputs it accepts, and what outputs it produces to improve tool-selection accuracy.

MCP (Model Context Protocol) Documentation – Tool Design – Emphasizes that tool descriptions should be explicit about capabilities and use cases to help models select the most appropriate tool for a given task.

Prompt Engineering Best Practices – Highlights that model behavior is strongly influenced by tool descriptions; clear, detailed descriptions are more effective than routing instructions or tool removal for improving tool selection.

Explanation:
The loss of source attribution occurs because metadata is not preserved through the summarization pipeline. The solution must maintain a clear separation between content and source metadata at every stage, ensuring that citations travel with the facts they support. Structured outputs enable both preservation and traceability throughout the workflow.

Correct Option:

B. Have each agent output structured data that separates content summaries from source metadata, including URLs, document names, and page numbers.
This is the most effective approach because it ensures source metadata is preserved as structured data throughout the entire pipeline. By separating content from metadata in structured outputs, each agent can carry forward the citation information without it being lost during summarization. The synthesis agent can then build a consolidated summary that maintains the link between each factual claim and its original sources, enabling the report generator to produce properly cited reports with full attribution.

Incorrect Options:

A. Have the report generator query the web-search agent to relocate sources for claims in the final report.
This is inefficient and unreliable. After the report is written, the report generator would need to make separate queries to locate sources for each claim, which may fail if the exact source cannot be rediscovered. This approach also assumes that the web-search agent can reliably find the same sources again, which is not guaranteed. It adds significant latency and complexity without ensuring consistent attribution.

C. Skip summarization and pass the complete raw outputs from the web-search and document-analysis agents directly to the report generator.
This overloads the report generator with excessive, raw information, increasing token usage and context window pressure. The report generator would need to synthesize the information on its own, effectively duplicating the synthesis agent's work. This also loses the value of having a dedicated synthesis agent that can identify patterns, resolve contradictions, and extract key insights from multiple sources.

D. Instruct the synthesis agent to embed source references inline within its summary text using a consistent citation format.
While this could work, it is brittle and relies on the synthesis agent to consistently format and track citations correctly in free text. Embedding citations inline increases the risk of errors (missed citations, incorrect formatting, mismatched references) compared to using structured data. It also makes it harder for downstream systems to programmatically extract and format citations, as they would need to parse the text for citation patterns.

Reference:

Anthropic Multi-Agent Coordination Best Practices – Recommends structured outputs with rich metadata (including source references) to maintain data lineage and enable accurate attribution in downstream stages.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of data lineage and transparent metadata to support ongoing monitoring, validation, and auditability.

Academic Citation Standards – Highlights the importance of preserving source metadata (author, title, publication date, page numbers) to enable proper attribution and verification of claims.

Explanation:
The agent has successfully gathered all necessary information and verified eligibility, but the backend failure prevents completing the refund. The best approach balances resolving what can be resolved (explaining billing, confirming eligibility) while being transparent about the limitation and offering clear options to the customer. This maintains trust and maximizes first-contact resolution while handling the error gracefully.

Correct Option:

C. Explain the billing, confirm refund eligibility, acknowledge the system issue preventing immediate processing, and offer escalation or retry later.
This is the best approach because it delivers maximum value despite the backend failure. The agent uses the information it already gathered to address the customer's underlying concern—explaining the charges and confirming eligibility—while being transparent about the processing limitation. By offering the customer a choice (escalate now or retry later), the agent respects customer autonomy and maintains trust. This approach can achieve partial first-contact resolution and may fully resolve the issue if the customer chooses to retry later, without escalating unnecessarily.

Incorrect Options:

A. Implement automatic retries with exponential backoff for process_refund, keeping the conversation open until the refund is successfully processed.
This is impractical because it keeps the customer waiting indefinitely while retries occur. The customer would be stuck in the conversation with no clear timeframe for resolution, degrading the experience. Exponential backoff could take minutes or hours, violating customer experience expectations. The agent lacks information about whether the backend is temporarily or persistently failing, so indefinite retries are not appropriate.

B. Confirm the refund will be processed and close the conversation, since the system has all necessary information to complete it automatically.
This is dishonest and risky. The agent has no guarantee the refund will ever be processed—the timeout could indicate a persistent failure. Making false promises to close the conversation would lead to unresolved issues, customer complaints, and potential regulatory/compliance problems. This approach violates transparency and trust principles in customer support.

D. Escalate immediately to a human agent since the refund action cannot be completed.
This is premature and unnecessary. The agent has already provided value by retrieving customer info, looking up the order, and determining eligibility. Escalating immediately discards this progress and forces the customer to repeat information to a human agent, reducing first-contact resolution and degrading the experience. The agent should leverage its completed work before considering escalation.

Reference:

Anthropic Claude Agent SDK Documentation – Human-in-the-Loop (HITL) Patterns – Recommends that agents maximize resolution value before escalating, using gathered information to explain status and offer choices.

Customer Support Best Practices – Emphasizes transparency about limitations, offering customers clear choices, and maintaining trust through honest communication about system issues.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of appropriate error handling, transparency, and maintaining customer trust in automated systems.

Explanation:
The root cause is that a single static prompt with only changed files cannot identify cross-file impacts because it lacks visibility into calling code. The solution must enable the review process to dynamically explore the codebase, following references to verify how changes affect other parts of the system.

Correct Option:

C. Redesign the review as a turn-limited agentic task that can read files and search the repository, following references to verify cross-file findings.
This is the most effective approach because it transforms the review from a static analysis into an adaptive exploration. The agent can use Read, Grep, and Glob tools to discover callers of changed functions, inspect their usage patterns, and verify whether they remain compatible with the changes. This dynamic approach can identify cross-file bugs that would be missed by any static prompt, regardless of how many files are included. The agent can follow the dependency graph naturally, reading only the files that are actually relevant to the specific changes being reviewed.

Incorrect Options:

A. Build a static dependency graph and include every file located within two dependency hops of a changed file.
This risks including too many files (exponential growth), overwhelming the context window and increasing costs. A two-hop dependency graph for a core utility function could include hundreds of files. It also lacks precision—many included files may be irrelevant, while some critical callers might be more than two hops away. Static inclusion cannot adapt to the specific nature of each change.

B. Add instructions asking the model to list external references and reason step by step about how each change could affect unseen callers.
Instructions alone cannot overcome the fundamental limitation of not having the actual code to analyze. The model can only guess about potential callers based on naming patterns or conventions, which is unreliable and prone to false positives or missed issues. Without the ability to read the actual caller code, the model cannot verify compatibility.

D. Run separate review passes for each changed file with its direct dependants, and then aggregate and deduplicate the findings through a final consolidation pass.
This is inefficient and still incomplete—including only direct dependants may miss transitive callers or indirect usages. Running multiple passes increases token usage and latency significantly. The aggregation step also adds complexity and potential for lost or duplicated findings. An agentic approach is more elegant and efficient because it reads only what's needed based on the specific changes.

Reference:

Anthropic Claude Agent SDK Documentation – Agentic Code Review Patterns – Recommends using agentic tasks with tool access for reviews that require cross-file analysis, enabling dynamic exploration of the codebase.

Software Engineering Code Review Best Practices – Highlights that cross-file impacts require understanding of call graphs and usage patterns, which is best achieved through interactive exploration rather than static analysis.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of comprehensive testing and validation that accounts for interactions across system components.

Explanation:
The 55% low-value test rate stems from a lack of clear, contextual guidance about what constitutes a valuable test in your specific codebase. Claude needs explicit standards—not just generic testing best practices, but your team's specific conventions, fixture patterns, and examples of meaningful vs. trivial tests. Documentation at the project level provides persistent, reusable guidance that addresses the root cause.

Correct Option:

D. Document testing standards in CLAUDE.md, including valuable-test criteria, available fixtures and their intended use cases, and examples distinguishing meaningful behavioral tests from trivial assertions.
This is the most effective approach because it addresses the root cause at the source—the prompt/context level. CLAUDE.md provides persistent, project-specific guidance that Claude will reference for every test generation request. By including concrete criteria (e.g., "a valuable test verifies business logic, not just that functions don't throw"), available fixtures with usage examples, and clear do/don't examples, you fundamentally shift Claude's behavior. This prevents low-value tests from being generated in the first place, rather than filtering them afterward.

Incorrect Options:

A. Implement two-phase generation in which a second Claude call scores each test against quality criteria, filtering out low-scoring tests before presenting results to developers.
While this could improve the quality of presented tests, it is inefficient—you're generating tests only to discard them, wasting tokens and latency. More importantly, it doesn't teach Claude to generate better tests initially. The second pass is a filter, not a solution to the underlying issue. This also adds complexity and cost without addressing why the first pass produces low-quality tests.

B. Add post-generation coverage analysis that automatically filters out any generated test that does not increase line coverage beyond existing tests.
Line coverage is a poor proxy for test value. A test can increase coverage by exercising a trivial getter or error-handling branch without providing meaningful behavioral validation. Conversely, a valuable test that verifies complex business logic might not increase coverage if the paths are already covered by other tests. This filtering approach risks discarding genuinely valuable tests while accepting superficial coverage-inflating ones.

C. Restrict test generation to directories where historical quality metrics show higher acceptance rates, disabling it for areas where generated tests consistently require substantial editing.
This is a workaround that avoids the problem rather than solving it. Disabling test generation in problem areas forfeits the productivity benefits of automation where they might be most needed (complex or legacy code). This approach also doesn't help improve quality in the restricted areas—they simply remain uncovered, and the root behavioral issue persists.

Reference:

Anthropic Claude Documentation – CLAUDE.md Best Practices – Recommends documenting project-specific conventions, standards, and examples to provide persistent, context-aware guidance for code generation tasks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of clear specifications, validation criteria, and documented standards to ensure model outputs meet quality and risk management requirements.

Testing Best Practices – Test Value Criteria – Emphasizes that valuable tests verify behavior and business logic, not just code execution; tests should validate specific outcomes and edge cases rather than merely exercising code paths.

Explanation:
The synthesis subagent needs access to the complete findings from the web-search and document-analysis subagents to perform its synthesis task effectively. The most straightforward and reliable approach is to provide these findings directly in the subagent's prompt, ensuring that all necessary information is available without requiring additional tool calls or shared memory access.

Correct Option:

B. Include the complete findings from both subagents directly in the synthesis subagent's prompt.
This is the most effective approach because it ensures the synthesis agent has immediate, complete access to all findings without requiring additional infrastructure or dependencies. Direct inclusion in the prompt is simple, reliable, and ensures that all necessary information is available in context for the synthesis task. This approach avoids the complexity of shared memory stores, callback mechanisms, or automatic inheritance, which can introduce reliability issues or incomplete data transfer.

Incorrect Options:

A. Pass reference identifiers and configure the subagent with read access to a shared memory store where the other subagents deposited their results.
This adds unnecessary complexity and introduces potential failure points. The subagent would need to make additional tool calls to retrieve data from the shared store, increasing latency and token usage. It also requires managing concurrent access and ensuring data is properly persisted. Direct inclusion is simpler, more reliable, and easier to debug.

C. Provide the subagent with tool definitions that allow it to request outputs from the other subagents through callbacks.
This creates circular dependencies and coordination challenges. The synthesis agent would need to call back to the coordinator or other subagents to retrieve data, introducing complexity and potential for deadlocks or incomplete data retrieval. This approach also violates the separation of concerns, as the synthesis agent would need to know how to interact with other subagents' interfaces.

D. Spawn the subagent with only a brief task description, relying on automatic context inheritance from the coordinator.
The Claude Agent SDK does not provide automatic context inheritance in the way this option describes. The coordinator's context (including findings) is not automatically passed to subagents. Relying on undocumented or assumed behavior is risky and would likely result in the synthesis agent lacking the necessary information, leading to incomplete or hallucinated outputs.

Reference:

Anthropic Claude Agent SDK Documentation – Subagent Configuration – Recommends explicitly passing all necessary context (including findings from other agents) in the subagent's prompt to ensure complete and reliable task execution.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of complete and accurate data transfer between system components to ensure reliable outputs and avoid gaps.

Multi-Agent System Design Best Practices – Highlights that explicit data passing is more reliable than shared memory or callback-based approaches, especially for ensuring consistency and completeness.

Explanation:
The coordinator can reason about delegation and plan the task flow, but it cannot execute the delegation because it lacks the necessary tool to spawn subagents. In the Claude Agent SDK, subagent invocation is handled through a specific tool (often named "Agent" or "Task" depending on the SDK version). Without this tool in the coordinator's allowedTools configuration, the coordinator cannot actually execute the delegation—it can only plan and describe it.

Correct Option:

B. The coordinator's allowedTools configuration does not include "Agent"—called "Task" in older SDK releases—so it cannot invoke the tool required to spawn subagents.
This is the most likely cause because it perfectly explains the observed behavior: the coordinator correctly plans delegation (generating messages like "I'll ask the web-search agent...") but cannot execute it because the tool required to spawn subagents is not available. The absence of the Agent/Task tool means the coordinator is limited to reasoning about delegation without the ability to perform it, resulting in no subagent execution despite no error logs (the tool simply isn't there to call).

Incorrect Options:

A. The AgentDefinition objects are configured correctly, but the coordinator's system prompt does not explicitly list the available subagent types.
While the system prompt can help the coordinator understand available subagents, the ability to actually invoke subagents depends on having the Agent/Task tool in allowedTools. Even without subagent types listed in the prompt, if the tool is available, the coordinator could discover them through tool descriptions or use them appropriately. The prompt is guidance; the tool is the mechanism.

C. Subagent context isolation prevents task descriptions from reaching subagents unless explicit context forwarding is configured in ClaudeAgentOptions.
Context isolation would affect what information subagents receive, but it would not prevent the coordinator from spawning them in the first place. If the coordinator could call the Agent/Task tool, subagents would be invoked regardless of context isolation; the issue would then be that subagents lack necessary context, not that they aren't spawned at all.

D. The coordinator's max_tokens setting is too low, causing the subagent invocation to be truncated before the agent-type parameter is specified.
If max_tokens were too low, you would typically see truncation errors or incomplete tool calls. The coordinator's reasoning would be cut off, and the logs would likely show errors or warnings about token limits. Since there are no errors and the coordinator completes its reasoning but fails to execute, a token limit issue is unlikely. The tool simply isn't available to call.

Reference:

Anthropic Claude Agent SDK Documentation – Subagent Configuration – Clarifies that the coordinator must have the Agent/Task tool in its allowedTools configuration to invoke subagents; without it, the coordinator can only plan but not execute delegation.

Anthropic Claude Agent SDK – Version Migration Notes – Notes that the subagent invocation tool was renamed from "Task" to "Agent" in newer SDK releases, which may cause configuration issues if not updated.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of proper configuration and validation of automated systems to ensure all components are correctly enabled and functioning.

Explanation:
The session contains valuable context from yesterday's analysis (architecture understanding, identified patterns, and relationships across 12 files), but three files have changed. A fresh session discards all this work, while blindly resuming risks stale assumptions. The optimal approach preserves the existing context while specifically addressing the changes to ensure accuracy.

Correct Option:

B. Resume the session and inform the agent which specific files changed for targeted re-analysis.
This is the most effective approach because it balances efficiency and accuracy. By resuming the session, you preserve all the valuable context the agent built yesterday—understanding of architecture, dependencies, and patterns. By informing the agent which 3 specific files changed, you enable targeted re-analysis of only the affected areas, allowing the agent to update its understanding without re-analyzing unchanged files. This approach minimizes redundant work while ensuring the agent's mental model is current.

Incorrect Options:

A. Start a fresh session to ensure the agent works with current codebase state without stale assumptions.
This discards an hour of valuable analysis work and forces the engineer to start from scratch. While it ensures accuracy, it is highly inefficient. The context about unchanged files (9 out of 12) remains valid and useful—throwing it away wastes significant effort. This approach fails to leverage the session persistence feature you built.

C. Resume the session and immediately have the agent re-read all 12 previously analyzed files.
This unnecessarily re-analyzes 9 files that haven't changed, wasting tokens and time. While it would ensure accuracy, it is inefficient and defeats the purpose of session persistence. The agent already built understanding of these files yesterday; re-reading them is redundant and adds latency without improving accuracy for the unchanged files.

D. Resume the session without informing the agent about the changed files.
This risks the agent operating on stale assumptions and providing inaccurate analysis. The agent would continue as if the codebase were unchanged, potentially giving outdated advice about the authentication module. This approach compromises accuracy and could lead to incorrect conclusions or bad recommendations, defeating the purpose of the tool.

Reference:

Anthropic Claude Agent SDK Documentation – Session Management – Recommends resuming sessions with specific context about changes to balance efficiency and accuracy, rather than full reset or blind continuation.

Software Engineering Best Practices – Incremental Analysis – Highlights that when working with codebases, it is efficient to preserve valid context and only re-analyze changed components.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of maintaining accurate, up-to-date data and context in automated systems while avoiding unnecessary rework.

Explanation:
To analyze why 35% of findings are dismissed, you need to understand which specific code patterns or constructs are triggering the false positives. By recording the detected_pattern for each finding, you can aggregate dismissals by pattern, identify which patterns are most frequently dismissed, and target prompt improvements specifically for those patterns.

Correct Option:

B. Add a detected_pattern field recording the specific code construct that triggered the finding, such as single-letter loop variable.
This is the most effective change because it enables granular analysis of dismissal patterns. By grouping dismissed findings by detected_pattern, you can identify which specific code constructs consistently generate false positives. For example, if you discover that 80% of dismissed findings have a detected_pattern of "force_unwrap_in_test_file," you can add project-specific instructions about acceptable force-unwrapping in test contexts. This field provides actionable, specific data to guide targeted prompt improvements.

Incorrect Options:

A. Add a model_confidence field from 0.0 to 1.0 and filter findings below a threshold calibrated against historical dismissal rates.
While confidence scores can be useful, they do not help you understand why findings are dismissed. Filtering below a threshold may reduce dismissals but does not address the root cause—the model flagging acceptable patterns. This approach masks the problem without providing the diagnostic data needed to improve prompts. It also risks filtering out low-confidence but genuine issues.

C. Expand the description field with more detailed explanations of why each issue matters and how it should be fixed.
While better explanations might help developers understand findings, this change does not provide diagnostic data for analyzing dismissal patterns. The goal is to understand what the system is getting wrong at a pattern level, not to improve individual finding descriptions. This change would consume more tokens without enabling the analysis needed to reduce false positives.

D. Remove the issue_category field and track dismissal rates only at the individual-finding level.
Removing data reduces your ability to analyze patterns. Without issue_category, you cannot identify whether dismissals are concentrated in security, style, or other categories. Tracking only at the individual-finding level provides no aggregation capability for pattern analysis. This change moves in the wrong direction—you need more granular data, not less.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends analyzing systematic errors by collecting structured data about output characteristics to identify patterns and target improvements.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of ongoing monitoring and granular data collection to identify model weaknesses and inform validation improvements.

Software Engineering Metrics Best Practices – Highlights that effective issue tracking requires structured fields that enable aggregation and pattern analysis to identify systemic problems.

Explanation:
False positives arise because the model lacks examples that illustrate the boundary between acceptable patterns (given your team conventions and deployment context) and genuine issues. Few-shot examples enable the model to learn the reasoning heuristics for distinguishing these categories, allowing it to generalize to novel patterns not explicitly covered in the prompt.

Correct Option:

B. Include few-shot examples containing annotated code snippets that distinguish acceptable project patterns from genuine issues in each category.
This is the most effective approach because examples teach the model the decision boundary rather than just listing rules. By providing concrete annotated snippets—e.g., "This pattern is safe in our deployment context because X, but this similar pattern is a genuine security issue because Y"—the model learns the contextual reasoning needed to generalize. Unlike exhaustive rule lists, few-shot examples help Claude recognize that the same pattern may be acceptable or problematic depending on context, enabling accurate judgment on novel code.

Incorrect Options:

A. Create a comprehensive specification of every pattern that must not be flagged and include the complete document in the system prompt.
This is impractical because the universe of acceptable patterns is unbounded—you cannot enumerate every safe pattern. A comprehensive specification would be too long for the context window and would still fail to cover novel patterns not anticipated. Rules also lack the contextual nuance that examples provide; the model may struggle to apply abstract rules to concrete code.

C. Use keyword-based post-processing to remove findings containing terms such as "convention," "context-dependent," or "trade-off."
This is a crude filter that will miss nuanced false positives and may filter out genuine issues that legitimately involve conventions or trade-offs. Keyword filtering cannot distinguish between a false positive where the pattern is acceptable and a genuine issue where the same terminology appears. This approach adds complexity without improving the model's reasoning capability.

D. Add general instructions telling Claude to be conservative and report only definite issues.
These instructions are vague and subjective. Claude's interpretation of "conservative" or "definite" may not align with your team's standards, and this approach does not provide the concrete, contextual guidance needed to distinguish acceptable patterns from genuine issues. Without examples illustrating the boundary, the model may still produce the same false positives or, worse, become overly conservative and miss genuine issues.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends using few-shot examples to teach nuanced decision boundaries, as examples are more effective than abstract rules for contextual judgment tasks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of validation and calibration using representative examples to ensure model outputs meet quality standards and reduce false positives.

NIST AI Risk Management Framework – Highlights that AI systems should be trained and validated using diverse, representative examples to ensure generalization and reduce systematic errors.

Explanation:
The agent handles three distinct issues across 48 turns, approaching context limits while the customer references an earlier issue (refund inquiry from turns 1–15). Summarization or sliding windows risk losing critical details, while relying solely on tools loses conversational context. The optimal solution extracts and persists structured data about each issue, enabling the agent to reference specific information without retaining the entire conversational history.

Correct Option:

D. Extract and persist structured issue data (order IDs, amounts, statuses) into a separate context layer.
This is the most effective approach because it decouples critical business data from the conversational history. By extracting structured data about each issue (e.g., refund order ID, amount, current status; subscription type, billing cycle; payment method details), you create a durable, compact, and queryable context layer. The agent can reference this structured data when the customer asks follow-up questions, without needing the full 45-turn conversation history. This preserves both accuracy and context efficiency, enabling the agent to handle all three issues throughout the session.

Incorrect Options:

A. Summarize earlier turns into a narrative description, preserving full message history only for the active issue.
While summarization can reduce token usage, narrative summaries are lossy and may omit critical details needed to answer the customer's follow-up questions. When the customer asks "What happened with my refund?", a summary might not contain the specific order ID, amount, or status needed to provide an accurate, actionable response. This approach risks losing precision for the sake of compression.

B. Implement sliding window context that retains the most recent 30 turns.
A sliding window would drop the refund conversation (turns 1–15) by the time the customer asks about it at turn 48. The agent would have no memory of the earlier refund discussion, forcing it to ask the customer to repeat information or rely solely on backend lookups without conversational context. This approach cannot handle the multi-issue, long-duration use case effectively.

C. Rely on MCP tools to re-fetch relevant information on demand when the customer references earlier issues.
While tools like get_customer and lookup_order can retrieve backend data, they cannot recover the conversational context—what the customer was told earlier, what commitments were made, or what previous explanations were given. The agent would be able to fetch raw order details but would lack the history of what was communicated, leading to inconsistent or repetitive responses.

Reference:

Anthropic Claude Agent SDK Documentation – Context Management – Recommends extracting and persisting structured data for long-running multi-issue sessions to maintain accuracy while managing context limits.

Customer Support Best Practices – Highlights the importance of maintaining full context across multiple issues in a single session, especially when customers reference earlier topics.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the need for accurate, complete data retention in automated systems to support consistent and reliable customer interactions.

Explanation:
The root cause is that a single API call with a large pull request (30+ files) exceeds the max_tokens limit, causing JSON truncation and parser failure. The solution must reduce the amount of content per call while preserving the complete review coverage. Splitting the review into multiple focused calls directly addresses the token constraint without sacrificing quality or completeness.

Correct Option:

A. Split the review into multiple API calls that each analyze a subset of the changed files, and then merge the resulting findings arrays.
This is the most effective approach because it directly addresses the token limitation by partitioning the work into smaller, manageable chunks. Each API call analyzes a subset of files, staying within the max_tokens limit, and the complete findings are merged post-processing. This approach is reliable, preserves the quality of each individual review, and can scale to arbitrarily large pull requests. It also allows for parallel processing if needed, reducing overall latency.

Incorrect Options:

B. Increase max_tokens to the model's maximum and instruct Claude to keep each finding description under 50 words.
While increasing max_tokens may help, it is not guaranteed to solve the problem for very large pull requests, and it increases costs for all reviews. Instructing Claude to keep descriptions under 50 words forces unnatural brevity that may reduce the actionability of findings. This approach is a band-aid that does not address the fundamental scaling issue and may degrade output quality.

C. Switch from tool_use to prompting Claude to return findings as a Markdown list.
This changes the output format but does not address the root cause of exceeding the token limit. Markdown lists are not inherently more token-efficient than structured JSON with tool_use. The parser would need to be redesigned, and the approach still risks truncation on large pull requests. This option adds work without solving the problem.

D. Add retry logic that detects truncated JSON and resends the request with instructions to report only critical and high-severity findings.
This approach degrades the review quality by filtering out lower-severity findings, which may include important issues. The retry logic would only trigger after a failure, causing delays and inconsistent results. It also does not guarantee that the reduced set will fit within the token limit, especially for very large pull requests. This approach sacrifices completeness for reliability, which is not acceptable for a comprehensive code review.

Reference:

Anthropic API Documentation – Error Handling – Recommends splitting large requests into multiple smaller calls to avoid token limits, rather than relying on max_tokens increases or output compression.

Software Engineering – Batch Processing Patterns – Highlights that splitting large workloads into smaller batches is a standard pattern for managing resource constraints while maintaining completeness.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of robust error handling and ensuring that automated systems can process large workloads reliably and completely.