Free CCAR-F Practice Test Questions 2026

Explanation:
The Message Batches API processes requests asynchronously with a potential delay of up to 24 hours. For a non-blocking CI workflow, the core feasibility hinges on whether delayed feedback (arriving hours later) can still be practically acted upon by developers without causing significant rework or context-switching costs. Cost savings are irrelevant if the feedback is no longer timely.

Correct Option:

C. Whether review feedback arriving up to 24 hours after pull-request creation remains actionable.
This is the primary factor. Since the API allows up to 24 hours of latency, the business value depends entirely on whether findings that arrive a day later can still be integrated into follow-up commits without disrupting the development cycle. If the codebase moves too fast, this delay renders the feedback obsolete, making the cost reduction pointless regardless of other technical factors.

Incorrect Options:

A. Whether your result-processing system can handle reviews arriving in a different order from the order in which they were submitted.
While order handling is a technical consideration for batch systems, it is not the most determining factor. The API can include sequence identifiers to reorder results. This is a solvable implementation detail, whereas the 24-hour latency is a hard, non-negotiable SLA constraint that directly impacts workflow viability.

B. Whether each review can be structured as a single request without multi-turn refinement.
This relates to API capability, not batch suitability. The Message Batches API supports single requests per batch item. Even if multi-turn refinement is needed, you can chain multiple batch calls. This does not determine whether the batch model itself is appropriate for your use case; latency and actionability do.

D. Whether reducing per-review latency from 30–60 seconds to near-instantaneous delivery matters to your workflow.
This is misleading because the Message Batches API increases latency (up to 24 hours) rather than reducing it. Instantaneous delivery is a characteristic of the synchronous API, not the batch API. Your workflow is non-blocking, so reducing latency is not a driver; the critical question is whether you can tolerate higher latency.

Reference:

Google Cloud Vertex AI Pricing & Batch API Documentation – Highlights that batch jobs are processed asynchronously with potential delays and are recommended for non-time-sensitive workloads.

SR Letter 11-7 (Federal Reserve) – On supervisory expectations for model risk management, emphasizing that the "timeliness" of data and outputs is a key factor in determining the ongoing effectiveness of a model or analytical process.

Explanation:
In AI-assisted development, the most effective iteration strategy is providing concrete, executable examples that illustrate the failure mode. Abstract instructions like "handle edge cases" are ambiguous, while test cases give Claude explicit constraints to satisfy. This approach transforms a vague requirement into a verifiable specification, enabling targeted fixes rather than guesswork.

Correct Option:

D. Provide a test case with example input containing null values and the expected output, then ask Claude to fix it.
This is the most effective approach because it gives Claude concrete, verifiable specifications. A test case removes ambiguity and allows Claude to debug systematically by comparing actual vs. expected behavior. It also enables you to validate the fix immediately and builds a regression test for future iterations, making the collaboration more reliable and efficient.

Incorrect Options:

A. Add "think harder about edge cases" to your prompt and request a complete rewrite of the migration logic.
This is ineffective because it relies on vague, unverifiable prompting. "Think harder" does not provide new information about the specific failure (null handling). Requesting a complete rewrite wastes tokens and risks introducing new issues while not guaranteeing the null problem is addressed, as Claude lacks concrete examples of what "correct" looks like.

B. Manually edit the generated code to fix the null handling, then continue working with Claude on other parts.
While this solves the immediate bug, it misses the opportunity to teach Claude the correct pattern for this context. Future similar requests may reproduce the same error, requiring repeated manual fixes. This approach also breaks the feedback loop, preventing Claude from learning the team's specific data quality requirements.

C. Describe the null value problem in detail and ask Claude to regenerate the entire script with improved edge case handling.
Detailed description is better than "think harder," but still lacks the precision of an executable test case. Regenerating the entire script is inefficient—it may fix null handling but could break working logic. Without a test case, you cannot verify the fix objectively, and Claude might misinterpret what "proper" handling means in your specific data context.

Reference:

Anthropic Claude Documentation – Best Practices for Iterative Development – Emphasizes providing specific examples, test cases, and failure scenarios rather than abstract instructions.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of empirical validation and testing with edge cases to ensure model (or code) outputs are robust under adverse conditions.

Agile Testing Principles – Stresses that executable specifications (tests) are more effective than textual descriptions for communicating requirements and verifying correctness.

Explanation:
Contracts with amendments create temporal complexity—multiple valid values exist for the same field, each applicable to different periods. The core problem is loss of provenance, not just incorrect extraction. The solution must preserve the relationship between values, their sources, and their effective dates to enable downstream systems to determine applicability based on context.

Correct Option:

B. Redesign the schema so amended fields capture multiple values, each with source location and effective date. This is the most effective approach because it fundamentally addresses the data model limitation. By capturing all values with metadata (source location and effective date), you preserve the complete audit trail and enable downstream systems to apply business logic correctly. This transforms the problem from "choosing the wrong value" to "providing all relevant information for context-aware decision-making."

Incorrect Options:

A. Preprocess documents with a classifier that identifies and removes superseded sections before the main extraction step.
This is risky because determining what is "superseded" requires legal interpretation that a simple classifier cannot reliably perform. Amendments may modify only specific subsections while leaving others intact. Removing sections could inadvertently delete still-active terms, creating compliance and legal risks. This approach also adds complexity without guaranteeing accuracy.

C. Add prompt instructions to always extract the most recent amendment value and ignore superseded original terms.
This oversimplifies contract law. Determining the "most recent" amendment that actually supersedes a specific clause requires legal reasoning about the scope and applicability of each amendment. Prompts alone cannot reliably perform this analysis, and ignoring superseded terms loses valuable historical context needed for dispute resolution or compliance audits.

D. Implement post-extraction validation using pattern matching to detect amendments and flag those extractions for manual review.
While flagging for review acknowledges the problem, it scales poorly. Every amended contract would require manual intervention, defeating the automation goals. Pattern matching cannot distinguish between amendments that modify specific clauses versus those that reference them; this approach creates a review bottleneck without improving extraction quality.

Reference:

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of data lineage, transparency, and capturing all relevant data attributes to support ongoing monitoring and validation.

Basel Committee on Banking Supervision – Principles for Effective Risk Data Aggregation and Reporting (BCBS 239) – Requires banks to maintain audit trails and ensure data is traceable to source systems, supporting the need for metadata preservation.

ISO 8000 (Data Quality Standards) – Highlights that data quality requires not just accuracy but also completeness and provenance, especially when multiple valid values exist for the same attribute.

Explanation:
The root cause is an overly restrictive prompt instruction that forces the model to self-censor uncertain or low-severity findings. The solution must remove this censorship while maintaining structured output for downstream filtering. The key is to shift the filtering responsibility from the model's generation phase to the post-processing phase, where business rules can be consistently applied.

Correct Option:

B. Instruct the model to report all findings with confidence and severity tags, deferring filtering to a downstream step.
This directly removes the suppression instruction while preserving structured, tagged output. The model now reports everything it identifies, including uncertain or lower-severity issues, with metadata tags. Downstream systems can then apply configurable filtering rules (e.g., only show critical issues) without losing access to the full finding set. This separates detection from decision-making, ensuring no genuine bugs are missed while maintaining actionable output.

Incorrect Options:

A. Enable extended thinking and instruct the model to reason step by step about every code change before producing its review.
While extended thinking might improve reasoning, it does not address the core problem—the model is still operating under an instruction to suppress uncertain findings. Step-by-step reasoning about issues it has been told to ignore doesn't help. This approach increases latency and token costs without fixing the underlying censorship issue.

C. Remove all severity-related instructions and allow the model to use its default judgment about which findings to report.
This is unpredictable because Claude's default behavior on severity thresholds is undocumented and may vary across prompts and versions. Without explicit guidance, the model might still suppress findings it personally deems minor. This approach also loses the structured severity metadata that the team needs for downstream filtering, making output less actionable.

D. Add a second review pass that rereads the diff using the same prompt and looks for anything the first pass may have missed.
A second pass with the same restrictive prompt will likely reproduce the same omissions. The model is not missing findings due to oversight but because it has been explicitly instructed to suppress them. Running the same prompt twice wastes resources and does not address the root cause—it simply repeats the same flawed instruction.

Reference:

Anthropic Claude Documentation – Prompt Design Best Practices – Recommends using confidence scores and metadata to maintain transparency, and suggests deferring filtering decisions to downstream systems rather than embedding them in prompts.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of reporting all relevant findings regardless of confidence, with transparency about uncertainty, to support human review and oversight.

NIST AI Risk Management Framework – Recommends designing AI systems to maximize transparency and provide appropriate context for outputs, enabling human users to make informed decisions.

Explanation:
Mid-process escalation in an AI agent workflow requires preserving the investigative context to enable seamless human takeover. The agent has already performed valuable work—gathering customer details, order information, and diagnosing the root cause. This context must be efficiently transmitted to the human agent to avoid redundant work and ensure a smooth customer experience.

Correct Option:

B. Compile a structured handoff with customer details, order info, and the identified issue before calling escalate_to_human.

This is the most effective approach because it encapsulates all the investigative work the agent has already completed. A structured handoff includes customer identity, order context, and the specific promotional pricing issue requiring manager approval. This enables the human agent to immediately understand the situation and take action without repeating the agent's diagnostic steps, directly supporting the 80%+ first-contact resolution target by reducing resolution time.

Incorrect Options:

A. Call escalate_to_human, passing only the customer's original message.
This discards all the valuable context the agent gathered through tool calls. The human agent would have to re-run get_customer and lookup_order, duplicating work and frustrating the customer who must repeat information. This approach violates the principle of efficient handoff and increases resolution time, potentially reducing first-contact resolution rates.

C. Attempt the refund with process_refund anyway, escalating only if the system rejects the transaction.
This risks policy violations and compliance issues. The agent has already determined that the issue requires manager approval due to promotional pricing errors—authorization boundaries exist for a reason. Attempting unauthorized refunds could lead to financial losses, audit failures, and regulatory scrutiny. Process_refund should only be called when the agent is authorized to approve the specific transaction.

D. Persist the complete conversation and tool response history to a database, then call escalate_to_human with a reference ID.
While persisting history is valuable for audit trails, this approach creates an unnecessary extra step that delays resolution. The human agent now must access a separate system to retrieve the context. This adds friction, increases handoff latency, and may still require the customer to wait while the human retrieves information, negatively impacting the customer experience.

Reference:

Anthropic Claude Agent SDK Documentation – Human-in-the-Loop (HITL) Patterns – Recommends structured handoffs that include tool call results, conversation state, and the agent's reasoning to enable seamless human takeover.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of transparency and audit trails in automated decision-making systems, particularly when escalating to human reviewers.

CCAR Stress Testing Guidelines – While focused on capital planning, these guidelines stress that automated processes should maintain clear escalation paths with complete documentation for human oversight and decision-making.

Explanation:
The root cause is a fundamental misalignment between the agents' role descriptions and their available tool sets. When an agent is told it is a "synthesis agent" but given access to web search tools, the system prompt conflicts with the tool exposure. The model interprets tool availability as permission to use them, overriding role-specific behavioral constraints and leading to out-of-specialization tool calls.

Correct Option:

C. The agents' role descriptions in their system prompts conflict with having access to tools outside those roles.
This is the primary cause because Claude models are highly responsive to both system prompts and available tools. When a synthesis agent has access to web search tools, the model reasonably assumes it may use them—tool availability implies authorization. The role description says "synthesize findings," but tool access says "you can search the web." This contradictory signal creates ambiguity, and models typically resolve it by treating tool access as the stronger instruction for what actions are permitted.

Incorrect Options:

A. The tool definitions consume too much context-window space, leaving insufficient room for task content.
While 18 tool definitions do consume context, modern Claude models have large context windows (200K+ tokens). The issue is not space but clarity and role specificity. Truncated or compressed context could affect quality, but it would not specifically cause agents to call tools outside their specialization—that requires a behavioral/directional misalignment, not a capacity issue.

B. Choosing from 18 tools instead of four or five relevant tools increases decision complexity beyond reliable selection thresholds.
While tool selection complexity does increase with more options, this is not the primary cause. Claude models can handle 18 tools effectively when properly configured. The core issue is not the raw number of tools but the mismatch between role definitions and available tools. Reducing tool count without fixing the role/tool alignment would only mask the symptom, not solve the root problem.

D. The coordinator cannot track which capabilities each subagent has, leading to misrouted tasks.
This incorrectly assumes the coordinator controls subagent tool access. In the Claude Agent SDK, each subagent is independently configured with its own tool set and system prompt. The coordinator delegates tasks based on content, but the subagents themselves make tool selection decisions. The coordinator does not need to track capabilities; rather, the subagent configuration must enforce role boundaries through tool restriction.

Reference:

Anthropic Claude Agent SDK Documentation – Subagent Configuration – Recommends restricting tool access to only those tools relevant to each subagent's specialization to prevent role-confusion and improve reliability.

Anthropic Prompt Engineering Best Practices – Emphasizes that system prompts and tool availability must be consistent; if a tool is available, Claude will consider using it, even if the prompt suggests otherwise.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of clear role definitions and boundaries in automated decision-making systems to prevent unintended behaviors and maintain system integrity.

Explanation:
When exploring an unfamiliar codebase, the most effective approach is to follow the architectural dependencies to build a mental model from the top down. Starting with the base abstraction (interface/class) provides the contract, and then tracing implementations reveals how specific features like invalidation are realized. This targeted approach respects context constraints while building comprehensive understanding.

Correct Option:

A. Analyze imports and class hierarchies to identify the base cache class, read that file to understand its interface, and then trace the specific invalidation implementations.
This is the most effective approach because it follows the natural dependency graph of the codebase. Understanding the base cache class first establishes the contract—what methods exist, what parameters they take, and what the expected behavior is. Then, tracing specific invalidation implementations becomes meaningful because you understand how they fit into the overall architecture. This is also the most context-efficient, as you only read files directly relevant to the task.

Incorrect Options:

B. Use Glob to find files matching common caching patterns such as cache.py or caching/, read the largest files first, and inspect smaller files afterward.
This is inefficient because file size is not correlated with architectural importance. The largest file might be utility code or test fixtures rather than the core logic. Reading files in arbitrary order also prevents building a coherent mental model, as you encounter low-level details before understanding the high-level structure. This approach wastes context on irrelevant files.

C. Use Read to load all 15 files sequentially and build a complete understanding of the caching implementation.
This would consume approximately 8,000 tokens worth of code content, which is well within Claude's context window, but it represents poor information foraging strategy. Reading everything sequentially is inefficient when you only need to understand the invalidation trigger mechanism. This approach also increases cognitive load, making it harder to identify patterns and relationships across files.

D. Use Grep to search for invalidate and expire, and then read only the matching line ranges with minimal surrounding context.
This is too narrow and risks missing critical understanding. Invalidation logic may be triggered indirectly through cache eviction policies, TTL configurations, or event listeners that don't explicitly use the words "invalidate" or "expire." Reading only matched lines without surrounding context provides fragments without architectural understanding, leading to incomplete or incorrect conclusions about how invalidation works.

Reference:

Anthropic Claude Agent SDK Documentation – Tool Usage Best Practices – Recommends using Grep and Glob for initial discovery, then following the dependency graph with targeted Read calls to build understanding efficiently while respecting context constraints.

Software Engineering Principles – Reading Code for Understanding – Emphasizes top-down comprehension, starting with interfaces and abstractions before drilling down to implementations.

Feathers, "Working Effectively with Legacy Code" – Advocates for identifying the "seam" (the boundary between the code you understand and the code you don't) and using characterization tests or dependency analysis to build understanding incrementally.

Explanation:
The root cause is that a single, flexible tool with free-text instructions produces inconsistent output formats because the model must infer intent and format simultaneously. This ambiguity leads to 35% of results requiring rework. The solution must enforce structured, predictable outputs for each distinct analysis type while maintaining clear, machine-readable contracts.

Correct Option:

A. Split the generic tool into purpose-specific tools—extract_data_points, summarize_content, and verify_claim_against_source—each with defined input and output contracts.
This is the most effective approach because it creates clear separation of concerns at the tool level. Each tool now has a specific purpose, defined input parameters, and a structured output schema. When the agent calls extract_data_points, it expects and receives structured data; when it calls summarize_content, it receives narrative text. This eliminates ambiguity, making the agent's behavior predictable and the synthesis agent's job easier with consistent, typed outputs.

Incorrect Options:

B. Retain the single tool but add an analysis_type enum requiring explicit selection among extraction, summarization, and verification modes.
While this adds structure, it still keeps the tool monolithic and relies on the agent to correctly map the enum to the desired output. The underlying tool behavior remains ambiguous—how does "extraction mode" differ from "summarization mode" in implementation? This approach does not enforce output contracts at the tool level and still allows inconsistency in how each mode is executed.

C. Have the coordinator preclassify each analysis request before passing instructions to the document-analysis agent.
This shifts the classification burden to the coordinator but does not solve the tool's output ambiguity. The document-analysis agent still receives a free-text instruction and must infer the expected output format. Preclassification may improve instruction quality but does not enforce structured outputs, leaving the same 35% failure rate intact.

D. Enhance the tool description with detailed examples showing how different instruction phrasings should map to different output formats.
While better documentation may help, it relies on the model correctly interpreting and applying examples in every case. Examples are not enforceable contracts; the model may still produce narratives for extraction requests or tables for summarization. This approach improves guidance but does not guarantee consistency, as the tool remains a single, flexible endpoint without output validation.

Reference:

Anthropic Claude Agent SDK Documentation – Tool Design Best Practices – Recommends creating purpose-specific tools with clear, structured outputs rather than generic tools with flexible instructions, to improve reliability and predictability.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of defined outputs, validation, and clear contracts in automated systems to support ongoing monitoring and reduce operational risk.

API Design Principles – Interface Segregation – Suggests that well-defined, purpose-specific interfaces reduce complexity and improve reliability compared to monolithic, flexible interfaces.

Explanation:
The 55% low-value test rate stems from a lack of clear, contextual guidance about what constitutes a valuable test in your specific codebase. Claude needs explicit standards—not just generic testing best practices, but your team's specific conventions, fixture patterns, and examples of meaningful vs. trivial tests. Documentation at the project level provides persistent, reusable guidance.

Correct Option:

C. Document your testing standards in CLAUDE.md, including valuable-test criteria, available fixtures and their intended uses, and examples distinguishing meaningful behavioural tests from trivial assertions.
This is the most effective approach because it addresses the root cause at the source—the prompt/context level. CLAUDE.md provides persistent, project-specific guidance that Claude will reference for every test generation request. By including concrete criteria (e.g., "a valuable test verifies business logic, not just that functions don't throw"), available fixtures with usage examples, and clear do/don't examples, you fundamentally shift Claude's behavior. This prevents low-value tests from being generated in the first place, rather than filtering them afterward.

Incorrect Options:

A. Restrict test generation to directories where historical quality metrics show higher acceptance rates, disabling it in areas where generated tests consistently require substantial editing.
This is a workaround that avoids the problem rather than solving it. Disabling test generation in problem areas forfeits the productivity benefits of automation where they might be most needed (complex or legacy code). This approach also doesn't help improve quality in the restricted areas—they simply remain uncovered, and the root behavioral issue persists.

B. Implement two-phase generation in which a second Claude call scores every test against quality criteria and filters out low-scoring tests before presenting them to developers.
While this could improve the quality of presented tests, it is inefficient—you're generating tests only to discard them, wasting tokens and latency. More importantly, it doesn't teach Claude to generate better tests initially. The second pass is a filter, not a solution to the underlying issue. This also adds complexity and cost without addressing why the first pass produces low-quality tests.

D. Add post-generation coverage analysis that automatically filters out every generated test that does not increase line coverage beyond the existing test suite.
Line coverage is a poor proxy for test value. A test can increase coverage by exercising a trivial getter or error-handling branch without providing meaningful behavioral validation. Conversely, a valuable test that verifies complex business logic might not increase coverage if the paths are already covered by other tests. This filtering approach risks discarding genuinely valuable tests while accepting superficial coverage-inflating ones.

Reference:

Anthropic Claude Documentation – CLAUDE.md Best Practices – Recommends documenting project-specific conventions, standards, and examples to provide persistent, context-aware guidance for code generation tasks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of clear specifications, validation criteria, and documented standards to ensure model outputs meet quality and risk management requirements.

Testing Best Practices – Test Value Criteria – Emphasizes that valuable tests verify behavior and business logic, not just code execution; tests should validate specific outcomes and edge cases rather than merely exercising code paths.

Explanation:
The core problem is that the pipeline is strictly sequential, making evidence gaps identified during analysis non-actionable because search has already concluded. The solution must create a feedback loop where the analysis stage can identify gaps and trigger targeted follow-up searches, iterating until coverage criteria are met. This transforms the system from a one-pass pipeline to an adaptive research workflow.

Correct Option:

C. Require the analysis agent to return specific evidence gaps to the coordinator, which launches targeted searches and invokes analysis again until the defined coverage criteria are satisfied.
This is the most effective architectural change because it creates an explicit feedback loop between analysis and search. The analysis agent now has a structured output for evidence gaps (e.g., "sources discuss API authentication but lack details about token-refresh patterns"), which the coordinator uses to launch targeted searches. The iterative cycle continues until coverage criteria are met, ensuring reports have depth on all subtopics without manual intervention. This approach is systematic, auditable, and maintains the division of responsibilities between specialized agents.

Incorrect Options:

A. Add a research-planning agent before the initial search phase to decompose every topic into detailed subquestions.
While better initial planning might help, it cannot anticipate all possible evidence gaps discovered during analysis. The problem is not just planning—it's the inability to act on insights discovered mid-process. Even with exhaustive planning, analysis may reveal unexpected gaps in retrieved sources. This approach still leaves the pipeline sequential and does not address the core feedback-loop deficiency.

B. Have the synthesis agent assign confidence scores to each report section and flag insufficiently supported sections for manual review.
This shifts the burden to human reviewers rather than solving the automation gap. Flagging gaps for manual review defeats the purpose of an automated research system—users will still receive incomplete reports requiring human follow-up. Confidence scores without automated remediation do not improve report depth; they merely document the deficiency. This approach also breaks the 80%+ first-contact resolution target by requiring manual intervention.

D. Have the coordinator look for general gap indicators in the analysis output and run additional searches without repeating the analysis stage.
This assumes that general indicators (e.g., "insufficient evidence") are sufficient to determine what additional searches are needed, which is unrealistic. The coordinator lacks the specific context about what information is missing—only the analysis agent understands the nuances of what was needed but not found. Running searches without specific gap descriptions is likely to retrieve irrelevant or redundant sources. Additionally, skipping the analysis stage means the new sources are never evaluated against the original criteria.

Reference:

Anthropic Claude Agent SDK Documentation – Multi-Agent Coordination Patterns – Recommends feedback loops where agents can signal incomplete work to the coordinator, enabling iterative refinement and targeted follow-up actions.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of ongoing monitoring, feedback mechanisms, and iterative validation to ensure model outputs meet quality standards.

BCBS 239 (Principles for Effective Risk Data Aggregation) – Highlights the need for systems to identify and address data gaps through established feedback and remediation processes, rather than simply flagging deficiencies for manual review.

Explanation:
The agent's behavior—asking for information already provided earlier in the conversation—strongly indicates a loss of conversational context. In the Claude Agent SDK, the agent's state is maintained through the conversation history passed in each API request. If this history is truncated or not passed correctly between turns, the agent will appear to have amnesia about prior exchanges, as each request is treated in isolation.

Correct Option:

B. The conversation history isn't being passed in subsequent API requests.
This is the most likely cause because the Claude Agent SDK relies on the full conversation history being included in each API call to maintain context. If the history is not properly persisted and passed back in subsequent requests, the agent loses all memory of previous turns. The behavior described—asking for the customer's name again after three verification questions—is the classic symptom of stateless request handling where each interaction is treated as a fresh conversation without prior context.

Incorrect Options:

A. The prompt lacks instructions telling Claude to remember information across multiple exchanges.
Claude does not need explicit instructions to remember information within a conversation—this is the default behavior when the full conversation history is provided. Memory is a function of the input context (the conversation history), not of prompt instructions. Adding memory-related prompts would not solve the problem if the history itself is not being passed.

C. The verification tool is clearing the agent's internal state after each successful validation step.
The verification tool (get_customer, lookup_order, etc.) does not control the agent's conversational state—the agent's state is managed by the SDK and the conversation history passed in API requests. Tools are stateless functions that return data; they cannot clear the agent's memory of previous exchanges. This explanation misunderstands the separation between tool execution and conversational memory.

D. Claude's memory retention is limited to two conversational turns by default, requiring explicit configuration to extend it.
This is incorrect. Claude has no such artificial limit on memory retention. As long as the full conversation history is provided within the context window (which is 200K+ tokens), Claude retains all information from the entire conversation. The default behavior is to retain the full history, not limit it to two turns. No explicit configuration is needed to extend memory beyond any default.

Reference:

Anthropic Claude Agent SDK Documentation – Conversation Management – Emphasizes that the conversation history must be passed in each API request to maintain state; the SDK provides built-in mechanisms for history persistence.

Anthropic API Documentation – Messages API – Clarifies that each request is stateless and requires the full conversation history to be provided for the model to maintain context across multiple turns.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of audit trails and state management in automated decision-making systems to ensure consistency and transparency across interactions.

Explanation:
The core issue is that Claude lacks clear, project-specific guidance on what constitutes a reportable finding versus a trivial or acceptable deviation. Developers are dismissing technically accurate but low-value findings because they don't align with team standards. Before adding infrastructure complexity, the simplest and most effective fix is to provide explicit criteria in the prompt that distinguish between issues that matter and those that can be skipped.

Correct Option:

A. Add explicit criteria defining which issues to report, such as bugs and security defects, and which issues to skip, such as minor style preferences and accepted local patterns.
This directly addresses the root cause by giving Claude clear, actionable guidelines. By explicitly defining "reportable" categories (bugs, security defects) and "skip" categories (minor style, accepted local patterns), you align Claude's output with developer expectations. This prompt-design change requires no infrastructure investment, is immediately testable, and preserves the detection of genuine issues while filtering out noise at the source.

Incorrect Options:

B. Implement a secondary classification model that filters Claude's findings according to predicted developer acceptance.
This adds significant infrastructure complexity without addressing why Claude generates dismissible findings in the first place. A secondary model introduces additional latency, cost, and maintenance overhead. More importantly, it's a post-hoc filter that doesn't prevent Claude from generating low-value findings—it only hides them after the fact, wasting tokens and compute on issues that will never be used.

C. Ask Claude to rate every finding's confidence from 1 to 10 and include only findings rated 8 or higher.
Confidence scores do not correlate with relevance or actionability. A finding about a minor style violation might receive high confidence (the code does violate the style rule) but still be dismissed as "not worth addressing." Conversely, a complex security issue might receive lower confidence but be highly valuable. This approach also adds token overhead for scoring and doesn't address the distinction between valuable and trivial findings.

D. Append instructions telling Claude to "only report findings you are highly confident are genuine problems."
This is vague and subjective—what constitutes a "genuine problem" to Claude may not match your team's definition. Claude might still report minor style preferences if it interprets them as "problems." This instruction also risks suppressing important but nuanced findings where confidence is moderate, potentially missing genuine bugs. Without explicit criteria, this change is unlikely to meaningfully reduce dismissal rates.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends providing explicit, concrete criteria rather than vague instructions to guide Claude's judgment on what to include or exclude.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of clear specifications, validation criteria, and documented standards to ensure model outputs meet quality and risk management requirements.

Software Engineering Code Review Best Practices – Highlights that effective code review processes define clear criteria for what issues should be flagged, distinguishing between mandatory fixes (bugs, security) and optional suggestions (style preferences).