Free CCAR-F Practice Test Questions 2026

Explanation:
The model can identify completely untested functions but fails at finer-grained branch coverage because it lacks explicit guidance to perform systematic verification. The solution must provide clear, structured instructions that force the model to enumerate all conditional paths and map them to test assertions, without adding pipeline complexity like multi-pass systems.

Correct Option:

B. Add explicit instructions requiring Claude to enumerate every conditional branch and exception path, then verify that each path has a corresponding test assertion.
This is the most effective approach because it directly addresses the gap detection problem through prompt engineering alone, requiring no infrastructure changes. By instructing Claude to systematically enumerate all conditional branches (if/else, switch/case, try/catch) and exception paths, then cross-reference each against the test file, you force the model to perform branch-level coverage analysis. This structured methodology transforms a vague "check coverage" task into a concrete, verifiable checklist, dramatically improving detection without pipeline overhead.

Incorrect Options:

A. Interleave the implementation and tests in the prompt, presenting each function immediately before its test cases.
While this might help with local context, it does not force the model to systematically check branch coverage. Without explicit instructions to enumerate and verify branches, the model may still read the code and tests without performing the necessary cross-referencing. Interleaving alone does not change the model's analytical approach to the coverage problem.

C. Implement a two-pass pipeline in which one model call extracts all conditional branches and another cross-references them against test assertions.
This adds significant pipeline complexity (two API calls, orchestration logic, result merging) without providing clear benefit over a well-crafted single-pass prompt. The two-pass approach also introduces latency and cost overhead. The same systematic enumeration can be achieved with explicit instructions in a single call, making this solution overengineered for the problem.

D. Include few-shot examples showing code with an uncovered branch and the corresponding review comment identifying the missing test case.
Few-shot examples are helpful but insufficient for ensuring comprehensive branch coverage. Examples demonstrate the desired output format but do not force the model to perform systematic enumeration of every branch in every function. Without explicit instructions to enumerate and verify all branches, the model may still miss branches that don't match the pattern shown in the examples.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends using explicit, structured instructions to guide Claude through multi-step reasoning tasks, especially for verification and analysis workflows.

Testing Best Practices – Branch Coverage – Emphasizes that branch coverage requires systematic identification of all conditional paths and explicit validation that each path is exercised by test cases.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Highlights the importance of systematic validation processes and explicit verification criteria to ensure model outputs meet quality standards.

Explanation:
False positives arise because the model lacks examples that illustrate the boundary between acceptable patterns (given your team conventions and deployment context) and genuine issues. Few-shot examples enable the model to learn the reasoning heuristics for distinguishing these categories, allowing it to generalize to novel patterns not explicitly covered in the prompt.

Correct Option:

B. Include few-shot examples in your prompt showing annotated code snippets that distinguish acceptable patterns from genuine issues in each category.
This is the most effective approach because examples teach the model the decision boundary rather than just listing rules. By providing concrete annotated snippets—e.g., "This pattern is safe in our deployment context because X, but this similar pattern is a genuine security issue because Y"—the model learns the contextual reasoning needed to generalize. Unlike exhaustive rule lists, few-shot examples help Claude recognize that the same pattern may be acceptable or problematic depending on context, enabling accurate judgment on novel code.

Incorrect Options:

A. Implement post-processing that uses keyword matching to filter out findings containing terms such as "convention," "context-dependent," or "trade-off."
This is a crude filter that will miss nuanced false positives and may filter out genuine issues that legitimately involve conventions or trade-offs. Keyword filtering cannot distinguish between a false positive where the pattern is acceptable and a genuine issue where the same terminology appears. This approach adds complexity without improving the model's reasoning capability.

C. Create a comprehensive written specification of all patterns that should not be flagged, and then include the full documentation in the system prompt.
This is impractical because the universe of acceptable patterns is unbounded—you cannot enumerate every safe pattern. A comprehensive specification would be too long for the context window and would still fail to cover novel patterns not anticipated. Rules also lack the contextual nuance that examples provide; the model may struggle to apply abstract rules to concrete code.

D. Add instructions to your system prompt to "be conservative," "only flag definite issues," and "consider that some patterns may be intentional."
These instructions are vague and subjective. Claude's interpretation of "conservative" or "definite" may not align with your team's standards, and this approach does not provide the concrete, contextual guidance needed to distinguish acceptable patterns from genuine issues. Without examples illustrating the boundary, the model may still produce the same false positives or, worse, become overly conservative and miss genuine issues.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends using few-shot examples to teach nuanced decision boundaries, as examples are more effective than abstract rules for contextual judgment tasks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of validation and calibration using representative examples to ensure model outputs meet quality standards and reduce false positives.

NIST AI Risk Management Framework – Highlights that AI systems should be trained and validated using diverse, representative examples to ensure generalization and reduce systematic errors.

Explanation:
The recall trade-off occurs because a single prompt with mixed examples forces the model to divide its attention across multiple concern types, diluting focus on each category. Adding examples for one category creates interference that reduces performance on others. The solution is to separate concerns into focused prompts, each optimized for its specific domain, and then aggregate results.

Correct Option:

A. Split the review into separate focused prompts—one for security and API design and another for business logic—each with dedicated examples, and then consolidate the findings before posting.
This is the most effective approach because it eliminates cross-category interference. Each prompt is now narrowly focused with dedicated examples, allowing the model to apply specialized reasoning without distraction. Security/API design receives its own optimized prompt with relevant examples, while business logic gets its own prompt with logic-bug examples. This separation enables each prompt to achieve higher recall in its domain. Consolidation before posting ensures developers receive a unified, actionable review.

Incorrect Options:

B. Replace the few-shot examples with a detailed checklist of specific logic edge cases to verify, such as division by zero in score calculations and boundary conditions in grading thresholds.
While checklists can help, they are less effective than examples for teaching nuanced detection. More importantly, this approach does not address the root cause—cross-category interference in the single prompt. The security/API design categories would still compete for attention, and logic recall might improve modestly while API-design recall continues to suffer from the diluted focus.

C. Upgrade to a more capable model tier because its stronger reasoning will handle both concern types in one prompt and eliminate the recall trade-off.
This is an expensive and unproven assumption. Even the most capable models suffer from context dilution when asked to perform multiple complex reasoning tasks simultaneously. Upgrading models does not guarantee elimination of the trade-off; it may shift the balance but not resolve the fundamental interference problem. Without addressing prompt architecture, you may incur higher costs without achieving the desired recall improvements.

D. Provide the full repository as context instead of only the changed files and surrounding code, giving the model deeper visibility into business-logic patterns.
While additional context might help business-logic detection, it does not address the cross-category interference problem. More context also increases token usage and latency, potentially introducing new challenges without fixing the core architectural issue. The model would still need to divide attention across security, API design, and business logic within a single prompt, maintaining the trade-off.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends splitting complex tasks into focused prompts, each optimized for a specific domain, to reduce interference and improve overall performance.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of targeted validation and specialized approaches for different model outputs to ensure quality and reliability.

Software Engineering – Modular Design Principles – Highlights that separation of concerns (e.g., specialized review categories) improves quality by allowing focused attention on each aspect rather than attempting to address all concerns simultaneously.

Explanation:
The inconsistency stems from the model lacking a clear, standardized representation of how to format material information. The schema defines the field but does not specify normalization rules for variations like "cotton blend" vs. "Cotton/Polyester mix." Few-shot examples teach the model the expected format and normalization conventions, providing concrete patterns to follow.

Correct Option:

D. Add few-shot examples showing two or three complete input-output pairs with standardized material-description formats.
This is the most effective approach because examples teach the model the specific formatting and normalization conventions you expect. By showing standardized outputs (e.g., always "Cotton/Polyester" instead of "Cotton/Polyester mix"), you provide concrete patterns that the model can generalize to new inputs. Few-shot examples are particularly effective for formatting consistency because they demonstrate the desired level of detail, terminology normalization, and field presence rules in context.

Incorrect Options:

A. Set the temperature to 0 to eliminate randomness and ensure deterministic outputs.
While temperature 0 reduces randomness, it does not address the underlying ambiguity about what constitutes a correctly formatted material description. The model may still produce variations like "cotton blend" vs. "Cotton blend" or omit the field entirely, as the core issue is normalization and standardization, not stochastic variation. Temperature controls creativity, not formatting consistency when the expected format is underspecified.

B. Switch to a more capable model tier because inconsistent extraction indicates insufficient model capability.
This is an expensive assumption. Formatting consistency is not primarily a capability issue—even the most capable models need clear guidance on formatting conventions. A less capable model with good examples can outperform a more capable model with ambiguous instructions. Upgrading models without addressing prompt design is unlikely to resolve the inconsistency and increases operational costs.

C. Make the materials field required instead of optional in the schema to force the model to always extract a value.
Making the field required does not address the formatting inconsistency—the model may still extract different formats or populate the field with incorrect or partially extracted information. This approach could actually increase errors by forcing extraction in ambiguous cases, leading to hallucinated or inaccurate material descriptions rather than consistent, accurate ones.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends using few-shot examples to teach formatting conventions, normalization rules, and output structure, especially for extraction and structured output tasks.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of standardization and consistency in data extraction processes to support downstream integration and ongoing monitoring.

JSON Schema Best Practices – Suggests that schema validation ensures structure but does not enforce content normalization; additional guidance (examples, rules) is needed for consistent formatting of field values.

Explanation:
The 5% failure rate occurs because the model lacks exposure to the full variety of document structures where required information can appear. While strict schemas enforce output structure, they do not teach the model how to locate information when it appears in non-standard formats or sections. The solution must provide examples that demonstrate the necessary flexibility in information location.

Correct Option:

B. Add few-shot examples demonstrating extractions from documents with varied structures, showing how to identify citations in different formats and locate methodology details across section types.
This is the most effective approach because it directly addresses the root cause by teaching the model to recognize required information regardless of where or how it appears in the source document. Examples showing citations in inline format, footnotes, and bibliographies, as well as methodology in dedicated sections, introductions, or embedded in other sections, train the model to be flexible in its information location strategy. This improves generalization without requiring brittle regex patterns or post-processing.

Incorrect Options:

A. Implement retry logic that resends requests when validation detects empty required fields.
Retrying without changing the prompt or providing additional guidance is unlikely to succeed. The model will make the same errors repeatedly because it lacks the knowledge of where to find the information in varied document structures. Retry logic simply wastes tokens and increases latency without addressing the underlying pattern recognition failure.

C. Build a regex-based post-processing layer that scans source documents for citation patterns and methodology keywords, populating empty fields when the model fails to extract them.
Regex-based extraction is brittle and cannot handle the wide variety of citation formats and methodology presentations found in real-world documents. This approach duplicates the model's task but with less intelligence, and may introduce incorrect or out-of-context data. It also adds significant pipeline complexity and maintenance overhead without improving the model's ability to extract information.

D. Modify the schema to make citations and methodology optional, and flag incomplete records for manual review instead of failing validation.
This accepts the 5% failure rate rather than addressing it, shifting the burden to manual reviewers. Making required fields optional reduces data quality and breaks downstream integration expectations (systems expecting complete records). This approach masks the problem without solving it and increases operational costs through manual review.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends few-shot examples with varied input structures to teach models to handle diverse formats and edge cases.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of robust validation and handling of varied data formats to maintain model accuracy and reliability.

Data Extraction Best Practices – Highlights the need for flexible extraction strategies that can handle varied document structures rather than rigid approaches that assume consistent formatting.

Explanation:
The 3% failure rate is due to documents exceeding the model's context window. The most cost-effective solution is to process only the failed documents with a chunking strategy. This avoids reprocessing the 9,700 successful documents and addresses the root cause (excessive document length) through segmentation, where each chunk stays within the context limit.

Correct Option:

D. Resubmit only the 300 failed documents after chunking them into smaller pieces, and then combine the partial extractions.
This is the most cost-effective approach because it targets only the 3% of documents that failed, avoiding unnecessary reprocessing costs for the 9,700 successful documents. Chunking directly addresses the context_length_exceeded error by breaking each long document into smaller segments that fit within the context window. While combining partial extractions requires some additional logic, this is a one-time implementation that handles the failure scenario efficiently without recurring cost overhead.

Incorrect Options:

A. Resubmit the entire 10,000-document batch using a model tier with a larger context window.
This is extremely costly and inefficient, as 97% of the documents would be reprocessed unnecessarily. Switching to a more expensive model tier for the entire batch (instead of only the 300 failures) increases costs significantly without providing any benefit for the documents that already succeeded. This approach ignores the cost-efficiency principle of handling only the failures.

B. Reprocess the entire batch with prompt caching enabled to reduce the cost of retrying requests with identical system prompts.
While prompt caching reduces token costs, reprocessing all 10,000 documents still incurs substantial costs and latency for documents that already succeeded. Prompt caching is a general optimization, not a targeted solution for context-length errors. The caching benefit is marginal compared to the cost of reprocessing 9,700 successful documents, making this approach inefficient.

C. Increase the max_tokens parameter for the 300 failed documents and resubmit them in a new batch.
The context_length_exceeded error occurs because the input document exceeds the model's context window, not because the output (max_tokens) is too long. Increasing the max_tokens parameter controls output length, not input capacity. This approach misunderstands the error and would not resolve the failures; the same documents would still exceed the context window on resubmission.

Reference:

Anthropic API Documentation – Error Handling – Clarifies that context_length_exceeded errors indicate the input exceeds the model's context window; the solution is to reduce input length through chunking or truncation.

Anthropic Prompt Engineering Best Practices – Handling Large Documents – Recommends chunking long documents into smaller pieces and combining results for extraction tasks, as this is more cost-effective than upgrading to larger context models for all requests.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of cost-effective risk management and targeted remediation, focusing resources on failures rather than reprocessing successful operations.

Explanation:
The 30% false positive rate stems from the model lacking awareness of your team's established conventions, architectural patterns, and internal tooling context. The model applies generic best practices without understanding that certain patterns are intentionally accepted in your codebase. The solution must provide project-specific context that overrides generic heuristics for these known acceptable patterns.

Correct Option:

D. Document the team's accepted patterns and intentional conventions in the project's CLAUDE.md file so the model receives this context during every review.
This is the most effective approach because CLAUDE.md provides persistent, project-wide guidance that Claude will reference for every review. By documenting explicit exceptions with contextual explanations (e.g., "force-unwrapping is acceptable in test files because test failures should crash fast and clearly"), you teach the model to recognize and accept these intentional patterns. This eliminates false positives at the source without requiring post-processing or manual filtering, and the guidance applies consistently across all future reviews.

Incorrect Options:

A. Build post-processing keyword filters that suppress findings containing terms such as "force unwrap," "large class," or "deprecated import" before results reach developers.
This is a crude, keyword-based approach that will filter out legitimate findings that happen to mention these terms in other contexts. For example, a genuine bug involving incorrect force-unwrap usage in production code would be incorrectly filtered out. This approach masks the symptom without addressing the model's lack of contextual understanding and risks missing real issues.

B. Configure the review to analyze only the changed lines in the diff without surrounding file context, reducing the amount of code the model evaluates during each review.
This does not address the false positive problem—the model will still flag the same intentional patterns based on the changed lines alone. Reducing context may actually worsen accuracy, as the model will have less information to understand the intent behind patterns. This approach also undermines the review's ability to catch issues that require broader context.

C. Have developers add inline suppression comments at flagged lines and preprocess diffs to exclude suppressed lines before sending code to the model.
This shifts burden to developers to manually annotate every intentional pattern in the codebase, which is unsustainable and defeats the purpose of automation. Developers would need to add suppression comments preemptively or retroactively, creating significant overhead. Preprocessing also risks excluding lines that contain both intentional patterns and genuine issues in adjacent code.

Reference:

Anthropic Claude Documentation – CLAUDE.md Best Practices – Recommends documenting project-specific conventions, exceptions, and context to guide Claude's behavior and reduce false positives in code review.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of context-aware model validation and the need to incorporate domain-specific knowledge to reduce false positives and improve output reliability.

Google's Code Review Guidelines – Highlights that effective code reviews should be context-aware, respecting team conventions and established architectural patterns rather than applying generic rules uniformly.

Explanation:
The team needs an efficient way to apply the checklist consistently during reviews while avoiding manual repetition. The solution should make the checklist easily accessible for review sessions without interfering with other tasks like feature development or debugging. The approach should also support the team's existing workflow of using Claude for multiple purposes.

Correct Option:

A. Create a /review slash command containing the checklist, invoked when starting reviews.
This is the most effective approach because slash commands provide an on-demand, task-specific activation mechanism. Developers can simply type /review to trigger the checklist workflow, making it easy and consistent to apply during review sessions. The slash command encapsulates the checklist so it doesn't interfere with other use cases like debugging or feature development. This maintains the team's existing workflow while streamlining the review process.

Incorrect Options:

B. Create a dedicated review subagent with the checklist embedded in its configuration.
While a dedicated subagent could work, it adds unnecessary architectural complexity for what is essentially a prompt-engineering need. Subagents are better suited for tasks that require specialized tool access or isolation from the main agent's context. The checklist alone does not require a separate subagent—a slash command or CLAUDE.md section would be simpler and more maintainable. This approach over-architects the solution.

C. Add the checklist to the project's CLAUDE.md file under a "Code Review" section.
CLAUDE.md provides persistent context across all interactions, which means the checklist would be included even when developers are using Claude for debugging or feature development (tasks where the checklist is irrelevant). This would consume context window space unnecessarily and could distract the model during unrelated tasks. The checklist should be applied only during reviews, not universally across all use cases.

D. Configure plan mode as the default for code review sessions.
Plan mode controls whether Claude executes actions directly or proposes a plan first—it does not address the need to apply a specific checklist during reviews. This approach misunderstands the purpose of plan mode and would not help the team consistently apply the 8-item checklist. Plan mode is orthogonal to the checklist delivery mechanism.

Reference:

Anthropic Claude Code Documentation – Custom Slash Commands – Recommends using slash commands to encapsulate task-specific instructions, making them easily invocable and reusable without affecting other use cases.

Anthropic CLAUDE.md Documentation – Advises using CLAUDE.md for universal, always-applied context rather than task-specific instructions that should only apply in certain scenarios.

Software Engineering Workflow Best Practices – Emphasizes task-specific automation that integrates seamlessly into existing workflows without interfering with other tasks or adding unnecessary complexity.

Explanation:
The synthesis agent misinterprets temporal differences as contradictions because it lacks the necessary metadata (publication dates, data-collection periods) to understand the chronological relationship between data points. The solution must provide temporal context as structured metadata from the source agents, enabling the synthesis agent to interpret the data correctly as a trend or growth over time rather than a contradiction.

Correct Option:

A. Require subagents to include publication dates and data-collection periods in their structured outputs.
This is the most effective approach because it provides the essential temporal metadata that the synthesis agent needs to interpret the data correctly. By including publication dates and data-collection periods in the structured output from both the web-search and document-analysis agents, the synthesis agent can recognize that 2024 data and 2021 data represent different points in time. This enables it to interpret the figures as a growth trend (18% to 35% over three years) rather than a contradiction, producing a more accurate and nuanced synthesis.

Incorrect Options:

B. Configure the web-search agent to return only results published during the previous six months.
This is too restrictive and would miss valuable historical context needed for trend analysis. It also does not solve the core interpretation problem—the synthesis agent would still misinterpret temporal differences if it encounters them in other contexts. This approach limits the system's ability to produce comprehensive reports that show growth patterns, which are often the most valuable insights in research.

C. Add a conflict-resolution agent that automatically discards older data whenever a newer value exists for the same metric.
This discards valuable historical context and would incorrectly treat all older data as irrelevant. In many research contexts, historical data is essential for showing trends, growth, and context. Simply discarding older data would produce misleading reports that lack important temporal context and could miss significant patterns or anomalies.

D. Instruct the synthesis agent to treat the newest value as authoritative and place all older findings in a separate historical section.
While this is better than discarding older data, it still incorrectly elevates the newest value as "authoritative" without considering that different studies may use different methodologies, samples, or definitions. The correct approach is to present all data with their temporal context and let the synthesis agent interpret the relationship, rather than imposing a rule that the newest is always the most relevant.

Reference:

Anthropic Multi-Agent Coordination Best Practices – Recommends structured outputs with rich metadata (including dates, sources, and methodology) to enable downstream agents to perform accurate synthesis and reasoning.

BCBS 239 (Principles for Effective Risk Data Aggregation) – Highlights the importance of data lineage, including timestamps and collection periods, to ensure accurate interpretation and aggregation across different sources.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the need for transparent metadata (including data provenance and collection periods) to support ongoing monitoring and interpretation of results.

Explanation:
The synthesis agent misinterprets temporal differences as contradictions because it lacks the necessary metadata (publication dates, data-collection periods) to understand the chronological relationship between data points. The solution must provide temporal context as structured metadata from the source agents, enabling the synthesis agent to interpret the data correctly as a trend or growth over time rather than a contradiction.

Correct Option:

A. Require subagents to include publication dates and data-collection periods in their structured outputs.
This is the most effective approach because it provides the essential temporal metadata that the synthesis agent needs to interpret the data correctly. By including publication dates and data-collection periods in the structured output from both the web-search and document-analysis agents, the synthesis agent can recognize that 2024 data and 2021 data represent different points in time. This enables it to interpret the figures as a growth trend (18% to 35% over three years) rather than a contradiction, producing a more accurate and nuanced synthesis.

Incorrect Options:

B. Configure the web-search agent to return only results published during the previous six months.
This is too restrictive and would miss valuable historical context needed for trend analysis. It also does not solve the core interpretation problem—the synthesis agent would still misinterpret temporal differences if it encounters them in other contexts. This approach limits the system's ability to produce comprehensive reports that show growth patterns, which are often the most valuable insights in research.

C. Add a conflict-resolution agent that automatically discards older data whenever a newer value exists for the same metric.
This discards valuable historical context and would incorrectly treat all older data as irrelevant. In many research contexts, historical data is essential for showing trends, growth, and context. Simply discarding older data would produce misleading reports that lack important temporal context and could miss significant patterns or anomalies.

D. Instruct the synthesis agent to treat the newest value as authoritative and place all older findings in a separate historical section.
While this is better than discarding older data, it still incorrectly elevates the newest value as "authoritative" without considering that different studies may use different methodologies, samples, or definitions. The correct approach is to present all data with their temporal context and let the synthesis agent interpret the relationship, rather than imposing a rule that the newest is always the most relevant.

Reference:

Anthropic Multi-Agent Coordination Best Practices – Recommends structured outputs with rich metadata (including dates, sources, and methodology) to enable downstream agents to perform accurate synthesis and reasoning.

BCBS 239 (Principles for Effective Risk Data Aggregation) – Highlights the importance of data lineage, including timestamps and collection periods, to ensure accurate interpretation and aggregation across different sources.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the need for transparent metadata (including data provenance and collection periods) to support ongoing monitoring and interpretation of results.

Explanation:
The recurring error pattern involves informal measurements—the model attempts to convert them to precise values or omits them entirely. The corrected extractions reveal the correct behavior: extract informal measurements verbatim rather than converting or omitting. The most efficient way to encode this learning is through few-shot examples that demonstrate the desired behavior for this specific edge case.

Correct Option:

B. Add few-shot examples to your prompt demonstrating correct handling of informal measurements—extracting them verbatim rather than converting or omitting them.
This is the most effective approach because few-shot examples directly teach the model the correct behavior for this specific edge case. By showing examples where "a handful" is extracted as "a handful" (not "1 cup" or omitted), the model learns the correct handling pattern. This approach requires minimal infrastructure changes, is immediately testable, and leverages the 847 corrected extractions as a training set for prompt examples without the complexity and cost of fine-tuning.

Incorrect Options:

A. Fine-tune the model on the 847 corrected extractions.
Fine-tuning is overkill for this scenario and would require significant infrastructure, expertise, and cost. It also requires careful dataset preparation and model hosting, adding operational complexity. Few-shot examples are far more efficient for addressing specific, well-defined edge cases like this one. Fine-tuning is better suited for broad, systemic improvements rather than targeted pattern fixes.

C. Implement a post-processing layer that uses pattern matching to detect informal measurement phrases in source text and automatically populate values when the extraction is empty.
Pattern matching is brittle and cannot handle the variety of informal measurement expressions ("a handful," "a splash," "a pinch," "to taste," etc.). This approach also duplicates the model's task with less intelligence and would require maintaining an ever-growing list of patterns. Moreover, it would only work when the extraction is empty, not when the model invents a specific amount—a pattern-matching fix would miss the invention case.

D. Update your JSON schema to add a "measurement_type" enum field (precise/informal).
While adding metadata can help downstream systems process data appropriately, it does not teach the model how to extract informal measurements correctly. The model would still need to know to extract "a handful" verbatim and set the measurement_type to "informal." Without examples demonstrating the expected extraction behavior, the schema change alone does not improve extraction accuracy.

Reference:

Anthropic Prompt Engineering Best Practices – Recommends using few-shot examples to teach models specific edge-case handling, especially when errors follow recurring patterns identified through human review.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of incorporating human feedback and validation results into model improvement cycles to address identified weaknesses.

Data Extraction Best Practices – Highlights that extraction systems should handle varying input formats and granularity by capturing values as they appear rather than forcing conversions that introduce error.

Explanation:
The core problem is a rigid pipeline that proceeds to report generation despite identified knowledge gaps. The synthesis agent has already identified which three research questions remain unanswered, but the system lacks a mechanism to act on this insight. The solution must create a feedback loop where identified gaps trigger targeted follow-up research before finalizing the report.

Correct Option:

B. Have the coordinator evaluate the synthesis output for gaps, then re-delegate to web search and document analysis with targeted queries before invoking synthesis again.
This is the most effective approach because it creates an explicit feedback loop where identified gaps drive targeted follow-up research. The coordinator, upon receiving gap information from synthesis, can launch precise queries to address the specific unanswered questions. This iterative refinement continues until coverage is complete or the system determines that information is truly unavailable. This maintains the separation of concerns (specialized agents for search, analysis, and synthesis) while enabling adaptive research depth.

Incorrect Options:

A. Increase the initial breadth of queries sent to web search and document analysis to reduce the probability of missing relevant information.
While broader initial queries might reduce gaps, they are inefficient and cannot guarantee coverage of all specific subtopics. Broad queries may also retrieve irrelevant information, increasing noise in the synthesis stage. More importantly, this does not address the need for iteration—even with the broadest initial search, gaps may still be identified during synthesis that require targeted follow-up.

C. Have the report-generation agent note which research questions could not be answered, so users understand the limitations of the final output.
This simply documents the failure rather than addressing it. Users would receive incomplete reports with disclaimers, which does not achieve the goal of comprehensive, cited reports. This approach violates the system's objective of producing complete research outputs and places the burden of handling gaps on users rather than the system.

D. Give the synthesis agent direct access to web-search tools so it can autonomously fill knowledge gaps without returning control to the coordinator.
This violates the separation of concerns and creates coordination complexity. The synthesis agent would need to manage search logic, analysis, and synthesis, which is beyond its specialization. It also bypasses the coordinator's ability to manage agent lifecycle and track progress. This approach would make the system less maintainable and harder to debug, and may lead to the synthesis agent conducting undisciplined searches that degrade output quality.

Reference:

Anthropic Claude Agent SDK Documentation – Multi-Agent Coordination Patterns – Recommends feedback loops where the coordinator evaluates outputs and re-delegates to specialized agents until coverage criteria are met.

SR Letter 11-7 (Federal Reserve) – Model Risk Management – Emphasizes the importance of iterative refinement and feedback mechanisms in automated decision-making systems to ensure completeness and quality.

BCBS 239 (Principles for Effective Risk Data Aggregation) – Highlights the need for systems to identify and address data gaps through established feedback and remediation processes, rather than simply documenting deficiencies.