CCAR-P Practice Questions

Yes, Yes, No, Yes, No

More capable reasoning configurations generally require greater computation, produce higher latency, and may consume larger token or rate-limit allocations than lighter alternatives. This makes the first statement generally accurate, although actual performance must be measured under the organization’s specific provider and service tier.

Representative workload testing is essential because generic benchmark rankings do not establish performance on the organization’s prompts, data, edge cases, tools, or required output format. Anthropic recommends benchmark tests specific to the use case, using actual prompts and data, followed by comparison of accuracy, quality, edge-case behavior, and cost. Choosing the Right Model

The strongest model is not automatically the correct production choice. Model selection must balance capability, speed, cost, throughput, and operational constraints. Tiered routing implements this principle by serving predictable requests with an efficient model and escalating difficult cases to a more capable model. Anthropic documents multi-model strategies in which lower-cost workers handle most traffic while harder decisions are escalated.

Recency is also not a valid selection criterion. A newly announced model must pass the workload’s existing evaluation, security, latency, and cost gates before replacing a validated production model.

Study Guide references/topics: Model trade-offs; representative evaluations; tiered routing; escalation architectures; cost–capability optimization; model migration validation.

===============

The retrieval pipeline is stated to be unchanged, so the first investigation should isolate changes in generation behavior and grounding instructions. Replaying the failing cases on the previous model version determines whether the regression follows the model upgrade or current model-prompt combination. Inspecting the system prompt establishes whether source-use rules, uncertainty behavior, citation requirements, or prohibitions against unsupported claims were removed or weakened. Increasing context or replacing the embedding model changes a component for which no failure evidence exists and may introduce additional noise. Lowering temperature may reduce variation but does not explain systematic contradiction of retrieved evidence. The investigation should compare identical requests, retrieved passages, prompts, and model versions before applying a fix. Define success criteria and evaluations

Lifecycle continuity depends on preserving validated knowledge and feeding operational learning back into subsequent phases. Carrying the evaluation framework and reference set forward, as stated in Option B, creates a stable baseline across prompt changes, model migrations, retrieval adjustments, and architectural revisions. Rebuilding the evaluation system each time would undermine longitudinal comparison because changes in the test framework could be mistaken for changes in solution performance.

Option D establishes the second essential continuity mechanism: lessons from discovery, design, implementation, deployment, and production monitoring become explicit inputs to the next phase. This closes the feedback loop and prevents recurring defects, invalid assumptions, and operational findings from being lost at organizational handoffs.

Option A is overly mechanical. Stakeholders should receive communications relevant to their responsibilities and decision rights, not indiscriminate notifications at every transition. Option C confuses comprehensive archival with lifecycle continuity; retention must follow business, regulatory, security, and records-management requirements rather than an unconditional “archive everything” policy. Option E is directly contrary to iterative architecture. Decisions should be documented and governed, but material evidence or changed requirements must be allowed to reopen them.

Study Guide references/topics: Lifecycle phases; evaluation continuity; reference datasets; feedback loops; lessons learned; decision records; iterative architecture governance.

===============

Function-level or section-level chunking for code modules — Code or Hierarchical Specifications

Tree-aware chunking that follows code or specification hierarchy — Code or Hierarchical Specifications

Per-record chunking where each record is one chunk — Heterogeneous Short Records

Semantic chunking along clause or paragraph boundaries — Long Structured Documents

Fixed-size chunking with overlap for short records of similar length — Heterogeneous Short Records

Hierarchical chunking that mirrors document section structure — Long Structured Documents

Chunking must preserve the structural unit that carries meaning in the source corpus. Code and hierarchical specifications are best divided at function, module, class, or tree boundaries because arbitrary token cuts can separate definitions from their implementation or parent context. Heterogeneous short-record collections should generally preserve each record as an independent chunk. Fixed-size overlapping chunks are also effective when records have broadly similar lengths and lack meaningful internal hierarchy. Long structured documents benefit from semantic boundaries such as clauses and paragraphs, while hierarchical chunking preserves relationships among sections, subsections, and parent headings. Anthropic notes that chunk size, boundaries, and overlap materially affect retrieval performance; therefore, one universal chunking method is inappropriate. Anthropic Contextual Retrieval

A controlled prompt experiment must begin with a falsifiable hypothesis and a predefined primary metric. Option C prevents the team from examining results first and then selecting whichever metric makes the candidate look successful. The metric might measure task accuracy, rubric score, citation validity, escalation rate, latency, cost, or another criterion directly connected to the hypothesis.

Option A determines whether the experiment can detect a practically meaningful improvement. The minimum detectable effect expresses the smallest difference worth acting upon, while the power calculation determines the required sample size. Without this step, the experiment may be too small to detect a real improvement or unnecessarily large and expensive.

Random assignment should then distribute representative traffic between the control and candidate prompts while controlling model version, retrieval configuration, tool availability, and other confounding variables.

Options B and D occur after data collection. Option E follows the completed analysis and decision. The team should also define significance thresholds, stopping rules, guardrail metrics, exclusion criteria, and treatment of repeated observations before launch.

Study Guide references/topics: Prompt A/B testing; hypothesis definition; primary metrics; minimum detectable effect; statistical power; random assignment; decision sequencing.

===============