What it does

An evaluation plan decides which conversations or cases get a quality evaluation, when, by whom or what, and against which criteria. It's the operational layer on top of the Quality Evaluation Agent: criteria define what good looks like, the plan defines which records get measured against it.

Key facts

• Prerequisites: quality evaluation enabled, the roles and privileges assigned, Copilot credits (pay-as-you-go) set up, and consent given for potential data movement across regions. For cases you also tick Enable bulk evaluations in the Copilot Service admin center before creating the plan.
• Conversations support three frequency types: Trigger (fires when a conversation closes), Recurring (daily, including previously closed conversations) and Once. Cases support Recurring (daily) only, plus a separate trigger-based option for resolved cases.
• Evaluation method is AI assisted, AI agent or Manual. AI agent mode needs every criteria question AI-enabled and doesn't allow manual editing; AI assisted needs at least one AI-enabled question and an assignee (team or user).
• Sampling appears only for case plans with recurring frequency. You choose absolute number or percentage, top or bottom selection, a filter date such as Created On, and optionally Representative sampling to spread records evenly across reps. If the sample can't be spread evenly, no batch is created and the run is only visible in run history.
• Participant evaluated (entire record, current owner, or IVR/bot only) is available for trigger-based plans only.
• Each batch run handles at most 20,000 records. Run history exists for recurring plans only; trigger-based plans produce evaluations you find on the Evaluations page by filtering on plan name.
• Active plans can't be edited. Pause, edit (everything except record type and evaluation criteria), then reactivate. Pausing finishes the current batch; resuming waits for the next scheduled time.
• Evaluation for emails is preview. On-demand evaluation now covers cases, conversations and emails.
• The overall evaluation score is the sum of the section scores out of 100, but questions marked Not Applicable are excluded and the score is calculated from the applicable questions only. The evaluation view shows a breakdown by section and question, including each question's weight and contribution.
• Trigger-based case plans fire on resolution, and a case can collect multiple evaluations if several plans match.

When to use / skip

Use plans for any ongoing QA programme: always-on scoring of closed conversations, daily sampled case reviews, or a one-off baseline. They're the way to get coverage without a supervisor picking records by hand.

Skip recurring plans with no conditions unless you genuinely want every eligible record scored. It costs credits and buries the useful signal.

Configuration decisions

• Trigger or recurring for conversations. Trigger for steady-state coverage of new closures; recurring or once for a backlog or baseline.
• Conditions. Add an exclusion for records that already have an evaluation, otherwise plans re-score the same records.
• Sampling. Pick percentage versus absolute count based on volume, and decide whether representative sampling is worth the risk of skipped runs.
• Evaluation method. Fully AI agent suits high-volume scoring; AI assisted suits calibration while the QA team still trusts the humans more.
• Expiry and owner. Set evaluation expiration days and an owner so unfinished assisted evaluations don't sit forever.

Gotchas

• Sampling fields lock once a plan is active. Pause the plan to change them.
• Run history's "records eligible" is the lower of condition matches and sample size, so a tight filter makes a generous sample irrelevant.
• No run history for trigger plans is by design. Don't tell the client the plan has failed because the tab is empty.
• Overlapping plans on the same cases create multiple evaluations and multiply credit spend.
• Record type and criteria can't be changed after creation, so a wrong choice means a new plan.
• Allowing Not Applicable on a question makes scores harder to compare. Two evaluations of the same criteria can be out of different numbers of questions, so agree upfront whether N/A is allowed and on which questions.

Consultant notes

• Start with one plan per queue, in manual or AI assisted mode, and compare against human scores for a fortnight before going to AI agent only. QA leads need to see the numbers match their instincts first.
• Budget credits from the sampling settings, not from total volume. Clients are usually surprised by how little sampling they need for a meaningful trend.
• Run the first bulk activation outside business hours. Activation waits for the data transfer to complete and people think it has hung.
• Write down who owns pausing and editing plans. It's a Quality Manager task, and it tends to land with whoever happens to be admin.

---

Evaluation features are still gaining options — worth revisiting after email evaluation leaves preview. Refreshed 2026-10-24 for Not Applicable scoring and the score breakdown.