3 versions · feature-quality-and-coaching-skills.md
v3last monthmarkdown import
… 6 unchanged lines …- Indicators and guardrails are reusable across plans, but each one has to be **activated** on the Quality and coaching page before a plan can use it. Build the library once, activate it, reference it everywhere.- Guardrails carry a Low/Medium/High priority and can optionally generate a suggested next response when a violation is detected.- **Conversation sampling** sets the percentage of matching conversations a plan actually evaluates, so you can cover a queue without paying to score every conversation in it.+ - **Case** evaluation plans get a richer sampling model, and only on **Recurring** frequency — the **Sampling** section doesn't appear for conversations or for trigger-based plans. Sampling mode is **Absolute Number** or **Percentage**; absolute values below 1 are rejected. **Selection strategy** takes records from the **Top** or **Bottom** of the list, ordered by a **Filter date type** you choose (Created On, Modified On).+ - **Sample by: Representative** spreads the sample evenly across representatives rather than taking a flat slice. You pick who's in scope by adding an **Owner** condition naming users or a team — at runtime the system resolves team membership and samples across those users.+ - Each evaluation batch run caps at **20,000 records**.+ - Run history exposes the sampling arithmetic in three columns: **Records identified by condition**, **Records identified by sampling**, and **Records eligible for evaluation** — the last being the lower of the first two.+ - An end date on a plan is optional. Left empty, the plan runs until someone deactivates or modifies it.- On-close plans have a **Weights** step: you set how much each quality indicator contributes to the overall score. Real-time plans don't — they use score ranges and actions instead.- Supervisors can trigger evaluations on demand: select closed conversations in the **Closed QAA conversations** view, choose **Request QAA evaluations**, and pick a plan. Useful for scoring conversations a plan's conditions never matched.- Runs on consumption-based billing and capacity (pay-as-you-go) — consumption-billed, not bundled.… 18 unchanged lines …- It bills on consumption and the real-time mode evaluates continuously — point it at every queue and the spend will surprise you. Scope with conditions and sampling from day one.- The views are labelled "QAA conversations" in the workspace even though the admin surface is called "Quality and Coaching". Same engine, two names — don't let that confuse a tenant walkthrough.- Microsoft attaches the standard employee-monitoring caveat to this feature: it's not for employment decisions, and you're on the hook for notifying end users (and representatives) that conversations may be monitored and recorded, and getting consent where the law requires it. Get that signed off before go-live, not after.+ - If the system can't distribute a representative-based sample evenly, it creates **no evaluation batch at all** — the run is recorded in run history with an execution result and nothing gets evaluated. It fails closed and quietly, so a plan can look active for weeks while producing nothing.+ - Sampling fields become read-only once a plan with sampling is active. Changing them means pausing the plan first, which catches people out mid-review-cycle.- Nudges fire inline during a live conversation. If your indicators are noisy or the score ranges are set too tight, representatives get pestered mid-call — tune conservatively and widen out once you trust the scoring.## Consultant notes… 5 unchanged lines …---- *Refreshed 8 Sept 2026 for conversation sampling, on-close weighting and on-demand evaluations — confirm public preview vs GA status in your target tenant, and revisit after the next release wave for billing and configuration changes.*+ *Refreshed 19 Sept 2026 for the case recurring-plan sampling model (absolute/percentage, selection strategy, representative distribution, 20,000-record batch cap) and optional plan end dates — confirm public preview vs GA status in your target tenant, and revisit after the next release wave for billing and configuration changes.*
v2last monthmarkdown import
… 3 unchanged lines …## Key facts- Two evaluation frequencies per plan: **In real-time** (continuous scoring + live nudges) and **On conversation close** (scored once, results after the conversation ends). It's set per plan, not per tenant.- The quality library has two object types: **quality indicators** (scored — empathy, greeting etiquette, etc., each with one or more questions) and **guardrails** (pass/fail behaviour rules — e.g. no financial advice, vulnerable-customer handling — evaluated for violations, not scored).- - Indicators and guardrails are reusable across plans. Build the library once, reference it everywhere.+ - Indicators and guardrails are reusable across plans, but each one has to be **activated** on the Quality and coaching page before a plan can use it. Build the library once, activate it, reference it everywhere.- Guardrails carry a Low/Medium/High priority and can optionally generate a suggested next response when a violation is detected.- - Runs on Microsoft Copilot credits (pay-as-you-go) — this is consumption-billed, not bundled.+ - **Conversation sampling** sets the percentage of matching conversations a plan actually evaluates, so you can cover a queue without paying to score every conversation in it.+ - On-close plans have a **Weights** step: you set how much each quality indicator contributes to the overall score. Real-time plans don't — they use score ranges and actions instead.+ - Supervisors can trigger evaluations on demand: select closed conversations in the **Closed QAA conversations** view, choose **Request QAA evaluations**, and pick a plan. Useful for scoring conversations a plan's conditions never matched.+ - Runs on consumption-based billing and capacity (pay-as-you-go) — consumption-billed, not bundled.- Requires the Omnichannel Administrator role to configure.- Surfaces in Copilot Service workspace as **Closed QAA conversations** and **Ongoing QAA conversations** views, with a Score column and a side pane showing the quality trendline, AI summary, and per-indicator detail.- Representatives see coaching nudges inline in the conversation, and the full nudge history in the **Consult** pane.… 3 unchanged lines …## Configuration decisions- Real-time vs on-close per plan — decide by queue risk, not blanket across the tenant.- - Which conversations each plan covers (conditions on queue / workstream). Scope real-time evaluation tightly to high-value queues to control credit burn.+ - Which conversations each plan covers (conditions on queue / workstream). Scope real-time evaluation tightly to high-value queues to control consumption.+ - The sampling percentage per plan. This is the cheapest lever you have on spend — a 10% sample of a high-volume queue gives you a trend without scoring every conversation.+ - For on-close plans, the weighting across indicators. An unweighted overall score treats a greeting-etiquette miss the same as a compliance failure, which is rarely what the client means.- Score ranges (Critical / Warning / Normal) for each scored indicator, and what each range triggers: supervisor notification, representative nudge, or both.- Whether guardrails generate a suggested next response or just flag the violation.- Which built-in indicators to copy-and-edit versus author from scratch.+ - Whether supervisors get to fire on-demand evaluations, and against which plan — it's an unbudgeted consumption path if nobody sets expectations.## Gotchas- - It bills on Copilot credits and the real-time mode evaluates continuously — point it at every queue and the spend will surprise you. Scope with conditions from day one.+ - It bills on consumption and the real-time mode evaluates continuously — point it at every queue and the spend will surprise you. Scope with conditions and sampling from day one.- The views are labelled "QAA conversations" in the workspace even though the admin surface is called "Quality and Coaching". Same engine, two names — don't let that confuse a tenant walkthrough.- Microsoft attaches the standard employee-monitoring caveat to this feature: it's not for employment decisions, and you're on the hook for notifying end users (and representatives) that conversations may be monitored and recorded, and getting consent where the law requires it. Get that signed off before go-live, not after.- Nudges fire inline during a live conversation. If your indicators are noisy or the score ranges are set too tight, representatives get pestered mid-call — tune conservatively and widen out once you trust the scoring.… 7 unchanged lines …---- *Freshly shipped (MS Learn docs dated June 2026) — confirm public preview vs GA status in your target tenant, and revisit after the next release wave for billing and configuration changes.*+ *Refreshed 8 Sept 2026 for conversation sampling, on-close weighting and on-demand evaluations — confirm public preview vs GA status in your target tenant, and revisit after the next release wave for billing and configuration changes.*
v13 months agomarkdown import
Initial version.