Almanac

Consultant-focused KB for Microsoft Dynamics 365 Contact Center: implementation notes, gotchas, and configuration decisions beyond the official docs — across voice and digital channels, routing, agent and supervisor experience, Copilot & AI, workforce engagement, analytics, administration and security.

2 versions · feature-quality-evaluation-agent.md
v2🤖 markdown-importlast weekmarkdown import
… 10 unchanged lines …
- Conversations: Contact Center standalone only; Cases: Customer Service only
- Quality score KPI in Agent Insights dashboard only populates if Quality Evaluation Agent is active
- Bulk evaluation available for cases (retroactive assessment of historical data)
+ - **Knowledge sources in criteria (preview):** a criterion can link one published Dynamics 365 knowledge article, and up to two questions per criterion can be grounded in it. The agent scores against the latest version of the article and cites it in the evaluation summary. It needs Dataverse Copilot indexing and knowledge sources switched on in Copilot Service admin center
+ - Knowledge-grounded criteria work for the **case** entity and **on-demand** evaluations only, can't be simulated, and only accept Dynamics 365 knowledge articles
+ - Questions can be marked **Not Applicable**: they drop out of the score and the total is calculated from the applicable questions only
+ - **Regenerate evaluation summary** is a separate switch in the quality evaluation settings, alongside scoring and the threshold
## When to use / skip
Use on any deployment where the client wants automated QA, consistent scoring, or quality-based KPI tracking. Essential if Agent Insights quality score is in scope. Skip if there's no QA process to digitise, the criteria design requires investment from the QA team.
… 8 unchanged lines …
- **Prerequisites checklist is the common failure.** All three (connection references, Power Automate flows, Copilot Studio agent) must show Ready. Clients enable the toggle and assume it's done: verify all three before testing.
- **Scoring is permanent.** Discuss threshold with the client first. Enabling with a poor threshold means living with it or complex remediation.
- **"Enable AI agents" is in Power Platform admin, not D365.** If Quality Evaluation Agent won't activate, check this setting: easy to miss.
+ - **Knowledge grounding has hard edges.** One article per criterion and two questions per criterion, no simulation, cases and on-demand only. Design the criteria around those limits before promising "scored against our policy library".
+ - **Not Applicable changes the arithmetic.** A call with several N/A answers is scored on fewer questions, so scores aren't directly comparable between interactions unless the QA team understands that.
- **Compliance obligation:** Reps must know interactions may be monitored, recorded, and scored. Legal requirement in many jurisdictions: build into project governance.
## Consultant notes
… 4 unchanged lines …
---
- *Source last updated: 2026-04-30 | Check this after email evaluation exits preview or new record types are added*
+ *Refreshed 2026-10-24 for knowledge sources in evaluation criteria, Not Applicable scoring and summary regeneration. Check this after email evaluation or knowledge-grounded criteria leave preview, or new record types are added.*
v1🤖 markdown-import3 months agomarkdown import
Initial version.