The evaluation tool in the AI development toolkit for Business Central is used to assess the quality and accuracy when extending Copilot in Business Central via prompt dialogs, or when creating custom Business Central agents, which recently launched in private preview. It provides a structured way for developers to validate AI capabilities in Business Central. Developers define datasets containing prompts and expected responses, and the tool automatically runs the evaluation and scores the results.

The tool was previously named "AI Test Tool" but has now been integrated directly into the AI Development Toolkit, and renamed to Evaluation.

Apart from a rename, in this release we are adding the ability to:

Manage configurations
Test with multi-languages datasets
Understand Copilot credits consumed
Test custom Business Central agents

The following sections cover these in more detail:

Configurations

As a step towards making it easier to manage AI quality at scale, we’re introducing new suite configuration parameters. Evaluation suites can now include configuration parameters such as capability, test type, and execution frequency. These settings apply to all tests within the suite, helping teams maintain consistency and structure across large AI evaluation scenarios. They’re also extensible and exposed through APIs for automation to pick up on.

In this example, the capability and run frequency is set.

Example of configuring capabilities and run frequency

And is exposed in the exported suite.xml to be used, for example, in a pipeline instead of interactively in the client UI.

<Root>   <AITSuite Code="DEMO" Description="Demo" TestRunnerId="130451"   Capability="Sales Order Agent" Frequency="Daily" TestType="Agent">     <Line CodeunitID="133540" Description="Tests" Dataset="SET" />   </AITSuite> </Root>

Languages

Business Central solutions are global, and AI experiences must work across languages. Thus, the evaluation tool now supports multilingual datasets. Developers define a single logical dataset entry and then add language-specific variants using language tags. This allows the same scenario to be tested across multiple languages, ensuring consistent Copilot and agent experiences worldwide.

In the evaluation setup, you can define the selected language to use in the supplied dataset.

Example of setting language for the test run

and you can also view and configure the dataset.

Example of creating dataset with tests in different languages

Copilot credit usage

AI features consume Copilot credits, and to understand usage, the evaluation tool now provides visibility into Copilot credit consumption. Developers can see how many credits are consumed per run, per test line, and per dataset entry. This transparency helps teams understand AI consumption, optimize prompts, and manage costs more effectively.

Example of tracking Copilot credit consumption for AI feature tests

In the YAML dataset file, there's now support for a name and a language attribute.

name: DATA language: en-US tests: - name: Case 1   query: Example query   test_setup: Setup.yaml - name: Case 2   query: Example query   test_setup: Setup.yaml

and the evaluation suite XML file will also contain the language tags.

<Root>   <AITSuite Code="DEMO" Description="Demo" TestRunnerId="130451">     <Language Tag="da-DK" Frequency="Weekly" />     <Language Tag="en-US" Frequency="Weekly" />     <Line CodeunitID="50000" Description="Tests" Dataset="DATA" />   </AITSuite> </Root>

Business Central agents

Finally, you can build and validate Business Central agents that you have built. You can select which agent to run in evaluations, inspect agent tasks per run, line, and dataset entry, and use the agent in your evaluations.

In your test, you can get the agent via the agent test context, e.g.,

  procedure GetAgentDuringTest(var AgentUserSecurityID: Guid)     var         AgentTestContext: Codeunit "Agent Test Context";     begin         AgentTestContext.GetAgentUserSecurityID(AgentUserSecurityID);     end;