QA Suites

divinci.qa evaluates a Release against a suite of test prompts using one or more score generators. Use it to catch regressions before you ship a configuration change.

Creating a suite

const suite = await divinci.qa.createSuite("ws_123", {
  name: "Support regression",
  description: "Core support questions",
  scoreGenerators: [
    { source: { /* scorer config */ }, weight: 1 },
  ],
});

Listing and updating

const suites = await divinci.qa.listSuites("ws_123");
const one = await divinci.qa.getSuite("ws_123", suite._id);
await divinci.qa.updateSuite("ws_123", suite._id, { name: "Support regression v2" });
await divinci.qa.deleteSuite("ws_123", suite._id);

Running a suite

Run a suite against a Release; results include per-test scores and the overall delta versus the previous run.

const result = await divinci.qa.run("ws_123", suite._id, {
  releaseId: "rel_abc123",
  concurrency: 4,
  maxTests: 100,
});

console.log(`${result.passedCount}/${result.testCount} passed`);
console.log("overall:", result.overallScore, "delta:", result.delta);

for (const test of result.results) {
  console.log(test.prompt, test.overallScore, test.scores);
}

Score generators

List the available scorers (graders) you can attach to a suite:

const scorers = await divinci.qa.listScoreGenerators("ws_123");
for (const s of scorers) {
  console.log(s.id, s.name, s.description);
}

Copying suites between workspaces

const copy = await divinci.qa.copySuiteFrom("ws_target", {
  sourceWhitelabelId: "ws_source",
  sourceSuiteId: "suite_abc",
  newSuiteName: "Imported suite",
});
console.log(copy.testsCopied);

Method reference

Method	Description
listSuites(workspaceId)	All suites in a workspace.
createSuite(workspaceId, options)	Create a suite.
getSuite(workspaceId, suiteId)	Fetch a suite.
updateSuite(workspaceId, suiteId, updates)	Update a suite.
deleteSuite(workspaceId, suiteId)	Delete a suite.
copySuiteFrom(workspaceId, options)	Copy a suite from another workspace.
run(workspaceId, suiteId, options)	Run a suite against a Release.
listScoreGenerators(workspaceId)	Available scorers.

Closely related: generate QA tests directly from content with trainingData.exportToQA.