# Agent Nova — Production Test Results

**Audience:** Client stakeholders  
**Environment:** Live Jira project **AN** (`agentnovatech.atlassian.net`)  
**Test date:** 17 September 2026  
**Purpose:** Confirm five recently delivered behaviours work end-to-end in production  

---

## Summary

| # | Capability | Result | Primary proof ticket |
|---|------------|--------|----------------------|
| 1 | Auto move ticket **To Do → In Progress** when Agent Nova starts | **Pass** | Verified on multiple runs (e.g. AN-52–AN-60) |
| 2 | Design **reject → clarify → redesign** loop (no fail on reject) | **Pass** | [AN-52](https://agentnovatech.atlassian.net/browse/AN-52) |
| 3 | **@mention assignee** on every Agent Nova comment | **Pass** | [AN-53](https://agentnovatech.atlassian.net/browse/AN-53) |
| 4 | **Duplicate ticket** detection + human pause | **Pass** | [AN-56](https://agentnovatech.atlassian.net/browse/AN-56), semantic [AN-57](https://agentnovatech.atlassian.net/browse/AN-57) |
| 5 | **Confluence “already documented”** check + human pause | **Pass** | [AN-59](https://agentnovatech.atlassian.net/browse/AN-59), retest [AN-60](https://agentnovatech.atlassian.net/browse/AN-60) |

**Overall:** All five points passed in production after required fixes were deployed (see notes below).

---

## Point 1 — Status moves to In Progress on start

**What we wanted**  
When Agent Nova accepts a labelled ticket, Jira status should move from **To Do** to **In Progress** immediately.

**How we tested**  
Created stories in project **AN** with label `agent-nova` and watched status after the run started.

**Result — Pass**  
Tickets consistently moved to **In Progress** as soon as Agent Nova started (for example AN-52, AN-53, AN-56, AN-59, AN-60).

---

## Point 2 — Design reject → clarification → redesign

**What we wanted**  
If a human rejects the design, the run must **not** fail. Agent Nova should ask for clarification, wait for a reply, produce a new design, and ask approve/reject again (unlimited until approve).

**How we tested** — [AN-52](https://agentnovatech.atlassian.net/browse/AN-52)

1. Waited for design review (Account Service Level).  
2. Commented: `reject — use Contact instead of Account`.  
3. Confirmed clarification ask (run stayed active).  
4. Replied with the corrected requirement.  
5. Confirmed a **new** design (Contact Service Level) and a fresh approve/reject ask.

**Result — Pass**

| Step | Observed |
|------|----------|
| After reject | State: needs clarification — **not failed** |
| After clarification | New Confluence design + design review again |
| Loop | Ready for another approve/reject |

---

## Point 3 — Assignee @mention on AI comments

**What we wanted**  
Every Agent Nova Jira comment should start with a real **@mention** of the ticket assignee (so they get notified), when an assignee is set.

**How we tested** — [AN-53](https://agentnovatech.atlassian.net/browse/AN-53)

1. Created a story assigned to **tester tester**.  
2. Waited for Agent Nova’s first comment (design ready).  
3. Inspected the comment body in Jira’s API for a real mention node.

**Result — Pass**  
Comment included `@tester tester` (account id matched the assignee). Unassigned tickets (e.g. AN-52) correctly had **no** mention.

---

## Point 4 — Duplicate ticket detection

**What we wanted**  
If a new ticket looks like an existing one, Agent Nova should comment with the **original ticket key**, pause for a human decision, then:

- `continue` → proceed  
- `duplicate` / `close as duplicate` → stop  

**How we tested**

| Ticket | Scenario | Outcome |
|--------|----------|---------|
| [AN-56](https://agentnovatech.atlassian.net/browse/AN-56) | Near-duplicate of Industry picklist (AN-53) | Paused as duplicate of **AN-53** → `close as duplicate` → run stopped |
| [AN-57](https://agentnovatech.atlassian.net/browse/AN-57) | **Same meaning, different wording** (“classify companies by sector”) | Still detected as duplicate (confidence ~0.98) |

**Result — Pass** (including semantic / differently worded duplicates)

**Note for client:** An earlier attempt failed because Atlassian removed an old Jira search API. That was fixed and redeployed before the successful AN-56 / AN-57 tests.

---

## Point 5 — Confluence “already documented” check

**What we wanted**  
On intake (after duplicate check), search Confluence. If the feature already appears documented, comment with **doc titles + links**, pause, then:

- `continue` → proceed anyway  
- `already done` / `close` → stop  

**How we tested**

| Ticket | Scenario | Outcome |
|--------|----------|---------|
| [AN-59](https://agentnovatech.atlassian.net/browse/AN-59) | Industry picklist (docs already exist) | After `continue` on Jira duplicate → **DOC review** with Confluence links → `already done` → stopped |
| [AN-60](https://agentnovatech.atlassian.net/browse/AN-60) | Fresh retest of the same flow | Same result: DOC review → `already done` → stopped |

**Result — Pass**  
References cited included existing Industry design pages (e.g. AN-53 / AN-54 / AN-55 technical designs). Confidence was high (~0.98–0.99).

**Note for client:** Search initially missed docs when the full ticket text was used as one search phrase. Keyword search was fixed and redeployed before AN-59 / AN-60 passed.

---

## What this means for day-to-day use

1. **Starting work** — Labelled tickets move to In Progress when Agent Nova picks them up.  
2. **Design feedback** — You can reject and refine designs without killing the run.  
3. **Notifications** — Assignees are @mentioned on Agent Nova comments.  
4. **Duplicate work** — Similar Jira tickets are flagged before build.  
5. **Existing docs** — Matching Confluence documentation is flagged before build.

Human control remains at each pause: continue, reject/redesign, close as duplicate, or mark already done.

---

## Tickets used in this test cycle (reference)

| Key | Role in testing |
|-----|-----------------|
| AN-50 / AN-51 | Early attempts blocked by temporary OpenAI credit/quota issue (resolved before later tests) |
| AN-52 | Point 2 — reject → clarify → redesign |
| AN-53 | Point 3 — assignee @mention |
| AN-56 / AN-57 | Point 4 — duplicate (exact + semantic) |
| AN-59 / AN-60 | Point 5 — Confluence already-documented |

---

## Sign-off

| Item | Status |
|------|--------|
| Five agreed capabilities exercised on production Jira **AN** | Complete |
| Blocking platform issues found during test | Fixed and re-verified |
| Ready for client demo / UAT walkthrough of these five behaviours | Yes |

*Prepared from live production runs on 17 September 2026.*
