Admin Lesson 6: Setting Up Triage

Admin Lesson 6: Setting Up Triage

Triage is the first piece of automation we set up after the foundations are in place. It's the highest-leverage thing you can configure, because every ticket your helpdesk receives goes through it. Get triage right and the rest of the system gets cleaner inputs to work with — assignment, routing, queues, lifecycles all key off the values triage assigns.

What triage actually does

Problem: A new ticket arrives in the helpdesk. Its priority is whatever the customer (or the helpdesk's default) said. Its type is unclassified or wrong. Its category is blank. Its assigned queue is whatever default the helpdesk picked. Someone has to read the ticket, decide what it really is, and set the right values — every time, on every ticket. That someone is usually a dispatcher, and it's the most repetitive job in the queue.

Horizon's answer: Triage. As soon as Horizon ingests a ticket, the configured triage policy reads the ticket content, runs it through an AI-powered flow, and recommends values for the fields you care about — Priority, Type, Status, Queue, whatever you've configured. Recommendations get applied back to the helpdesk automatically (or held for review, depending on configuration).

The result: every ticket arrives already classified. Dispatchers stop reading every new ticket. Routing rules and assignment policies have correct field values to key off. The work that used to take a person an hour a day takes the system a few seconds per ticket.

Where triage lives

Open the admin panel and navigate to Components → Triage. You'll find:

PageWhat it does
Triage Policiesthe triage flows themselves — one or more, scoped by conditions
Triage Actionsthe catalog of actions a triage policy can take (set Priority, set Type, set Queue, etc.)
Triage Policy Hintscontext you give the AI to bias its decisions (per company, per customer, per picklist value)
Training Importshistorical tickets imported with known correct classifications, used to train and tune the model
Triage Logsper-ticket audit trail — what the AI saw, what it recommended, what it committed

You'll spend most of your setup time on Triage Policies and Hints. The other pages are reference, training data, and audit.

The triage policy model

A triage policy is made of steps. Each step is one classification — "figure out the priority", "figure out the type", "figure out which queue". Steps run in sequence. Each step:

  • Reads the ticket content (and any output from earlier steps).
  • Optionally reads picklist values from the helpdesk (e.g. "the helpdesk's allowed Priority values").
  • Asks the AI to pick a value, with a confidence score.
  • Optionally applies picklist exclusions — values you've explicitly told the system never to pick.

When all steps complete, the triage policy produces a change set — the proposed field changes for the ticket. That change set is then committed back to the helpdesk via the integration.

Because steps run in sequence and earlier steps' outputs inform later ones, the order matters. Classify type first, then priority can use the type as context. Classify queue last, once everything else is known.

Setting up your first triage policy

A reasonable first triage policy classifies the three highest-leverage fields: Type, Priority, and Queue. You can layer on more later.

  1. Navigate to Triage Policies → Create.
  2. Give the policy a name (e.g. "Default Inbound Triage").
  3. Define the conditions that select which tickets this policy applies to. Most accounts start with a single catch-all policy (no conditions) and add narrower policies later as needed.
  4. Save the policy.
  5. On the edit page, use the Steps relation manager to add the classification steps you want, in the order you want them to run.
  6. For each step, configure:
    • Which field is being classified.
    • Which AI model / triager type (single, pair, text) to use.
    • Any picklist exclusions.

After creating the steps, send a test ticket through. The Triage Logs page will show you exactly what the AI saw, what it picked, and the confidence scores.

Hints — biasing the AI without rewriting the policy

Problem: Your AI triage works well in general but gets a few cases consistently wrong. "All tickets from CompanyX are project work, never service desk." "Anything mentioning Microsoft Defender is always Type=Security." "The 'Triage' picklist value should never be picked because we don't use it."

Horizon's answer: Hints. Three flavors:

Hint typeWhat it does
Customer hintsbias triage decisions for tickets from a specific customer / contact
Company hintsbias triage decisions for tickets from a specific company
Picklist hintsguide the AI on what a specific picklist value means or when to pick it

There are also picklist exclusions at the policy level — values you've told the system to never pick at all, regardless of what the ticket content suggests.

Hints are how you fix triage misclassifications without rewriting the whole policy. Add a hint, send a test ticket, watch the log. If the hint moves the AI toward the right answer, you're done.

Training imports

Problem: Your account has thousands of historical tickets that were already correctly classified by humans. The AI doesn't know about them. You're starting cold.

Horizon's answer: Training imports. Upload (or sync) historical tickets with their known correct classifications. The AI uses them as training signal, so it starts your account with a baseline of how this MSP actually classifies things rather than a generic default.

Training imports are managed under Training Imports. The exact import format and refresh cadence depends on your helpdesk and how Horizon is configured to source the historical data.

You don't have to use training imports to ship triage. Most accounts go live with a baseline policy and a handful of hints, then layer in training data as the model needs to get sharper.

Iterating on triage

Triage isn't fire-and-forget. Plan to spend the first two weeks watching the Triage Logs page after go-live and tightening the policy:

  1. Open Triage Logs.
  2. Spot a misclassified ticket.
  3. Decide if the fix is:
    • A new step (something the policy isn't classifying yet)
    • A reordered step (later step needed to run earlier)
    • A new hint (customer / company / picklist)
    • A picklist exclusion (the system is picking a value it shouldn't)
    • Reworded ticket content / better-trained inbound (out of scope for triage itself)
  4. Make the change.
  5. Send a test ticket. Watch the log. Confirm the fix.

After the first two weeks, the touch frequency drops dramatically. Most mature accounts revisit triage once a quarter at most.

Lifecycle integration

Triage is the first thing in the ticket lifecycle. Every other lifecycle action — auto-assign, scheduled todo creation, routing — happens after triage commits. If triage fails or stalls, the rest of the lifecycle stalls with it. Triage Logs is your first stop when "the system didn't do anything to this ticket" tickets land in your queue.

When this is set up correctly

  • Every new ticket has Type, Priority, and Queue set within seconds of arrival.
  • The Triage Logs page shows >95% commit success rate.
  • Dispatchers report they almost never have to reclassify a ticket by hand.
  • Routing and assignment downstream get clean inputs.

Tip: Don't over-design the first triage policy. Three steps (Type, Priority, Queue) and a catch-all rule is the right starting point for almost every account. You'll add hints, exclusions, and additional steps as you watch real tickets flow through. Triage tuning is observational — you can't predict the edge cases from a whiteboard.

    • Related Articles

    • Admin Lesson 7: Setting Up Scheduled Todos

      With triage running, every new ticket arrives classified. The next layer is the calendar layer — turning classified tickets into committed time on someone's day. That's what scheduled todos do. This lesson covers the admin-side configuration that ...
    • Admin Lesson 1: Welcome to the Admin Seat

      Horizon doesn't run itself. Triage rules, todo policies, assignment plugins, watchlists, forms — these are decisions someone has to make. That someone is you. This lesson sets the frame: what an admin actually does, how the three panels split up the ...
    • Admin Lesson 12: Ticket and Form Lifecycles

      Lifecycles are how everything you've configured so far gets wired together. Triage classifies, assignment picks an owner, scheduled todos land on the calendar, helpdesk actions write back — but something has to decide when each of those happens for a ...
    • Admin Lesson 9: Auto-Assign with Round-Robin

      Triage classifies. Booking links and scheduled todos handle the calendar layer. The next gap is who picks up the ticket. Without auto-assignment, dispatchers manually route every ticket to a specific person — fast at first, then slower as volume ...
    • Admin Lesson 16: Manual vs Automatic — Calibrating Your Account

      You've configured the building blocks. Now the question becomes how aggressively should the system act on its own? Every major piece of Horizon — triage, assignment, todo creation, watchlist transitions, form lifecycles — has a manual mode and an ...