Assistant quality
Open in Guilda WorkHow many queries the assistant answers, how many it holds back and why.
Updated on
- Management
- Tax review
- Legal assistant
The Quality tab of the Legal assistant (only for the office's management and reviewers) shows how the assistant has behaved in your office. It includes the team's questions, which is why not everyone sees it.
What it does not measure
It does not measure whether the answers are legally correct: that is decided by a professional's review. It measures how the assistant works: how much it answers, how much it holds back, with which model and how long it takes.
The figures
Choose the period (7, 30 or 90 days). At the top you will see four figures: the queries, the percentage answered with sources, the percentage with insufficient evidence and the median time.
Why queries go unanswered
A query that goes unanswered can have four causes, counted separately here:
- There were no sources enabled that matched the question. If this happens a lot, coverage is missing: check the Sources tab.
- The answer did not pass the server's citation check: the model proposed something the sources did not support and it was discarded. The most common reasons appear below.
- More details about the case were needed, and the assistant asked for them.
- Technical failures.
You will also see how many queries needed more than one model call (repairs), the average number of calls per query and the time within which 95 % of queries are resolved.
The latest unanswered
The Latest queries without a grounded answer list lets you open each one and see its cause, to decide whether another source, another model or training the team to phrase the question better is needed.
Guilda Work