Quality Assurance (QA)

In the Analytics section, you can get a high level overview about how your live CSR's are performing. Let's go through each of these one-by-one.

Inbound Calls: The total inbound calls that occurred in the selected date range.

Missed Opportunities: The total amount of qualified leads that did not result in a booking for the selected date range.

True Abandoned: The total amount of abandoned calls (disconnects, hang-ups, etc.) that occurred in the selected date range.

Booking Rate: The total percentage of qualified leads that were booked in the selected date range.

Top Missed Opportunities shows you an overview of any high-value calls that did not result in a booking within the selected date range. You can click on View Call to be taken directly to the call details and analysis so you can get a better idea of what may have prevented the job from being booked.

Call Flow Summary gives you a visual overview of the call outcomes for the selected date range. Hover your mouse over each of the color coded sections to see the total calls and percentages.

Score Analytics gives you an overview of the percentage of pass/fails for each of your scoring metrics applied to calls within the selected date range. You can click on each of these to quickly view a list of all calls that the pass or fail score was applied to. *Note: Total evaluations may vary for each of these in rare cases that certain metrics either wouldn't apply to a particular call or the AI failed to properly evaluate the metric.

Top Tags: Totals of your top custom tags applied to calls within the selected date range (you can create and customize these tags in Setup).

Top Unbooked Call Reasons: Totals of your top unbooked call reasons applied to calls within the selected date range. *Note: Call reasons are pulled from ServiceTitan, and you can customize these in your Broccoli Dashboard > Settings > Custom Instructions > Call Reason Instructions.

Call Trends shows you a visual day by day total of booked, unbooked, and excused calls to easily identify any trends. Hover over each section of the bar graph to see totals for each.

Agent Performance gives you a breakdown of each of your live CSR's score, booking rate, booked calls, booked score, unbooked calls, and unbooked score for the selected date range so you can easily see who is performing well. Click on Coach to navigate to the QA Coach.

The QA Coach is where you can view insights for each of your individual CSR's within the selected date range. Here you can see similar metrics to the above, but on an individual basis.

In addition, you can also view an individuals top 3 performing metrics, top 3 metrics for improvement, and other metrics. Hover your mouse over each of the metrics to see specific values.

Top 3 Best Performing Metrics: The top 3 highest performing metrics for an individual CSR within the selected date range.

Top 3 Metrics for Improvement: Here you can see the top 3 areas where an individual CSR may need work. You will see specific calls that contributed to these metrics beneath. Click on Coach to be taken to the specific call.

Other Metrics: An overview of all other metrics that were graded within the selected date range.

In the History section of your QA feature you can see all your previously graded calls within the selected date range, similar to the Calls section of your inbound CSR agent.

You can filter calls by status as well as any tags that are applied. It is good practice to leave notes in the review section, and mark calls 'reviewed' for easy reference in the future.

Clicking on any call will open the call details where you can view the recording, summary, transcript, and scoring metrics applied to the call.
​

*Note: If you have feedback regarding QA, click on the 'Feedback' option at the top of the call details to submit your feedback directly to our team!

Statuses, Call Reasons, and Tags are applied to the call by the CSR in ServiceTitan. These are not generated by Broccoli.

Occasionally you may see a call with a status similar to the following:

This is a suggested change to the status that is recommended by the AI after scoring the call, which will happen if the AI feels that the original 'Unbooked' status applied by the CSR may have not been correct because the call may not have qualified as a legitimate lead. In these cases, you are able to make the final decision by clicking on the call details and choosing to either 'Accept' or 'Reject' the suggested change.

If you feel the AI suggestions don't necessarily align with what your business would typically consider a qualified lead, you can customize this by navigating to your Broccoli Dashboard > Settings > Custom Instructions > Lead Detection Instructions and adding an instruction to specify what may or may not qualify as a lead.

At the bottom of the Call Details you can see the Call Scoring and which of the metrics were passed or failed during the call. Hover over the question mark to view the pass or fail reasoning provided by the AI. If you disagree with the score, you can also click on the pencil icon to manually change it.

Click on Coach to navigate to the Coach Feedback section where you can view the recommended coaching items to go over with your CSR for the call.

In this section, you can view the Coaching Feedback recommended by the AI for any individual call, as well as the call score.

This allows you to easily go over the recommended coaching items with your CSR's during your next coaching session, and you can mark each item as Done to indicate that coaching has been completed with the CSR.

In the Strengths section, you can easily review and reference the areas where the agent performed well. Clicking Play will start the recording during that section of the call.

In Areas for Improvement, you can easily review and reference the areas where the agent may need to work on improving their performance.

In this section, you can easily set up and configure your QA feature to start grading your live CSR's calls.

Keep in mind that QA will not be active unless you toggle the QA Settings to 'Live.' If you ever would like to pause your QA for any reason, you can toggle this off.

Add any call types (these are pulled from ServiceTitan) that you would like Broccoli to include when grading your calls.

Customize the following options:

Add your CSR profiles (these are pulled from ServiceTitan). Include any CSR's you would like to be graded with QA.

Broccoli's built in Call Metrics should work well out of the box. But if you would like to customize these further or add additional metrics for scoring, you can do so here:

If you would like any custom tags to be added to QA calls, you can customize this here: