Solved

How to rate the effectiveness / quality of the AI

  • 6 June 2023
  • 7 replies
  • 75 views

Unlike some other AI products, I don’t see a way to rate the answer to a question to note whether the AI worked correctly / effectively or if it missed the mark, in order to hopefully improve the tool moving forward. How would you like us to track and provide you with feedback on the feature as we experiment with it? 

icon

Best answer by Sydney Sundell 6 June 2023, 01:20

View original

7 replies

Userlevel 4

Hi @Tara Davenport ! Great question, this is something we’re currently working on bringing into the Answers experience. If you have any particularly off responses that you see, feel free to share screenshots with us directly in the meantime. 

@Sydney Sundell what’s the best method for sharing screenshots as I have an example where wording the question slightly differently received completely different answers?

 

Yeah it would be great to be able to more easily correct or share the wrong answer within the UI of Answers. There have been some that are very wrong like this -
 

 

Additionally as an admin, I would love to see all the questions and the answers my team was getting.  I have less visibility and no analytics or sentiment to know if users are more easily getting the answers they need. 

Userlevel 2

Yip, had the same question. My first question didn’t reveal any good answer although I knew the content was there. And the second attempt got me an “Answers is currently not available” error message.

Userlevel 4

@Patricia Leighton if you are comfortable sharing screenshots in this group, feel free to do so directly here—if not, please send to bturner@getguru.com and ssundell@getguru.com 

@Renee Lawlor thanks for the example, yes we hear you! We’re quickly working to add an analytics view so you can see where Answers are/aren’t helping your team. More to come soon. 

@Lars Böhnke thanks for letting us know—please feel free to share examples if you’re comfortable! 

Same experience as @Renee Lawlor. It looks like the answer provided was the second holiday on the list, but it lacked the context of knowing that it’s June and therefor the next one would not be in January. It did link the correct card, but the summary reply was way off
 

 

Userlevel 4

Thanks for the example, @Trish Diaz ! Passing along to our data science team 👍

Reply