Building an AI Voice Agent for 360 Degree Feedback: How Your360 AI Uses State Machines, Evals, and Layered Synthesis

Listen to this episode on: Spotify | Apple Podcasts

What does it take to build an AI voice agent that people will talk to for 44 minutes about their own blind spots? Your360 AI found out, one call at a time.

In this episode of Just Now Possible, Teresa Torres talks with Jared Goralnick (Co-Founder and CEO), Eda Goksel Jackson (Head of Design), and Alex Krasun (AI Engineer) at Your360 AI. Their product brings interview-based 360 degree feedback, once reserved for the C-suite, to anyone. A voice AI coach named Tam runs the self-discovery call, interviews each feedback provider, and walks the recipient through their report.

They get specific about what it took: replacing a single prompt with state machines so Tam doesn't lose the thread, moving from Retell to LiveKit to control every layer of a cascading voice stack, and running AI evals with personas built from calls that failed. Then comes the hard part: synthesis that is accurate, corroborated, and genuinely anonymous.

You'll hear how they balance observability against sensitive data, why reasoning models are a poor fit for voice, and how structural and substantive eval passes catch both outright mistakes and missed patterns.

Show Notes

Guests

• Jared Goralnick, Co-Founder and CEO, Your360 AI
• Eda Goksel Jackson, Head of Design, Your360 AI
• Alex Krasun, AI Engineer, Your360 AI

Key Takeaways

• They built the product one call at a time. The self-discovery agent shipped and was validated a month before feedback collection existed, and synthesis came after that.
• A single prompt couldn't hold a structured conversation. State machines keep the agent on track while still letting it go deep when someone wants to.
• Voice AI is not a survey. People answer three questions in their first sentence, so the agent has to track what has already been covered and decide when to push and when to move on.
• If a customer won't let you look at conversations in the early days, that tells you something about the market. Find design partners who will, then build automated flags so humans only step in when something goes wrong.
• Synthesis needs two kinds of evals. Structural passes catch "don't mess up" errors like misattributed feedback. Substantive passes re-run the analysis with few rules to find patterns the first pass missed.

In this episode

• How interview-based 360 feedback differs from annual reviews and survey-based developmental 360s
• The three calls with Coach Tam: self-discovery, feedback collection, and the 360 report debrief
• Why the first debrief call broke a 60-minute platform limit, and why the median debrief is now 44 minutes
• Using state machines to manage transitions between strengths, growth areas, and requested areas
• Evals built from personas based on failed calls, used to regression-test model and provider upgrades
• Automated flagging on sentiment, customer value, and product quality instead of routine transcript review
• Usability testing a voice call: screen recordings plus follow-up interviews that compare perception with what actually happened
• Moving from Retell to LiveKit for full control over speech-to-text, the LLM, and text-to-speech
• Model tradeoffs for voice: personality vs. time to first token, and why reasoning models get in the way
• Optimizing feedback collection calls for completion and low friction, including calls over the phone
• Anonymity rules: corroboration required for critical feedback, groups shown only at three or more providers
• Progressive reveal in the debrief so recipients process one theme at a time
• Delivering feedback through each company's values and leadership principles
• Organizational heat maps for interdependent teams and independent cohorts
• What's next: turning awareness into action between a person's first and second 360

Resources & Links

• Your360 AI
• LiveKit – the open-source voice orchestration layer Your360 AI moved to
• Retell AI – the voice platform they started on
• Hogan Assessments – the interview-based assessment Teresa compares it to
• Deepgram and ElevenLabs – speech-to-text and text-to-speech layers in a cascading voice stack
• Fearless Feedback by Rebecca Glenn – the coach who helped shape the report format

Chapters

00:00 Meet the Team
01:35 What 360 Feedback Means
04:11 Who Uses Your 360
05:33 Origin Story and AI Fit
08:25 How the 360 Works
13:55 Making Voice Feel Human
22:18 Testing and Evals
23:43 Privacy and Observability
27:28 Building in Stages
32:33 Usability Testing Calls
34:31 Testing Call Perception
35:12 State Machines for Flow
36:58 Flexible Transitions Tuning
38:27 Owning the Voice Stack
40:42 Model Tradeoffs and Costs
42:28 Feedback Interviews Constraints
44:49 Synthesis Goals and Anonymity
49:11 Progressive Report Delivery
52:02 Pattern Based Summaries
54:25 Multi Pass QA Framework
59:46 Org Level Insights Lenses
01:02:45 Why Voice Beats Surveys
01:05:58 Roadmap Awareness to Action
01:07:30 Closing Thanks

Full Transcript

Podcast transcripts are only available to paid subscribers.