Transcription
& Call Summary

Starting transcription mid-call without breaking the call

Transcription is opt-in — not automatically active, and not all customers want it enabled for all calls. The entry point lives in the call settings menu, which keeps it accessible without surfacing it as a default. When recording and transcription are active, the in-call state indicators confirm it's running — critical for compliance in regulated contexts where participants need to know they're being recorded.

Bringing AI to Meetings with the Sample Builder ↗

Designing for the gap between "call ended" and "summary ready"

Azure AI Language's summarization runs post-call. There's a window — typically 15–30 seconds — where the transcript is finalizing and the summary hasn't generated yet. A user who hits "end call" has an immediate expectation: they're done with the call, they want to see what came from it. A blank screen with a spinner and no timeline is a trust problem. It communicates nothing about what's happening or when it will end.

Summary above transcript. Synthesis before record.

The transcript is the complete record — every word, speaker-attributed, searchable. The summary is the distillation — discussion points, decisions, action items. These serve two different needs at two different times: the summary is useful immediately after the call, the transcript is useful later when you need to verify something specific.

I structured the post-call view with the summary at the top, transcript below. This isn't just visual hierarchy — it's a position on what the primary job of this screen is. Most users, most of the time, want to quickly understand what happened and what comes next. The transcript is the safety net, not the entry point. The structure makes both accessible without forcing a choice.

The Azure AI Language output includes three structured sections: main discussion points, decisions made, and action items. I preserved that three-part structure in the summary view — it gives the output shape and scannability regardless of how long the call was or what domain it came from.

Configurable through the Sample Builder — no AI infrastructure required

The Sample Builder packaged transcription and call summary as toggles in the Azure Portal wizard. A developer, PM, or field team member could enable AI call summary without writing a line of Azure AI SDK code — the Sample Builder wired up the Azure AI Speech and Azure AI Language integrations behind the scenes. This was the "5-minutes-to-wow" motion: pair the AI capability with a frictionless demo experience so customers and internal teams could see it working before committing to a production implementation.

I worked with design and DevRel on a Sample Builder walkthrough video to relaunch the builder on YouTube and anchor the Build conference demo for transcription and meeting summary.

Sample Builder walkthrough video produced for Build and YouTube relaunch.

The same transcript produces a different document in each vertical

Healthcare, financial services, and education were the three verticals I focused on when scoping what a "good summary" meant. For a telehealth appointment, the physician needs to see the patient's reported symptoms, what was decided, and what the follow-up instructions are — in that order. For a financial consultation, the advisor needs decisions made, account references, and compliance-relevant commitments. For an education context, the summary is more about accessibility — giving students a searchable record of a session.

Azure AI Language's three-part structure (discussion points → decisions → action items) maps reasonably well to all three scenarios, but the implementation left room for the Sample Builder to expose configuration options for what gets included and how it's labeled. This gave customers a path toward scenario-specific summaries without requiring the library to hard-code industry-specific logic.