This is the Trace Id: 4ae34008ec62cc8e83c331397d20e64c
5/19/2025

Indiana Pacers use Azure AI Foundry to create first live arena captioning service

The Pacers Sports & Entertainment (PS&E) technical team needed a custom, live-game captioning system that could deliver instantaneous captions to its deaf, hard-of-hearing, and non-English-speaking fans on arena screens and in its mobile app.

Using Microsoft Azure AI Foundry and Azure Functions, the organization developed a real-time captioning solution tailored to the pace and personality of live basketball.

This live, in-arena, and mobile app solution now delivers instant captions with an error rate of just 1.14%, setting the stage for future AI-powered fan experiences.

Pacers Sports and Entertainment

At Pacers Sports & Entertainment (PS&E), the fan experience isn’t just a priority—it is an obsession. From its founding in 1967 as one of the first professional basketball teams in the nation to today, the Pacers franchise has never stopped raising the bar on how to deliver thrilling live-game experiences to its fans. This time, it was the arena tech development team pushing the envelope, taking on a challenge no other NBA or WNBA arena had solved: how to make the full energy of the game accessible to fans who are deaf, hard of hearing, or non-English speakers.   

“We love our fans and have always prioritized giving them an immersive arena experience,” says Carl Ceresoli, CTO, Strategic & Executive Perspective at PS&E. “We wanted to try something we knew was within our reach—use leading-edge technology to create a super exciting, memorable live-game experience to meet the needs of our fans.” 

The opportunity was clear, but delivering on it required more than just vision. It involved solving for speed, precision, and real-time performance—without sacrificing the personality that makes live basketball commentary so electric.

Turning accessibility into a competitive edge

The pace of basketball, the personality of the announcers, the layered audio environment of a live arena—none of it fit the limits of traditional speech-to-text tools. The team needed a custom solution. 

“The system had to follow fast, unscripted announcers, recognize game-specific lingo and player names, and still deliver captions instantly to keep fans engaged in real time. It was a bold challenge,” says Jared Chavez, Technical and Model Development Lead at PS&E. 

Azure AI Speech within Microsoft Azure AI Foundry made it possible to deliver real-time captions with the speed and accuracy required for a live game-day experience.

“The system had to follow fast, unscripted announcers, recognize game-specific lingo and player names, and still deliver captions instantly to keep fans engaged in real time. It was a bold challenge.”

Jared Chavez, Technical and Model Development Lead, PS&E

The PS&E team turned what could have been a complex, multi-surface rollout into a streamlined implementation that exceeded expectations. A key factor was how Azure AI Foundry interoperated easily with its existing infrastructure, allowing the team to deliver quickly without disrupting their development workflows.

The initial rollout included near-instantaneous English captions across the Pacers and Fever mobile apps and arena screens at Gainbridge Fieldhouse. Spanish support followed soon after, and the team is continuing to expand, bringing in 12 additional languages and laying the groundwork for even broader global access.

With the models trained and tuned for accuracy, the focus shifted to delivering captions in real time—directly to fans. Timing and synchronization remain critical: captions must appear on both mobile and arena screens instantly, with no perceptible delay.

Captions are delivered through an event-driven architecture using Azure Functions and Azure SignalR Service. This real-time setup syncs both the app and in-arena displays from a single source, ensuring smooth playback with virtually no lag.

Building a custom captioning engine with Azure AI Foundry

To deliver real-time captions that could match the speed and style of live basketball, the development team needed to train custom speech models. Azure AI Speech in Azure AI Foundry provided a highly secure, containerized environment where the team could experiment, test, and refine their models without risking production stability. This ensured that any changes or updates to the models were thoroughly vetted in a controlled setting before being deployed to production. 

To fine-tune their Azure AI Speech model, developers built a custom training dataset using real game audio paired with curated announcer phrases. They incorporated hundreds of hours of Pacers and Fever broadcasts to help the model learn each announcer’s cadence, slang, and signature calls. Structured text files tagged with player, coach, and official names helped improve transcription accuracy, ensuring the model could recognize fast-paced, domain-specific language in real time.

“Originally, the audio file for a single game took us upwards of five days to manually cut the game audio,” says Chavez. “Now, using our custom script, it takes us roughly two minutes to cut and encode a game and unload for translation.”      

For live transcription and translation, the team used Azure AI Speech to process announcer audio in real time. By tailoring the model to the Pacers’ broadcast style, they reduced the speech-to-text transcription error rate by 87%—a level of precision that made it possible to extend captioning from mobile apps to arena screens with confidence. 

The following architecture diagram shows how the solution turns announcers’ audio to live text by transcribing and translating, using Azure AI Speech in Azure AI Foundry. Azure Functions then processes the data, and Azure SignalR Service helps ensure real-time playback device output. 

Real-time performance, tuned for live-game intensity 

With the models trained and tuned for accuracy, the focus shifted to delivering captions in real time—directly to fans. The challenge for the developers at this juncture was making sure the timing and synchronization of the displayed captions appeared on both mobile and arena screens instantly, with no perceptible delay.

Captions are delivered through an event-driven architecture using Azure Functions and Azure SignalR Service. This real-time setup syncs both the app and in-arena displays from a single source, ensuring smooth playback with virtually no lag. 

“Because everything flows through a single endpoint, it’s a huge upgrade over what you typically see with TV closed captioning,” says Chavez. Built-in moderation filters also helped prevent inappropriate or misinterpreted content from reaching the screen, reinforcing PS&E’s commitment to accessibility, trust, and a high-quality fan experience.

Behind the scenes, the project offered valuable lessons for any team looking to bring AI into production quickly and responsibly.

Developer insights for fast, flexible AI builds

Although the project started with the goal of improving accessibility, it quickly pivoted to delivering production-ready AI solutions under real-world constraints. For PS&E, the experience offered clear takeaways for any team building AI into customer-facing scenarios, especially in high-pressure, live environments.

One of the biggest lessons: a perfect roadmap is not needed to get started. The PS&E team leaned into fast feedback loops, automation, and real-time performance monitoring, cutting manual audio processing time from five days to two minutes with a custom-built script—a 99.97% time reduction for model training. Azure AI Foundry and Microsoft transcription services gave them the tools to iterate quickly and safely, testing new audio inputs, retraining models, and fine-tuning accuracy without slowing down game operations.   

“We’re proud to pioneer one of the most advanced AI implementations in the league. This experience welcomes fans who previously couldn’t fully engage—and it’s built on a foundation that turns our developer wish lists into reality.”

Jared Chavez, Technical and Model Development Lead, PS&E

The PS&E team also learned the value of starting small and scaling with confidence. Using custom speech inside the Azure AI Speech service, the team built up a training library of game clips, announcer phrases, and structured text entries. Then they used built-in analytics to validate progress. The result was a measurable improvement in model performance, including increased accuracy in detecting player names, sponsor mentions, and real-time dialogue. With each iteration, the team pushed well past their original 10% error rate goal, landing at just 1.14%.  

Just as critical was the decision to architect for change from the start. Azure Functions and a serverless SignalR implementation gave PS&E a flexible, future-ready architecture for pushing real-time captions across mobile apps and arena screens with virtually no lag. By using a standard WebView, the team kept integration simple, paving the way for future capabilities like sentiment tracking and personalized fan experiences.

Arena and developer impact  

Today, fans at Gainbridge Fieldhouse can follow every call, every play, and every courtside moment—thanks to real-time captions that deliver speed, accuracy, and accessibility. What began as a bold idea became a blueprint for how live entertainment can be more immersive and intelligent.

“We’re proud to pioneer one of the most advanced AI implementations in the league. This experience welcomes fans who previously couldn’t fully engage—and it’s built on a foundation that turns our developer wish lists into reality,” says Chavez.

The captioning system now supports Pacers, Fever, and All-Star games, and is built to scale across future arena experiences, from concerts and comedy shows to family events. Future iterations will unlock next-gen capabilities like sentiment analysis and personalized fan engagement. 

With Azure AI Foundry for training, Azure AI Speech for transcription, and Azure Functions for delivery, PS&E didn’t just close a gap—it redefined what accessible, AI-powered fan engagement can look like.

Discover more about the Indiana Pacers on Facebook, Instagram, X/Twitter, and YouTube.

Take the next step

Fuel innovation with Microsoft

Talk to an expert about custom solutions

Let us help you create customized solutions and achieve your unique business goals.

Drive results with proven solutions

Achieve more with the products and solutions that helped our customers reach their goals.

Follow Microsoft