How Huawei’s Ascend Optimizes Airline Booking Engine Latency

Why Airlines Should Care About AI Latency

Every second counts when a traveler searches for a flight. A slow booking engine can send passengers to a competitor. But behind the scenes, airlines are increasingly turning to large language models to power chatbots, dynamic pricing, and personalized offers. These models are smart, but they’re also slow—especially when they need to coordinate across many chips.

That’s where communication bottlenecks come in. In a MoE (Mixture of Experts) model, the AllToAll communication step alone can eat up more than 30% of the total end-to-end time. And in ultra-long-context scenarios—like reading a full customer history or a 1M-token policy document—the Host to Device (H2D) transfer of KV cache becomes a new bottleneck, directly impacting Time To First Token (TTFT). For airlines, this means the difference between a snappy chatbot reply and a frustrating pause.

Huawei’s 2012 Lab has been tackling exactly this. Their technical expert, Li Baichao, presented at AICon Shenzhen on how they optimized Pangu model training and inference on the Ascend 950 platform. The results: over 10% improvement in MoE AllToAll performance and over 10% faster TTFT in long-sequence inference. Let’s break down what they did and why it matters for aviation tech.

The Real Bottlenecks in AI Communication

Large models aren’t just big—they’re distributed. Training and inference spread across hundreds of chips, and those chips need to talk constantly. In MoE architectures, experts are scattered, and every token must be routed to the right expert. That routing involves an AllToAll collective, where every chip sends data to every other chip. It’s a massive traffic jam.

For airlines using MoE-based models for fare prediction or customer service, that jam means higher latency and lower throughput. And when you add long-context features—like analyzing a traveler’s past 50 trips—the KV cache grows huge. Moving that cache from host memory to the device can take longer than the actual computation.

Huawei found that on their Ascend 950, the H2D transfer for 1M-token contexts was a major TTFT culprit. They also discovered that existing solutions, like DeepEP, didn’t map well to Ascend’s 910A3 topology. So they had to go custom.

Topology-Aware AllToAll: Not One-Size-Fits-All

One of the key insights from Huawei’s work is that communication optimizations are hardware-specific. DeepEP, a popular library for MoE communication, wasn’t suitable for Ascend 910A3. But on Ascend 950, they could leverage the network topology and a dedicated communication accelerator called CCU.

By adapting the AllToAll implementation to the Ascend 950’s topology, they achieved a 10% performance boost in the EP (Expert Parallel) communication domain. That’s not a trivial gain—in a model serving millions of flight queries, a 10% reduction in communication time can translate to a noticeable drop in response times.

The lesson for airlines: don’t assume your AI infrastructure is portable. A solution that works on NVIDIA H20 might actually hurt performance on Ascend or other chips. You need to tune for your specific hardware.

Speeding Up KV Cache Transfer with Omni Cache

For long-sequence inference, the KV cache is the new bottleneck. Huawei’s answer was Omni Cache, a software solution that optimizes H2D and D2H (Device to Host) communication. Combined with a hardware feature—the Ascend 950 provides a dedicated H2D path for each NPU—they managed to cut TTFT by over 10%.

Why does this matter for airlines? Consider a customer service bot that needs to reference a long policy document or a passenger’s entire booking history. That’s a long context. If the KV cache transfer is slow, the bot takes forever to start generating a response. With Omni Cache, the transfer is faster, so the bot replies quicker, keeping the passenger happy.

Huawei’s approach combines hardware and software: the dedicated H2D path reduces contention, while Omni Cache handles the orchestration. This dual approach is a model for how airlines can optimize their own AI stacks.

What Airlines Can Learn from Huawei’s Optimization Strategy

Huawei’s presentation wasn’t just about their own models. They shared a broader framework for communication optimization that any enterprise can apply:

• Identify the bottleneck: Is it AllToAll in MoE? Or H2D in long-context? Measure first.
• Use topology awareness: Understand your network layout and design communication patterns that fit it.
• Leverage dedicated accelerators: If your hardware has special units like CCU, use them.
• Customize your communication operators: Sometimes you need new collective semantics.
• Overlap computation and communication: The goal is to hide communication time behind computation.

For airlines, this means not just buying AI infrastructure, but actually tuning it. A generic deployment might work, but a tuned one can give you a competitive edge.

The Trade-Off: Performance vs. Portability

One honest point Huawei made: their optimizations are specific to Ascend 950. The same strategies fail on other platforms like Ascend 910A2/A3 or NVIDIA H20. In some cases, they even cause performance degradation.

That’s a tough pill to swallow. Airlines often want to avoid vendor lock-in. But if you’re serious about performance, you may need to accept some hardware-specific tuning. The key is to decide what matters more: being able to switch hardware easily, or getting the best possible latency on your current setup.

Huawei’s approach sacrifices generality for performance. They’re betting that the gains are worth it. For airlines with high-traffic booking systems, that bet might pay off.

Practical Steps for Your AI Infrastructure

So what can an airline’s tech team do today? Start by profiling your AI workloads. Identify where communication time is dominating. If you’re using MoE models, focus on AllToAll. If you’re dealing with long contexts, look at KV cache transfers.

Next, consider your hardware’s unique features. Does your accelerator have dedicated communication paths? Are there custom operators you can write to better match your model’s patterns? Huawei’s example shows that even small tweaks can yield double-digit improvements.

Finally, think about orchestration. Can you overlap communication with computation? Huawei’s future work involves fusion operators and multi-stream pipelines to hide communication even further. That’s a direction any AI team can explore.

Looking Ahead: The Future of AI in Airlines

As AI models get bigger and more complex, communication will only become more critical. Airlines that invest in understanding and optimizing their AI infrastructure now will be better positioned to deliver fast, reliable services.

Huawei’s work on Ascend 950 is a case study in how to squeeze out performance. It’s not just about having the best model; it’s about making that model run efficiently in production. For airlines, that efficiency translates directly to customer satisfaction and revenue.

The next time you book a flight, remember the invisible work happening behind the scenes. And if you’re in the tech team, consider how you can apply these lessons to your own systems.