NVIDIA enables high-performance HSTU generative recommender deployment

NVIDIA has introduced an end-to-end inference workflow for Hierarchical Sequential Transduction Unit (HSTU) generative recommender systems using NVIDIA Dynamo-Triton. This approach treats recommendation as sequence modeling over user behavior rather than isolated retrieval and ranking stages.

The workflow integrates PyTorch Ahead-of-Time (AOT) Inductor compilation, FlexKV-backed KV caching, and native C++ validation to address the latency challenges associated with long user histories and large embedding tables. In testing on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU, the configuration achieved speedups of up to 4.47x for a three-layer HSTU model and 5.93x for an eight-layer model when compared to AOTI configurations without KV caching.