Quadric Chimera GPNPU 如何支持可变长度提示词
Variable-Length Prompts on Chimera GPNPU
Quadric Chimera GPNPU 支持单一编译图处理从短语音指令到数千 token 的任意提示词长度，最长可达配置容量。传统编译推理栈通常针对固定长度（如 1024 token）运行，短提示词的其余位置靠 padding 计算，需要为不同长度构建多个二进制文件并逐一验证。Chimera GPNPU 只需一个二进制即可覆盖所有长度，减少 padding 浪费和多版本维护成本。
How a flexible binary can serve every length up to the capacity you set.
If you are sizing a language model for an edge product, the prompts it will see span a wide range. A spoken command is a dozen tokens, a pasted document is several thousand, and the same device serves both. Somewhere in the software stack, a decision is made about the gap between the longest prompt you provisioned for and the prompt that actually arrives. That decision sets how many binaries you build and qualify. It also sets how much of every inference is spent computing padding.
Most compiled inference stacks do not make that decision at all. A graph compiled for 1024 tokens processes 1024 positions every time it runs. If you send it 50 tokens, it computes the other 974 as padding. To serve short prompts efficiently as well as long ones, you have to build a separate binary for each length range and qualify all of them. Quadric Chimera GPNPU does not work this way. It gives you one guarantee and one optimization.
Read more here.
Fig.1: One graph serves every length up to the configured capacity. Source: Quadric
来源： Semiconductor Engineering 芯片与封装 · semiengineering.com