Skip to content

Qwen3.8 hardware · Qwen3.5 edge inference · Gated DeltaNet accelerator · hybrid attention LLM · on-device Qwen

Qwen3.5 to Qwen3.8: Why Hybrid Attention Suits Edge Silicon

Fanshi Semiconductor engineering team · Published

Qwen3.5, Qwen3.6 and Qwen3.8 use a hybrid layout in which three Gated DeltaNet linear-attention layers alternate with one gated full-attention layer. Only the full-attention quarter of the network keeps a KV cache that grows with context; the linear layers carry a fixed-size state however long the prompt is. For edge silicon that means a smaller, more predictable memory budget, and because the published layout has not changed between these releases, a newer Qwen model is a new build on the same inference cores rather than a new architecture.

One layout, three releases

Qwen3.5 was released in February 2026, Qwen3.6 in April and Qwen3.8 in August. Across all three, the dense 27B checkpoint publishes the same language-model layout: 64 layers arranged as 16 repeats of three Gated DeltaNet blocks followed by one Gated Attention block, each with its own feed-forward network. The Gated Attention blocks use 24 query heads and 4 key-value heads with a head dimension of 256; the Gated DeltaNet blocks use 16 query-key heads and 48 value heads with a head dimension of 128. Native context length is 262,144 tokens.

The same 3:1 pattern runs down to the smallest member of the family. Qwen3.5-0.8B has 24 layers arranged as 6 × (3 Gated DeltaNet + 1 Gated Attention), with 2 key-value heads in its attention layers. The 27B checkpoints are natively vision-language models, and every checkpoint mentioned here is published under the Apache 2.0 licence.

Why the layout matters for memory

In a standard transformer, every layer keeps a key-value (KV) cache, and it grows with every token of context. On long prompts the KV cache, not the weights, becomes the memory problem.

A Gated DeltaNet layer works differently. Instead of storing keys and values for every past token, it folds them into a recurrent state of fixed size that is updated as each token arrives. A 1,000-token prompt and a 200,000-token prompt leave behind the same amount of state.

In the Qwen hybrid only one layer in four is full attention, so only a quarter of the network keeps a growing KV cache. Take Qwen3.5-0.8B: its 6 full-attention layers, each with 2 key-value heads of dimension 256, store 2 × 6 × 2 × 256 = 6,144 values per token, about 12 KB at 16-bit precision. An otherwise identical model with all 24 layers on full attention would need four times as much.

What it means for compute-in-memory and near-memory designs

Weights still dominate decode traffic. In a dense model, every generated token reads every weight once whatever the attention type, so the case for compute-in-memory and near-memory compute, moving weights less or not at all, is unchanged.

What changes is the second memory budget. With a conventional transformer, a chip has to be sized for the KV cache of the longest prompt it will ever see. With a hybrid model, three quarters of the layers carry a fixed-size state that can be budgeted once, and only the attention layers grow with context. The memory plan becomes smaller and far more predictable, which is exactly what purpose-built edge silicon needs.

Linear-attention layers also stream naturally: each token updates the state and moves on, without looking back over the whole history. That suits low-latency, long-running workloads such as assistants, agents and continuous sensor streams.

Two layer types, two Fanshi cores

A hybrid model needs hardware that handles both layer types well. Fanshi's FS-Linear core targets linear attention and DeltaNet-style recurrent architectures; FS-Attention targets hybrid-attention language models, including the Qwen3 and Qwen3.5 families. Together they cover the Gated DeltaNet and gated full-attention layers that make up the Qwen hybrid.

Because Qwen3.6 and Qwen3.8 keep the Qwen3.5 layout, supporting a newer release means mapping new weights onto the same cores, not designing a new architecture. On the MaskROM route, where weights are fixed at manufacture, a newer checkpoint is a new build of the chip's weight content while the cores and data path carry over. On the SRAM compute-in-memory and near-memory routes, new weights are loaded rather than cast.

Model size, context length and performance targets for any specific Qwen release are assessed per project. Send the target model and workload to info@fanshi-silicon.com and we will reply with a route recommendation and an evaluation plan.

Sources

Architecture figures are taken from the official Qwen model cards on Hugging Face: Qwen/Qwen3.5-0.8B, Qwen/Qwen3.5-27B, Qwen/Qwen3.6-27B and Qwen/Qwen3.8-27B. Release months are from the citations on those cards. The KV cache arithmetic is derived from the published head counts and dimensions.

Bring the question to your project

Send the target model, performance and power goals. We reply with a route and an evaluation plan.

Request an evaluation kit