<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel>
<title>Fanshi Semiconductor insights</title><link>https://www.fanshi-silicon.com/en/insights/</link><description>Answers from the Fanshi engineering team: compute-in-memory chips, choosing an edge LLM inference chip, inference cost, MaskROM, FPGA prototyping, the memory wall and Qwen hybrid attention.</description><language>en</language>
<lastBuildDate>Mon, 16 Nov 2026 01:00:00 GMT</lastBuildDate><atom:link href="https://www.fanshi-silicon.com/en/feed.xml" rel="self" type="application/rss+xml"/>
<item><title>Model-on-Chip Explained: Hardwired LLM ASICs, Cloud vs Edge</title><link>https://www.fanshi-silicon.com/en/insights/hardwired-llm-chip-model-specific-asic/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/hardwired-llm-chip-model-specific-asic/</guid><pubDate>Sun, 15 Nov 2026 01:00:00 GMT</pubDate><description>A model-specific ASIC, sometimes called a hardwired LLM chip, fixes the weights of one large language model into the silicon at manufacture. Inference no longer moves weights out of memory, so decode speed and energy efficiency can be orders of magnitude better than a general-purpose GPU. The trade-off is that the model cannot be swapped freely. There are two kinds: cloud chips chase tokens per second per rack and cost per token, while edge chips chase watt-level power, offline operation and device cost.</description></item>
<item><title>LLM Inference ASICs Compared: Five Chip Architectures in 2026</title><link>https://www.fanshi-silicon.com/en/insights/llm-inference-asic-comparison/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/llm-inference-asic-comparison/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>LLM inference hardware now falls into five architectures: general-purpose GPUs, wafer-scale engines, language processing units, transformer-only ASICs and model-on-chip designs that fix one model into silicon. They differ in how much flexibility is given up for speed. Reported speeds for the same 8B model range from about 230 tokens per second per user on an NVIDIA H200 to about 17,000 on a hardwired Taalas HC1, but the figures mix vendor claims with independent measurements and are not directly comparable. All five are data-centre products; devices need a different class of chip.</description></item>
<item><title>Tokens per Second Explained: How Fast Does an LLM Need to Be?</title><link>https://www.fanshi-silicon.com/en/insights/tokens-per-second-explained/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/tokens-per-second-explained/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>Tokens per second is the rate at which a language model generates output. A person reads at roughly 5 to 7 tokens per second, so a chat answer feels fluid at 10 to 30. Reasoning models, agents and control loops need hundreds to thousands, because most of the tokens they generate are never read by a person. On a device, the ceiling is set by memory bandwidth divided by model size, which is why edge chips try to stop moving weights at all.</description></item>
<item><title>How to Run an LLM Without a GPU: Five Hardware Options for Devices</title><link>https://www.fanshi-silicon.com/en/insights/run-llm-without-gpu-edge-hardware/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/run-llm-without-gpu-edge-hardware/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>A language model can run without a data-centre GPU on five kinds of hardware: a CPU with a quantized model, the NPU inside a phone or PC chip, an embedded GPU module, an FPGA, or a dedicated LLM chip. The first three share one bottleneck, memory bandwidth, because every token requires reading all the weights. A dedicated chip removes that bottleneck but only pays off when the model is settled and the product ships in volume.</description></item>
<item><title>Taalas HC1 Explained: Specs, Speed, Limits and What It Means for Devices</title><link>https://www.fanshi-silicon.com/en/insights/taalas-hc1-explained/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/taalas-hc1-explained/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>The Taalas HC1 is a data-centre inference chip announced in February 2026 that hardwires one model, Llama 3.1 8B, into silicon. The company reports close to 17,000 tokens per second per user, against roughly 230 on an NVIDIA H200 for the same model. Published details include TSMC 6 nm, an 815 mm² die and about 53 billion transistors. The trade-offs are one model per chip, aggressive 3-bit and 6-bit quantization in the first silicon, and servers of around 2.5 kW. It is a cloud product; devices need the same idea at one watt to a few tens of watts.</description></item>
<item><title>NPU vs GPU Module vs Dedicated Chip for On-Device LLMs</title><link>https://www.fanshi-silicon.com/en/insights/npu-vs-dedicated-llm-chip/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/npu-vs-dedicated-llm-chip/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>An NPU, an embedded GPU module and a dedicated LLM chip can all run a language model inside a device, but they fail in different ways. An NPU is already in the product and costs nothing extra, yet it shares system memory and depends on the vendor toolchain. A GPU module runs any model that fits in its memory and draws tens of watts. A dedicated chip removes the memory bottleneck and uses the least power, but the model is decided in advance. The choice comes down to three questions: how settled the model is, how many units you ship, and how many watts you have.</description></item>
<item><title>On-Device LLMs for Robots: Why Embodied AI Needs Local Inference Hardware</title><link>https://www.fanshi-silicon.com/en/insights/on-device-llm-for-robots/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/on-device-llm-for-robots/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>A robot that plans with a language or vision-language model cannot depend on a data centre: the network adds delay that varies from moment to moment, and connectivity is not guaranteed where robots work. Running the model on board turns inference speed into a fixed part of the control loop. The hardware then has to meet three constraints at once: tokens per second high enough for the planning rate, a power budget of roughly ten to seventy watts on the robot body, and behaviour that stays the same on every unit shipped.</description></item>
<item><title>Running an LLM Offline in a Handheld: Translators, Recorders, Earbuds and Toys</title><link>https://www.fanshi-silicon.com/en/insights/offline-llm-handheld-voice-devices/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/offline-llm-handheld-voice-devices/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>A translator, a voice recorder, a pair of earbuds or a toy has one to three watts for the whole device and a battery that has to last the day. Running a language model there means a small model, typically under one billion parameters, on hardware that does not spend its power moving weights around. Because such products ship in volume with a model that stays the same for the life of the product, they are the clearest fit for a chip with the model fixed into it.</description></item>
<item><title>In-Car LLMs: Why the Cockpit Assistant Is Moving On Board</title><link>https://www.fanshi-silicon.com/en/insights/in-car-llm-on-device-inference/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/in-car-llm-on-device-inference/</guid><pubDate>Mon, 16 Nov 2026 01:00:00 GMT</pubDate><description>A cockpit assistant built on a language model has to answer in a tunnel, in an underground car park and on a road with no coverage. It also handles speech from everyone in the cabin, and a vehicle stays in service for ten years or more, during which a cloud model bills for every request. These pressures move inference into the car. The hardware then needs enough tokens per second for a multi-step assistant, a power budget of tens of watts, weights that can be updated over the air, and a supply that lasts as long as the vehicle programme.</description></item>
<item><title>What Is a Compute-in-Memory AI Chip? Three Routes Explained</title><link>https://www.fanshi-silicon.com/en/insights/what-is-compute-in-memory-chip/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/what-is-compute-in-memory-chip/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>A compute-in-memory (CIM) chip performs multiply-accumulate operations inside the memory array, so model weights never leave the storage cells. It targets the largest cost in LLM inference: moving weights from memory to the compute units. By where the weights live, there are three main routes: MaskROM compute-in-memory, SRAM compute-in-memory and near-memory compute.</description></item>
<item><title>How to Choose an Edge LLM Inference Chip: 7 Questions</title><link>https://www.fanshi-silicon.com/en/insights/how-to-choose-edge-inference-chip/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/how-to-choose-edge-inference-chip/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>To choose an edge inference chip, answer seven questions first: which model, how many parameters, how often it changes, what power budget, how fast decoding must be, whether data may leave the device, and how many units you will ship. Those answers largely decide between a general-purpose GPU or NPU and a dedicated compute-in-memory or near-memory ASIC.</description></item>
<item><title>LLM Inference Cost: API vs Cloud GPU vs Edge Chip Over 3 Years</title><link>https://www.fanshi-silicon.com/en/insights/inference-compute-cost/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/inference-compute-cost/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>LLM inference cost comes in three forms. API usage is priced per million tokens and is an ongoing expense; a self-hosted cloud GPU amortises rental or depreciation to about $0.70 per million tokens; a dedicated edge chip is bought once, after which the marginal cost of each inference approaches zero. At 10 million tokens a day, three years of a frontier-model API costs about $192,000, nearly a thousand times an illustrative $210 chip.</description></item>
<item><title>MaskROM Compute-in-Memory: What Casting Weights in Silicon Means</title><link>https://www.fanshi-silicon.com/en/insights/maskrom-cim-explained/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/maskrom-cim-explained/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>MaskROM compute-in-memory casts model weights into a via array in the silicon through the mask patterns used at tape-out, so the weights become part of the chip's structure. At runtime nothing has to be read out or moved: multiply-accumulate happens in place in the array, which gives the lowest power and marginal cost per inference of any route. The trade-off is a fixed model; changing it requires new masks.</description></item>
<item><title>FPGA Prototyping for LLM Accelerators: IP Licence to Tape-out</title><link>https://www.fanshi-silicon.com/en/insights/fpga-prototyping-for-llm-ip/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/fpga-prototyping-for-llm-ip/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>FPGA prototyping for LLMs means deploying an inference core IP on an FPGA evaluation board before tape-out and running the full path with a real model and real interfaces. It puts an engineering sample in your hands before the ASIC exists, exposes model-mapping, quantization-accuracy, interface and integration issues early, and gives the tape-out decision measured evidence.</description></item>
<item><title>The Memory Wall: Why Edge LLMs Stall and How to Get Around It</title><link>https://www.fanshi-silicon.com/en/insights/memory-wall-edge-llm/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/memory-wall-edge-llm/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>The memory wall is the gap between fast-growing compute and slow-growing memory bandwidth, which leaves compute units waiting for data most of the time. In LLM decoding every token reads all of the weights, so the throughput ceiling equals memory bandwidth divided by model size. Edge devices have only tens of GB/s of bandwidth, so conventional architectures cannot run a capable LLM; compute-in-memory and near-memory compute get around the wall by moving weights less, or not at all.</description></item>
<item><title>Qwen3.5 to Qwen3.8: Why Hybrid Attention Suits Edge Silicon</title><link>https://www.fanshi-silicon.com/en/insights/qwen-hybrid-attention-in-silicon/</link><guid isPermaLink="true">https://www.fanshi-silicon.com/en/insights/qwen-hybrid-attention-in-silicon/</guid><pubDate>Mon, 09 Nov 2026 01:00:00 GMT</pubDate><description>Qwen3.5, Qwen3.6 and Qwen3.8 use a hybrid layout in which three Gated DeltaNet linear-attention layers alternate with one gated full-attention layer. Only the full-attention quarter of the network keeps a KV cache that grows with context; the linear layers carry a fixed-size state however long the prompt is. For edge silicon that means a smaller, more predictable memory budget, and because the published layout has not changed between these releases, a newer Qwen model is a new build on the same inference cores rather than a new architecture.</description></item>
</channel></rss>
