The Short Version
Qwen3.8-27B is a 27 billion parameter model from Alibaba that reads text, images and video, released in August 2026 under the Apache 2.0 license. A 4-bit version is about 16 to 18 GB and runs on a 24 GB graphics card or a Mac with 24 GB of memory or more. Measured speed at a short context is about 75 tokens per second on an RTX 5090 and 31 on an M5 Max. On the 5090 and 4090, speed drops sharply at long contexts in llama.cpp, so plan your context size before choosing hardware.
Model Facts
| Item | Detail |
|---|---|
| Parameters | 27.78 billion |
| Input and output | Text, image and video in. Text out |
| Context window | 262,144 tokens, extendable to 1,000,000 |
| License | Apache 2.0 |
| Thinking mode | On by default, with three effort levels |
| Recent downloads on Hugging Face | About 6.96 million |
It is the base for five of the top 20 trending models on Hugging Face as of November 13, 2026. See the trending list.
Step 1: Check Your Memory
Alibaba's model card gives no hardware guidance. These figures are from Unsloth, which publishes the most widely used quantized files.
| Quantization | Total memory needed | Fits on |
|---|---|---|
| 2-bit | 9 to 11 GB | 12 GB cards, with a clear quality loss |
| 3-bit | 12 to 14 GB | 16 GB cards |
| 4-bit | 16 to 19 GB | 24 GB cards, or a Mac with 24 GB |
| 6-bit | 23 to 26 GB | 32 GB cards, or a Mac with 32 GB or more |
| 8-bit | 31 GB | Two cards, or a Mac with 48 GB or more |
| Full precision | 56 GB | A Mac with 64 GB or more, or a workstation |
The "fits on" column is our reading of those figures. Context adds to this. One test measured a 4-bit file using 18 GB of video memory at an 8,000-token context, 20 GB at 32,000, 26 GB at 128,000 and 34 GB at 256,000. A 24 GB card therefore holds the 4-bit model with roughly 64,000 to 112,000 tokens of context, not the full window.
Step 2: Pick an App
| App | Name to look for | Notes |
|---|---|---|
| Ollama | qwen3.8, with the tag 27b | The default is a 4-bit build shown as 18 GB. Tags also exist for 8-bit (30 GB) and Apple MLX builds |
| LM Studio | qwen/qwen3.8-27b | Lists 16 GB as minimum system memory. Offers GGUF and MLX builds |
| llama.cpp | A GGUF file from Hugging Face | Most control. Needs a separate vision file for images |
| vLLM or SGLang | The original weights, or an NVFP4 build | For serving several users. NVFP4 needs an RTX 50 series or other Blackwell GPU |
For most people on one machine, Ollama or LM Studio is the easy route. On a Mac, choose an MLX build: one test measured 68 tokens per second with MLX against 36.6 with the GGUF build on the same M5 Max.
Step 3: Choose a File Size
There is no official GGUF from Alibaba. Sizes below are from the Unsloth repository.
| File | Size |
|---|---|
| UD-Q2_K_XL | 9.95 GB |
| UD-Q3_K_XL | 13.15 GB |
| UD-Q4_K_M | 16.56 GB |
| UD-Q5_K_M | 19.87 GB |
| UD-Q6_K | 21.98 GB |
| Q8_0 | 29.05 GB |
| BF16, full precision | 54.66 GB |
| Vision file (mmproj), needed for images | 0.93 GB |
The same label differs by publisher. A "Q4_K_M" file is 16.56 GB from Unsloth, 16.81 GB from LM Studio, 17.44 GB from bartowski and 19.40 GB from ggml-org. On Apple silicon, the MLX 4-bit build is 16.07 GB and the 8-bit build 29.52 GB.
Step 4: Know What Speed to Expect
These are published measurements from Hardware Corner, using llama.cpp with a 4-bit file. Figures are generation speed in tokens per second.
| Hardware | 4,000-token context | 32,000 | 64,000 | 128,000 |
|---|---|---|---|---|
| RTX 5090, 32 GB | 74.8 | 29.0 | 26.2 | 22.8 |
| RTX 4090, 24 GB | 46.2 | 42.2 | 38.4 | Not measured |
| RTX 3090, 24 GB | 40.3 | 37.0 | 34.0 | Not measured |
| Two RTX 5060 Ti, 16 GB each | 22.5 | 20.4 | Not measured | 15.6 |
| Apple M5 Max, 128 GB | 31.4 | 21.6 | 18.2 | Not measured |
| Nvidia GB10, the DGX Spark class | 12.2 | 10.9 | 9.7 | 8.0 |
For how these compare with other models, see local LLM tokens per second benchmarks.
Step 5: Plan Around the Long-Context Slowdown
The RTX 5090 is the fastest card at a short context and falls to about 29 tokens per second at 32,000 tokens, below a 4090 at the same context. The same test found the older Qwen3.6-27B at 63.7 tokens per second on a 5090 at 64,000 tokens, against 26.2 for Qwen3.8. Tom's Hardware saw the same pattern and called llama.cpp performance on the 5090 at long context "dire". It suspects a bug, because the 3090 does not show it.
- For chat and short tasks: any of the cards above is comfortable.
- For coding agents with long context on a 5090: llama.cpp is the wrong engine today. SGLang's published figures show about 153 tokens per second per user on a 5090 with an NVFP4 build and its speculative decoding head, at an 8,192-token input.
- For two 5090s: Tom's Hardware measured 70 to 80 tokens per second in vLLM across the full context, and 100 to 110 with the speculative head.
Step 6: Set the Sampling Options
These are the settings on Alibaba's model card.
| Setting | Thinking mode | Non-thinking mode |
|---|---|---|
| Temperature | 1.0 | 0.7 |
| Top-p | 0.95 | 0.80 |
| Top-k | 20 | 20 |
| Presence penalty | 0.0 | 1.5 |
Thinking is on by default at the highest of three effort levels. Lower effort or switching thinking off makes replies start sooner. The card warns that lower effort does not always reduce total time in multi-step agent work.
Known Issues
- Tool calling: the model repository has open reports of chat-template errors and tool-call loops. Alibaba last changed the official template on November 7, 2026, so use a recent download.
- CPU-only runs in llama.cpp: an open issue reports repetition loops for this model family on the CPU backend.
- Ollama: several bugs around loading, tool calls and thinking levels were fixed during the autumn. Update before troubleshooting.
- Modified versions: many popular files are community edits of this model with refusals removed. They are not Alibaba's release. See abliterated model.
Is It Worth Running?
Alibaba's own evaluations show large gains over Qwen3.6-27B on coding and computer-use tests, for example 61.7 against 53.5 on SWE-bench Pro and 84.3 against 63.9 on OSWorld-Verified. Those are vendor numbers. We found no credible independent comparison with Gemma 4 31B. If you run long contexts on a 5090 with llama.cpp, the older model is currently much faster.
Sources
Alibaba's model card; file sizes from the Unsloth GGUF repository; memory guidance from Unsloth's guide; speeds from Hardware Corner, Tom's Hardware and the SGLang cookbook; app listings from Ollama and LM Studio. Read November 13, 2026. Presenc AI did not run these tests.