mradermacher

/

Qwen3.8-Flash-Next-Uncensored-i1-GGUF

Like 38

I second this. Other Q3.8 models work just fine.

0.01.025.406 I common_params_fit_impl: getting device memory data for initial parameters:
0.01.080.791 E llama_model_load: error loading model: tensor 'blk.1.ffn_up_exps.weight' data is not within the file bounds, model is corrupted or incomplete
0.01.080.811 E llama_model_load_from_file_impl: failed to load model
0.01.081.764 E common_fit_params: encountered an error while trying to fit params to free device memory: failed to load model
0.01.081.770 I common_fit_params: fitting params to free memory took 0.06 seconds
0.01.122.722 E llama_model_load: error loading model: tensor 'blk.1.ffn_up_exps.weight' data is not within the file bounds, model is corrupted or incomplete
0.01.122.725 E llama_model_load_from_file_impl: failed to load model
0.01.122.728 E cmn common_init_: failed to load model '/ai/models/Qwen3.8-Flash-Next-Uncensored.i1-Q4_K_M-00001-of-00003.gguf'
0.01.122.729 E srv load_model: failed to load model, '/ai/models/Qwen3.8-Flash-Next-Uncensored.i1-Q4_K_M-00001-of-00003.gguf'
0.01.122.731 I srv operator(): operator(): cleaning up before exit...
0.01.124.256 E srv llama_server: exiting due to model loading error

Hello,

since Q5_K_S showed the same error I just tried to merge the three files into a single gguf with cat and they work flawlessly that way.

Thanks for providing those quants!

The slightly smaller i1-Q4_0 is also unloadable by llama.cpp (b10679), although the error occurs at a later block than in the case of i1-Q4_K_M.
0.00.364.220 E llama_model_load: error loading model: tensor 'blk.3.ffn_down_exps.weight' data is not within the file bounds, model is corrupted or incomplete

Interestingly, the unloadable (i hope that's a word :) quants, although their sha256 ckecksums are ok, seem to not be listed in the little table with available quants on the right-side in HF page.
Stay within what's listed or venture in the repo at your own risk :D

PS: After concatenating the shards I can confirm qwen's (over)thinking capabilities are restored. It might be worth checking how the spit was exactly done, because the shards don't look like they've been split with anything like "llama-gguf-split --split-max-size 32G". They look more like a naive split with something like "dd".

All mradermacher models use the legacy split format instead of the gguf-split format for the following reasons:

• The main reason is that unlike gguf-split the legacy split format supports zero copy. This means you can split and combine files without having to read/write any data or using any additional storage on a file system that supports zero copy. So once the GUUF is done we can simply split it without having to read/write any data and withing a fraction of a second and the same everyone can do after downloading our quants. The gguf-split file format was so terrible designed that it is incompatible with zero copy and so every time you want to split or concatenate GGUFs in that format you need to perform a full read and write of all the data which would make our IO bottleneck and storage issues much worse than they already are. Many of our users would also lack double the storage of a massive model required to concatenate them again. Sure one technically could run a gguf-split without concatenation but not archiving GGUFs in thair concatenated form is ugly.
• The split format we are using was invented by TheBloke and the standard everyone used at the time Team Mradermacher started. We predate the gguf-split file format and switching to it would confuse our millions of existing users that are all used to the legacy split format.
• Our convenient download page (https://hf.tst.eu/model#Qwen3.8-Flash-Next-Uncensored-GGUF for this model) that let's user's browser concatenate GGUFs while downloading relies on the legacy file format as the new file format can no longer easily be concatenated using JavaScript.
• The reason files where spitted in the first place is because HuggingFace used to have a 50 GB per file limit. With the introduction of XET this got removed. The only reason we even still split them if larger than 100 GB is because of https://github.com/huggingface/xet-core/issues/592 but with the implementation of a WebAssambly based XET client the CAS Bridge might soon be history after which splitting GGUFs is a thing of the past unless we are talking of trillion parameter sized models as XET has its per file limit set to 500 GB. And even with the current per file limit us having to split models is very rare and is only needed for a hand full of massive models while back in the LFS days even 70B models had to be spitted.

since Q5_K_S showed the same error I just tried to merge the three files into a single gguf with cat and they work flawlessly that way.

This is exactly how you have to merge any quants stored in the legacy split format before being able to open them inside llama.cpp. If you are on a file system such as BTRFS this will happen instantly without requiring any additional storage thanks to zero copy. Otherwise just use the convenient download page so they get merged while downloading.

Thanks for taking the time to explain - this was all news to me.
Is there any way to download from the convenient download page on my Ubuntu server without GUI?
I tried to copy the download link from my windows machine over but wget and curl just time out after a while...

I was shocked to find the new splits couldn't be concatenated. +1 to sticking with a sensible filesystem-based split -- THE WAY WE ALWAYS DID SINCE 45 YEARS :)