Xing 4.0 29B-A4B

XingChen-AGI/Xing4.0-29B-A4B-GGUF

4B active / 29B totalMoEText generationText input

XingChen-AGI · 4 recorded configurations

Measured setup

Hardware

Placement unverified

Runtime and tool

live llama-server

Quantisation

IQ4_NL

MTP draft depth

Every option opens a recorded setup. Where the corpus recorded nothing at this page's other settings, the option names the fields it switches with it. Allocation and runtime detail stay in the technical record below.

No throughput measurements were recorded for this setup, so no rates are shown or plotted. The technical record below keeps the recorded outcome.

No prefill or decode measurements were recorded; no values are plotted.

PP · left axis · not measuredTG · right axis · not measured

View exact measurements

Exact context depth and measured fresh prefill, bulk prefix, and decode rates for Xing4.0 29B-A4B
Context depth (tokens)	Fresh prefill PP (tok/s)	TG (tok/s)
No single-stream context measurements were recorded.

Technical record and provenance

Source 1 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-context-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c1
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: context
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 0 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 2,048 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 4,096 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 8,192 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 16,384 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 32,768 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 65,536 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 131,072 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error ; 261,312 tokens · 1 stream: error — llama-server exited: hunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=964242 0.00.083.216 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.083.220 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.083.220 I cmn common_param: device_info: 0.00.202.167 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.202.180 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.202.298 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.202.429 I srv init: using 23 threads for HTTP server 0.00.202.431 I srv init: The UI is disabled 0.00.202.431 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.202.511 W srv llama_server: ----------------- 0.00.202.513 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.202.513 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.202.514 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.202.514 W srv llama_server: ----------------- 0.00.202.528 I srv start: binding port with default address family 0.00.203.860 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.203.863 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.232.957 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.232.998 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.233.015 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.233.018 I llama_model_loader: - kv 1: general.type str = model 0.00.233.025 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.233.026 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.233.027 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.233.029 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.030 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.233.032 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.233.032 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.233.033 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.233.034 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.233.035 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.233.035 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.233.036 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.233.036 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.233.038 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.233.039 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.233.039 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.233.040 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.233.042 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.233.043 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.233.043 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.233.044 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.233.044 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.233.045 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.233.046 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.233.046 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.233.047 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.233.048 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.233.048 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.233.049 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.233.049 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.233.050 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.233.050 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.233.051 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.233.052 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.233.052 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.233.053 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.233.054 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.233.054 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.233.055 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.233.056 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.233.057 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.233.057 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.233.058 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.252.464 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.297.715 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.302.392 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.302.396 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.302.397 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.302.398 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.302.399 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.302.402 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.302.403 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.302.403 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.302.404 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.302.405 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.302.406 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.302.407 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.302.413 I llama_model_loader: - type f32: 406 tensors 0.00.302.414 I llama_model_loader: - type q6_K: 1 tensors 0.00.302.414 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.302.414 I llama_model_loader: - type bf16: 327 tensors 0.00.302.418 I print_info: file format = GGUF V3 (latest) 0.00.302.420 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.302.422 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.303.370 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.303.378 E llama_model_load_from_file_impl: failed to load model 0.00.303.383 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.388 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.303.390 I srv operator(): operator(): cleaning up before exit... 0.00.304.281 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=1). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-context-physical1

Source 2 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c2
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 2 streams: error — llama-server exited: hunks_count u32 = 8 0.00.297.247 I llama_model_loader: - type f32: 406 tensors 0.00.297.247 I llama_model_loader: - type q6_K: 1 tensors 0.00.297.248 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.297.248 I llama_model_loader: - type bf16: 327 tensors 0.00.297.252 I print_info: file format = GGUF V3 (latest) 0.00.297.254 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.297.256 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.298.245 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.298.254 E llama_model_load_from_file_impl: failed to load model 0.00.298.259 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.298.274 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.298.278 I srv operator(): operator(): cleaning up before exit... 0.00.299.289 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=966989 0.00.089.303 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.089.306 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.089.307 I cmn common_param: device_info: 0.00.197.883 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.197.896 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.198.000 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.198.135 I srv init: using 23 threads for HTTP server 0.00.198.137 I srv init: The UI is disabled 0.00.198.137 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.198.215 W srv llama_server: ----------------- 0.00.198.217 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.198.217 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.198.217 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.198.217 W srv llama_server: ----------------- 0.00.198.231 I srv start: binding port with default address family 0.00.199.558 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.199.561 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.228.511 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.228.550 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.228.567 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.228.569 I llama_model_loader: - kv 1: general.type str = model 0.00.228.575 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.228.576 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.228.577 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.228.579 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.228.580 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.228.580 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.228.581 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.228.582 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.228.583 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.228.583 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.228.584 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.228.585 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.228.585 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.228.587 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.228.587 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.228.588 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.228.589 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.228.591 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.228.592 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.228.592 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.228.593 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.228.593 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.228.594 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.228.595 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.228.595 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.228.596 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.228.597 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.228.597 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.228.597 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.228.598 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.228.598 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.228.599 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.228.600 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.228.600 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.228.601 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.228.602 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.228.603 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.228.604 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.228.604 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.228.605 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.228.605 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.228.606 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.228.607 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.228.607 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.247.357 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.292.559 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.297.226 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.297.230 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.297.232 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.297.233 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.297.233 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.297.236 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.297.237 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.297.237 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.297.239 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.297.240 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.297.240 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.297.241 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.297.247 I llama_model_loader: - type f32: 406 tensors 0.00.297.247 I llama_model_loader: - type q6_K: 1 tensors 0.00.297.248 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.297.248 I llama_model_loader: - type bf16: 327 tensors 0.00.297.252 I print_info: file format = GGUF V3 (latest) 0.00.297.254 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.297.256 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.298.245 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.298.254 E llama_model_load_from_file_impl: failed to load model 0.00.298.259 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.298.274 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.298.278 I srv operator(): operator(): cleaning up before exit... 0.00.299.289 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=2). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 3 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c3
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 3 streams: error — llama-server exited: hunks_count u32 = 8 0.00.291.564 I llama_model_loader: - type f32: 406 tensors 0.00.291.565 I llama_model_loader: - type q6_K: 1 tensors 0.00.291.565 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.291.565 I llama_model_loader: - type bf16: 327 tensors 0.00.291.569 I print_info: file format = GGUF V3 (latest) 0.00.291.571 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.291.573 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.292.557 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.292.565 E llama_model_load_from_file_impl: failed to load model 0.00.292.571 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.577 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.584 I srv operator(): operator(): cleaning up before exit... 0.00.293.628 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967166 0.00.082.243 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.082.246 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.082.246 I cmn common_param: device_info: 0.00.194.444 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.194.456 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.194.568 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.194.685 I srv init: using 23 threads for HTTP server 0.00.194.688 I srv init: The UI is disabled 0.00.194.688 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.194.758 W srv llama_server: ----------------- 0.00.194.762 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.194.763 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.194.763 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.194.764 W srv llama_server: ----------------- 0.00.194.776 I srv start: binding port with default address family 0.00.196.079 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.196.081 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.222.824 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.222.869 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.222.898 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.222.901 I llama_model_loader: - kv 1: general.type str = model 0.00.222.908 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.222.909 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.222.910 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.222.912 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.222.913 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.222.914 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.222.915 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.222.915 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.222.916 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.222.917 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.222.917 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.222.918 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.222.918 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.222.920 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.222.921 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.222.922 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.222.923 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.222.924 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.222.925 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.222.926 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.222.926 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.222.927 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.222.927 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.222.928 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.222.928 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.222.929 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.222.930 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.222.930 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.222.931 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.222.931 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.222.932 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.222.932 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.222.933 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.222.933 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.222.934 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.222.935 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.222.936 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.222.937 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.222.937 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.222.938 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.222.938 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.222.939 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.222.940 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.222.940 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.242.046 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.286.651 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.291.546 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.291.549 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.291.550 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.291.551 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.291.552 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.291.555 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.291.555 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.291.556 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.291.557 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.291.558 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.291.559 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.291.559 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.291.564 I llama_model_loader: - type f32: 406 tensors 0.00.291.565 I llama_model_loader: - type q6_K: 1 tensors 0.00.291.565 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.291.565 I llama_model_loader: - type bf16: 327 tensors 0.00.291.569 I print_info: file format = GGUF V3 (latest) 0.00.291.571 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.291.573 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.292.557 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.292.565 E llama_model_load_from_file_impl: failed to load model 0.00.292.571 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.577 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.584 I srv operator(): operator(): cleaning up before exit... 0.00.293.628 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=3). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 4 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c4
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 4 streams: error — llama-server exited: hunks_count u32 = 8 0.00.274.480 I llama_model_loader: - type f32: 406 tensors 0.00.274.481 I llama_model_loader: - type q6_K: 1 tensors 0.00.274.481 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.274.482 I llama_model_loader: - type bf16: 327 tensors 0.00.274.486 I print_info: file format = GGUF V3 (latest) 0.00.274.489 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.274.491 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.275.645 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.275.653 E llama_model_load_from_file_impl: failed to load model 0.00.275.659 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.275.665 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.275.667 I srv operator(): operator(): cleaning up before exit... 0.00.276.785 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967268 0.00.080.417 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.080.420 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.080.420 I cmn common_param: device_info: 0.00.184.772 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.184.783 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.184.884 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.185.036 I srv init: using 23 threads for HTTP server 0.00.185.039 I srv init: The UI is disabled 0.00.185.040 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.185.125 W srv llama_server: ----------------- 0.00.185.128 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.185.128 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.185.128 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.185.128 W srv llama_server: ----------------- 0.00.185.148 I srv start: binding port with default address family 0.00.186.476 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.186.479 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.211.346 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.211.379 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.211.394 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.211.403 I llama_model_loader: - kv 1: general.type str = model 0.00.211.410 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.211.411 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.211.412 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.211.414 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.211.415 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.211.416 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.211.417 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.211.417 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.211.418 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.211.419 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.211.420 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.211.420 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.211.421 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.211.423 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.211.423 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.211.424 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.211.425 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.211.426 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.211.427 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.211.427 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.211.428 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.211.428 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.211.428 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.211.429 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.211.429 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.211.430 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.211.430 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.211.431 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.211.431 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.211.432 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.211.432 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.211.433 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.211.433 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.211.433 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.211.434 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.211.435 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.211.435 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.211.436 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.211.436 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.211.437 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.211.437 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.211.438 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.211.438 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.211.439 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.227.948 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.270.197 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.274.458 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.274.462 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.274.463 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.274.464 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.274.465 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.274.468 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.274.469 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.274.470 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.274.471 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.274.473 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.274.473 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.274.474 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.274.480 I llama_model_loader: - type f32: 406 tensors 0.00.274.481 I llama_model_loader: - type q6_K: 1 tensors 0.00.274.481 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.274.482 I llama_model_loader: - type bf16: 327 tensors 0.00.274.486 I print_info: file format = GGUF V3 (latest) 0.00.274.489 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.274.491 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.275.645 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.275.653 E llama_model_load_from_file_impl: failed to load model 0.00.275.659 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.275.665 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.275.667 I srv operator(): operator(): cleaning up before exit... 0.00.276.785 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=4). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 5 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c5
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 5 streams: error — llama-server exited: hunks_count u32 = 8 0.00.291.202 I llama_model_loader: - type f32: 406 tensors 0.00.291.203 I llama_model_loader: - type q6_K: 1 tensors 0.00.291.203 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.291.204 I llama_model_loader: - type bf16: 327 tensors 0.00.291.207 I print_info: file format = GGUF V3 (latest) 0.00.291.209 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.291.210 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.292.131 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.292.139 E llama_model_load_from_file_impl: failed to load model 0.00.292.144 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.149 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.151 I srv operator(): operator(): cleaning up before exit... 0.00.293.159 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967385 0.00.086.648 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.086.651 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.086.652 I cmn common_param: device_info: 0.00.200.635 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.200.648 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.200.743 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.200.863 I srv init: using 23 threads for HTTP server 0.00.200.865 I srv init: The UI is disabled 0.00.200.865 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.200.936 W srv llama_server: ----------------- 0.00.200.938 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.200.938 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.200.938 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.200.938 W srv llama_server: ----------------- 0.00.200.951 I srv start: binding port with default address family 0.00.202.267 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.202.270 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.228.725 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.228.762 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.228.780 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.228.782 I llama_model_loader: - kv 1: general.type str = model 0.00.228.788 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.228.789 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.228.789 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.228.791 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.228.792 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.228.793 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.228.794 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.228.794 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.228.795 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.228.796 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.228.796 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.228.797 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.228.798 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.228.799 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.228.800 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.228.801 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.228.801 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.228.803 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.228.804 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.228.804 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.228.806 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.228.806 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.228.807 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.228.807 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.228.807 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.228.808 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.228.809 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.228.809 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.228.810 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.228.810 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.228.811 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.228.811 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.228.812 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.228.812 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.228.813 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.228.814 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.228.815 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.228.816 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.228.817 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.228.817 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.228.818 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.228.819 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.228.820 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.228.820 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.246.017 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.286.909 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.291.184 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.291.188 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.291.188 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.291.190 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.291.190 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.291.193 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.291.194 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.291.195 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.291.196 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.291.197 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.291.197 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.291.198 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.291.202 I llama_model_loader: - type f32: 406 tensors 0.00.291.203 I llama_model_loader: - type q6_K: 1 tensors 0.00.291.203 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.291.204 I llama_model_loader: - type bf16: 327 tensors 0.00.291.207 I print_info: file format = GGUF V3 (latest) 0.00.291.209 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.291.210 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.292.131 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.292.139 E llama_model_load_from_file_impl: failed to load model 0.00.292.144 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.149 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.292.151 I srv operator(): operator(): cleaning up before exit... 0.00.293.159 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=5). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 6 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c6
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 6 streams: error — llama-server exited: hunks_count u32 = 8 0.00.296.537 I llama_model_loader: - type f32: 406 tensors 0.00.296.538 I llama_model_loader: - type q6_K: 1 tensors 0.00.296.538 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.296.539 I llama_model_loader: - type bf16: 327 tensors 0.00.296.542 I print_info: file format = GGUF V3 (latest) 0.00.296.544 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.296.545 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.297.523 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.297.531 E llama_model_load_from_file_impl: failed to load model 0.00.297.536 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.297.541 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.297.544 I srv operator(): operator(): cleaning up before exit... 0.00.300.754 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967530 0.00.079.265 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.079.269 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.079.269 I cmn common_param: device_info: 0.00.196.196 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.196.208 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.196.315 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.196.441 I srv init: using 23 threads for HTTP server 0.00.196.443 I srv init: The UI is disabled 0.00.196.444 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.196.519 W srv llama_server: ----------------- 0.00.196.520 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.196.521 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.196.521 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.196.521 W srv llama_server: ----------------- 0.00.196.534 I srv start: binding port with default address family 0.00.197.846 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.197.849 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.227.127 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.227.173 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.227.190 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.227.192 I llama_model_loader: - kv 1: general.type str = model 0.00.227.199 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.227.200 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.227.201 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.227.203 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.227.204 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.227.205 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.227.206 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.227.206 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.227.207 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.227.208 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.227.209 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.227.209 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.227.210 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.227.211 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.227.212 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.227.213 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.227.213 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.227.215 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.227.216 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.227.216 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.227.217 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.227.217 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.227.218 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.227.218 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.227.219 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.227.219 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.227.220 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.227.221 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.227.221 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.227.222 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.227.222 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.227.223 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.227.223 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.227.224 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.227.224 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.227.225 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.227.226 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.227.227 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.227.227 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.227.228 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.227.228 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.227.229 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.227.230 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.227.230 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.246.396 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.291.547 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.296.518 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.296.521 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.296.522 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.296.523 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.296.524 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.296.527 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.296.528 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.296.529 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.296.529 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.296.530 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.296.531 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.296.532 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.296.537 I llama_model_loader: - type f32: 406 tensors 0.00.296.538 I llama_model_loader: - type q6_K: 1 tensors 0.00.296.538 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.296.539 I llama_model_loader: - type bf16: 327 tensors 0.00.296.542 I print_info: file format = GGUF V3 (latest) 0.00.296.544 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.296.545 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.297.523 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.297.531 E llama_model_load_from_file_impl: failed to load model 0.00.297.536 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.297.541 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.297.544 I srv operator(): operator(): cleaning up before exit... 0.00.300.754 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=6). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 7 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c7
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 7 streams: error — llama-server exited: hunks_count u32 = 8 0.00.324.093 I llama_model_loader: - type f32: 406 tensors 0.00.324.094 I llama_model_loader: - type q6_K: 1 tensors 0.00.324.094 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.324.095 I llama_model_loader: - type bf16: 327 tensors 0.00.324.098 I print_info: file format = GGUF V3 (latest) 0.00.324.100 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.324.103 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.325.019 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.325.027 E llama_model_load_from_file_impl: failed to load model 0.00.325.033 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.325.038 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.325.042 I srv operator(): operator(): cleaning up before exit... 0.00.326.131 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967634 0.00.084.824 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.084.828 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.084.828 I cmn common_param: device_info: 0.00.222.364 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.222.380 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.222.514 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.222.678 I srv init: using 23 threads for HTTP server 0.00.222.681 I srv init: The UI is disabled 0.00.222.682 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.222.772 W srv llama_server: ----------------- 0.00.222.774 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.222.775 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.222.775 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.222.776 W srv llama_server: ----------------- 0.00.222.791 I srv start: binding port with default address family 0.00.224.162 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.224.166 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.253.937 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.253.986 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.254.011 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.254.013 I llama_model_loader: - kv 1: general.type str = model 0.00.254.020 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.254.021 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.254.022 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.254.024 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.254.025 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.254.026 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.254.027 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.254.028 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.254.029 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.254.029 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.254.030 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.254.031 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.254.031 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.254.033 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.254.034 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.254.035 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.254.036 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.254.037 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.254.038 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.254.039 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.254.039 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.254.040 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.254.040 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.254.041 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.254.041 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.254.042 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.254.043 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.254.043 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.254.043 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.254.044 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.254.045 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.254.045 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.254.046 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.254.046 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.254.047 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.254.048 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.254.048 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.254.049 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.254.050 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.254.050 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.254.051 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.254.052 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.254.053 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.254.053 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.273.324 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.319.370 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.324.072 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.324.076 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.324.077 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.324.078 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.324.078 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.324.081 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.324.082 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.324.083 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.324.084 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.324.085 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.324.086 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.324.086 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.324.093 I llama_model_loader: - type f32: 406 tensors 0.00.324.094 I llama_model_loader: - type q6_K: 1 tensors 0.00.324.094 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.324.095 I llama_model_loader: - type bf16: 327 tensors 0.00.324.098 I print_info: file format = GGUF V3 (latest) 0.00.324.100 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.324.103 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.325.019 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.325.027 E llama_model_load_from_file_impl: failed to load model 0.00.325.033 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.325.038 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.325.042 I srv operator(): operator(): cleaning up before exit... 0.00.326.131 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=7). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Source 8 · 2026-10-15

Source run ID: pub-20261016-trpro-xing40-iq4-nl-concurrency-2026-10-14-single-gpu-isolated-gpu1-co-scheduled-mtp-1-c8
Page entry ID: xing4-0-29b-a4b-iq4-nl-single-gpu-20261016-mtp-1-233c93090e
Entry sweep: concurrency
Raw entry label: Xing4.0 29B-A4B IQ4_NL — Single GPU requested — startup failed; placement unverified
Configuration label: Single GPU requested — startup failed; placement unverified
Measured date: 2026-10-15
Hardware: Not observed (outcome-only startup failure)
Runtime: llamaCpp · version: 0.3.0-dev (build 50, commit fc6545d) built with GNU 15.2.0 for Linux x86_64
Weight placement: Unverified (startup failed)
Observed devices: None observed
Requested devices: CUDA0
Split mode: None
Placement detail: Single GPU requested — startup failed; placement unverified
KV-cache precision: Q8_0
Flash Attention: Enabled
Batch size: 512
Microbatch size: 512
Prompt tokens: 512
Generated tokens: 256
Speculative decoding: MTP · draft max 1
Warm-ups: 1
Measured repetitions: 3
Timing method: server durations aligned to client completion timestamps; aggregate-all uses client wall time
Measurement outcomes: 8,192 tokens · 8 streams: error — llama-server exited: hunks_count u32 = 8 0.00.298.046 I llama_model_loader: - type f32: 406 tensors 0.00.298.047 I llama_model_loader: - type q6_K: 1 tensors 0.00.298.047 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.298.047 I llama_model_loader: - type bf16: 327 tensors 0.00.298.050 I print_info: file format = GGUF V3 (latest) 0.00.298.053 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.298.055 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.298.996 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.299.005 E llama_model_load_from_file_impl: failed to load model 0.00.299.010 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.299.015 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.299.017 I srv operator(): operator(): cleaning up before exit... 0.00.299.881 E srv llama_server: exiting due to model loading error __AUTO_BENCH_HOST_LOCK=shared __AUTO_BENCH_PID=967745 0.00.088.613 I cmn common_param: common_params_print_info: build 50 (fc6545d) with GNU 15.2.0 for Linux x86_64 0.00.088.617 I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg) 0.00.088.617 I cmn common_param: device_info: 0.00.198.473 I cmn common_param: - CUDA0 : NVIDIA GeForce RTX 3090 (24126 MiB, 23861 MiB free) 0.00.198.485 I cmn common_param: - CPU : AMD Ryzen Threadripper PRO 3945WX 12-Cores (126346 MiB, 126346 MiB free) 0.00.198.582 I cmn common_param: system_info: n_threads = 12 (n_threads_batch = 12) / 24 | CUDA : ARCHS = 860 | USE_GRAPHS = 1 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | LLAMAFILE = 1 | OPENMP = 1 | REPACK = 1 | 0.00.198.717 I srv init: using 23 threads for HTTP server 0.00.198.720 I srv init: The UI is disabled 0.00.198.720 I srv init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable 0.00.198.803 W srv llama_server: ----------------- 0.00.198.805 W srv llama_server: CORS is set to allow all origins ('*') and no API key is set 0.00.198.806 W srv llama_server: this can be a security risk (cross-origin attacks) 0.00.198.806 W srv llama_server: more info: https://github.com/ggml-org/llama.cpp/pull/25655 0.00.198.806 W srv llama_server: ----------------- 0.00.198.820 I srv start: binding port with default address family 0.00.200.127 I srv load_model: loading model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.200.132 I srv load_model: local path '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.229.387 I llama_model_loader: loaded meta data with 60 key-value pairs and 977 tensors from /home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf (version GGUF V3 (latest)) 0.00.229.434 I llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 0.00.229.455 I llama_model_loader: - kv 0: general.architecture str = xing4_0 0.00.229.457 I llama_model_loader: - kv 1: general.type str = model 0.00.229.464 I llama_model_loader: - kv 2: general.sampling.top_p f32 = 0.950000 0.00.229.465 I llama_model_loader: - kv 3: general.sampling.temp f32 = 1.000000 0.00.229.466 I llama_model_loader: - kv 4: general.sampling.penalty_repeat f32 = 1.050000 0.00.229.468 I llama_model_loader: - kv 5: general.name str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.229.469 I llama_model_loader: - kv 6: general.finetune str = 29b_12.05T_wodsa_id20_s3_256k_8_26090... 0.00.229.469 I llama_model_loader: - kv 7: general.size_label str = 64x4.2B 0.00.229.470 I llama_model_loader: - kv 8: xing4_0.block_count u32 = 41 0.00.229.471 I llama_model_loader: - kv 9: xing4_0.context_length u32 = 262144 0.00.229.471 I llama_model_loader: - kv 10: xing4_0.embedding_length u32 = 3584 0.00.229.472 I llama_model_loader: - kv 11: xing4_0.feed_forward_length u32 = 9216 0.00.229.472 I llama_model_loader: - kv 12: xing4_0.attention.head_count u32 = 32 0.00.229.473 I llama_model_loader: - kv 13: xing4_0.attention.head_count_kv u32 = 1 0.00.229.473 I llama_model_loader: - kv 14: xing4_0.rope.scaling.type str = yarn 0.00.229.475 I llama_model_loader: - kv 15: xing4_0.rope.scaling.factor f32 = 64.000000 0.00.229.476 I llama_model_loader: - kv 16: xing4_0.rope.scaling.original_context_length u32 = 4096 0.00.229.477 I llama_model_loader: - kv 17: xing4_0.rope.scaling.yarn_beta_fast f32 = 32.000000 0.00.229.478 I llama_model_loader: - kv 18: xing4_0.rope.scaling.yarn_beta_slow f32 = 1.000000 0.00.229.479 I llama_model_loader: - kv 19: xing4_0.rope.freq_base f32 = 10000.000000 0.00.229.480 I llama_model_loader: - kv 20: xing4_0.attention.layer_norm_rms_epsilon f32 = 0.000001 0.00.229.481 I llama_model_loader: - kv 21: xing4_0.expert_count u32 = 64 0.00.229.481 I llama_model_loader: - kv 22: xing4_0.expert_used_count u32 = 4 0.00.229.482 I llama_model_loader: - kv 23: xing4_0.expert_group_count u32 = 1 0.00.229.482 I llama_model_loader: - kv 24: xing4_0.expert_group_used_count u32 = 1 0.00.229.483 I llama_model_loader: - kv 25: xing4_0.expert_gating_func u32 = 2 0.00.229.483 I llama_model_loader: - kv 26: xing4_0.leading_dense_block_count u32 = 2 0.00.229.484 I llama_model_loader: - kv 27: xing4_0.vocab_size u32 = 131072 0.00.229.484 I llama_model_loader: - kv 28: xing4_0.attention.q_lora_rank u32 = 768 0.00.229.485 I llama_model_loader: - kv 29: xing4_0.attention.kv_lora_rank u32 = 512 0.00.229.485 I llama_model_loader: - kv 30: xing4_0.attention.key_length u32 = 576 0.00.229.486 I llama_model_loader: - kv 31: xing4_0.attention.value_length u32 = 512 0.00.229.486 I llama_model_loader: - kv 32: xing4_0.attention.key_length_mla u32 = 192 0.00.229.487 I llama_model_loader: - kv 33: xing4_0.attention.value_length_mla u32 = 128 0.00.229.487 I llama_model_loader: - kv 34: xing4_0.expert_feed_forward_length u32 = 1024 0.00.229.488 I llama_model_loader: - kv 35: xing4_0.expert_shared_count u32 = 1 0.00.229.488 I llama_model_loader: - kv 36: xing4_0.expert_weights_scale f32 = 2.000000 0.00.229.489 I llama_model_loader: - kv 37: xing4_0.expert_weights_norm bool = true 0.00.229.490 I llama_model_loader: - kv 38: xing4_0.rope.dimension_count u32 = 64 0.00.229.491 I llama_model_loader: - kv 39: xing4_0.rope.scaling.yarn_log_multiplier f32 = 0.100000 0.00.229.491 I llama_model_loader: - kv 40: xing4_0.nextn_predict_layers u32 = 1 0.00.229.491 I llama_model_loader: - kv 41: xing4_0.hyper_connection.count u32 = 4 0.00.229.492 I llama_model_loader: - kv 42: xing4_0.hyper_connection.sinkhorn_iterations u32 = 20 0.00.229.493 I llama_model_loader: - kv 43: xing4_0.hyper_connection.epsilon f32 = 0.000001 0.00.229.493 I llama_model_loader: - kv 44: tokenizer.ggml.model str = llama 0.00.229.494 I llama_model_loader: - kv 45: tokenizer.ggml.pre str = default 0.00.248.398 I llama_model_loader: - kv 46: tokenizer.ggml.tokens arr[str,131072] = ["<_unk>", "<_start>", "<_end>", "<_p... 0.00.293.304 I llama_model_loader: - kv 47: tokenizer.ggml.scores arr[f32,131072] = [0.000000, -1000.000000, -1000.000000... 0.00.298.026 I llama_model_loader: - kv 48: tokenizer.ggml.token_type arr[i32,131072] = [2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, ... 0.00.298.030 I llama_model_loader: - kv 49: tokenizer.ggml.bos_token_id u32 = 1 0.00.298.031 I llama_model_loader: - kv 50: tokenizer.ggml.eos_token_id u32 = 2 0.00.298.032 I llama_model_loader: - kv 51: tokenizer.ggml.add_bos_token bool = false 0.00.298.032 I llama_model_loader: - kv 52: tokenizer.ggml.add_eos_token bool = false 0.00.298.035 I llama_model_loader: - kv 53: tokenizer.chat_template str = {%- macro visible_text(content) -%}\n ... 0.00.298.036 I llama_model_loader: - kv 54: general.quantization_version u32 = 2 0.00.298.037 I llama_model_loader: - kv 55: general.file_type u32 = 25 0.00.298.037 I llama_model_loader: - kv 56: quantize.imatrix.file str = /gemini/space/shixq7/imatrix.dat 0.00.298.038 I llama_model_loader: - kv 57: quantize.imatrix.dataset str = /gemini/space/shixq7/calibration.txt 0.00.298.039 I llama_model_loader: - kv 58: quantize.imatrix.entries_count u32 = 592 0.00.298.039 I llama_model_loader: - kv 59: quantize.imatrix.chunks_count u32 = 8 0.00.298.046 I llama_model_loader: - type f32: 406 tensors 0.00.298.047 I llama_model_loader: - type q6_K: 1 tensors 0.00.298.047 I llama_model_loader: - type iq4_nl: 243 tensors 0.00.298.047 I llama_model_loader: - type bf16: 327 tensors 0.00.298.050 I print_info: file format = GGUF V3 (latest) 0.00.298.053 I print_info: file type = IQ4_NL - 4.5 bpw 0.00.298.055 I print_info: file size = 18.72 GiB (5.15 BPW) 0.00.298.996 E llama_model_load: error loading model: unknown model architecture: 'xing4_0' 0.00.299.005 E llama_model_load_from_file_impl: failed to load model 0.00.299.010 E cmn common_init_: failed to load model '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.299.015 E srv load_model: failed to load model, '/home/mrx/models/Xing4.0-29B-A4B-GGUF/xing4_0-29b-IQ4_NL.gguf' 0.00.299.017 I srv operator(): operator(): cleaning up before exit... 0.00.299.881 E srv llama_server: exiting due to model loading error
Notes: Automated llama-server KV-depth sweep (mtp-1, concurrency=8). Existing depth is contextTokens; PP=512, TG=256, warmups=1, repetitions=3, Q8_0 K/V. Arithmetic mean and population standard deviation of measured repetitions. PP/TG server durations are aligned to client completion timestamps for concurrent phase spans; aggregate-all includes SSH/HTTP wall time. Synthetic repeated-token prompts. Engine was reused as installed; no build was performed. Publication projection of reviewed source measurements; values and raw evidence unchanged. Single GPU requested — startup failed; placement unverified. Installed runtime rejected architecture xing4_0. No successful throughput or observed GPU placement is claimed.
Source article: /benchmarks/
Raw log path: research/benchmarks/logs/trpro/2026-10-15-independent-gpus/remaining-campaign/xing40-iq4-nl-concurrency-physical1

Compare measured entries