Failed to replace the LLM mode to QWen2.5

[AlexHe99]

Replace with QWen2.5-3B-Instruct but failed with log. Any update to support QWen2.5?

 13 Namespace(model_path='./checkpoints', llm_path='/DATA/Qwen/Qwen2.5-3B-Instruct', top_k=20, top_p=0.8, temperature=0.8, ip='172.16.75.123', port='8081', max_users=3, llm_exec_nums=1, timeout=180)
 14 [2025-02-13 17:06:53.299] the number of speech encoder params: 341.3681640625M
 15 ^MLoading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 15.74it/s]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,     15.72it/s]
 16 /home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.wei    ght_norm`.
 17   WeightNorm.apply(module, name, dim)
 18 [2025-02-13 17:06:55.275] Checkpoint: loading from checkpoint ./checkpoints/audiollm/final.pt for GPU
 19 [2025-02-13 17:06:58.125] reading a config file from ./checkpoints/decoder/model.json
 20 [2025-02-13 17:06:58.126] reading a config file from ./checkpoints/decoder/model.json
 21 [2025-02-13 17:06:58.126] reading a config file from ./checkpoints/decoder/model.json
 22 [2025-02-13 17:07:01.790] Removing weight norm...
 23 [2025-02-13 17:07:01.797] Removing weight norm...
 24 [2025-02-13 17:07:01.800] Removing weight norm...
 25 [2025-02-13 17:07:01.803] Removing weight norm...
 26 [2025-02-13 17:07:01.804] Removing weight norm...
 27 [2025-02-13 17:07:01.809] Removing weight norm...
 28 [2025-02-13 17:07:01.821] Start Freeze-Omni sever
 29  * Serving Flask app 'server'
 30  * Debug mode: off
 31 ^[[31m^[[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.^[[0m
 32  * Running on https://172.16.75.123:8081
 33 ^[[33mPress CTRL+C to quit^[[0m
 34 10.176.245.8 - - [13/Feb/2025 17:07:03] "GET /socket.io/?EIO=4&transport=polling&t=mxce1ske HTTP/1.1" 200 -
 35 /home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/transformers/models/qwen2/modeling_qwen2.py:216: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas bac    kend to hipblas (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:310.)
 36   freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
 37 10.176.245.8 - - [13/Feb/2025 17:07:04] "GET /socket.io/?EIO=4&transport=polling&t=mxcn6kxp&sid=ytVjS-kcmfultMxuAAAA HTTP/1.1" 200 -
 38 message handler error
 39 [2025-02-13 17:07:04.209] Traceback (most recent call last):
 40 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 824, in _handle_event
 41     ret = handler(auth)
 42           ^^^^^^^^^^^^^
 43 [2025-02-13 17:07:04.210] TypeError: handle_connect() takes 0 positional arguments but 1 was given
 44 [2025-02-13 17:07:04.210]
 45 During handling of the above exception, another exception occurred:
 46
 47 [2025-02-13 17:07:04.210] Traceback (most recent call last):
 48 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/engineio/server.py", line 450, in run_handler
 49     return self.handlers[event](*args)
 50            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 51 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 652, in _handle_eio_message
 52     self._handle_connect(eio_sid, pkt.namespace, pkt.data)
 53 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 538, in _handle_connect
 54     success = self._trigger_event(
 55               ^^^^^^^^^^^^^^^^^^^^
 56 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 617, in _trigger_event
 57     return handler(*args)
 58            ^^^^^^^^^^^^^^
 59 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 282, in _handler
 60     return self._handle_event(handler, message, namespace, sid,
 61            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 62 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 826, in _handle_event
 63     ret = handler()
 64           ^^^^^^^^^
 65 [2025-02-13 17:07:04.210]   File "/DATA/Demo/Freeze-Omni/Freeze-Omni/bin/server.py", line 326, in handle_connect
 66     connected_users[sid].append(GlobalParams(tts_pool, pipeline_pool))
 67                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 68 [2025-02-13 17:07:04.210]   File "/DATA/Demo/Freeze-Omni/Freeze-Omni/web/parms.py", line 44, in __init__
 69     init_outputs = self.pipeline_obj.pipeline_proc.speech_dialogue(None, stat='pre',
 70                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 71 [2025-02-13 17:07:04.210]   File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/pipeline.py", line 62, in speech_dialogue
 72     past_key_values = self.model.set_system_role(extra_inputs)
 73                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 74 [2025-02-13 17:07:04.210]   File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/audioLLM.py", line 241, in set_system_role
 75     _, past_key_values, stat, _ = self._generate_one_step(
 76                                   ^^^^^^^^^^^^^^^^^^^^^^^^
 77 [2025-02-13 17:07:04.210]   File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/audioLLM.py", line 413, in _generate_one_step
 78     state_logits = self.predictor_head(
 79                    ^^^^^^^^^^^^^^^^^^^^
 80 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
 81     return self._call_impl(*args, **kwargs)
 82            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 83 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
 84     return forward_call(*args, **kwargs)
 85            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 86 [2025-02-13 17:07:04.210]   File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 125, in forward
 87     return F.linear(input, self.weight, self.bias)
 88            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 89 [2025-02-13 17:07:04.210] RuntimeError: mat1 and mat2 shapes cannot be multiplied (36x2048 and 3584x4)
 90 10.176.245.8 - - [13/Feb/2025 17:07:04] "POST /socket.io/?EIO=4&transport=polling&t=mxcn5ij9&sid=ytVjS-kcmfultMxuAAAA HTTP/1.1" 200 -

[heavendai]

@AlexHe99
The speed decoders are not aligned with qwen2.5, which is useless if qwen2.5 is successfully replaced.

[AlexHe99]

The Freeze-Omni is great for me and looking forward it will bed updated soon it with new SOTA model like Qwen2.5, DeepSeek-R1-Qwen2.5, etc.