Not personally surprised that a MoE model performs so well.
I used Mixtral a lot for coding Rust, and it had qualities no other model had except GPT 3.5 and later Claude Sonet. The funny thing is Mixtral was based on Llama 2 which was not trained on code that much.
DeepSeek v3: 671B parameters on total, and 37B activated sounds very good even though impossible to run locally.
Question if some people happen to know: For each query it activates just that many of parameters, 37B, and no more?
But you don't need cluster networking or nvlink so much like with splitting out llama 405B. You could even split them out with friends over internet levels of bandwidth.
I used Mixtral a lot for coding Rust, and it had qualities no other model had except GPT 3.5 and later Claude Sonet. The funny thing is Mixtral was based on Llama 2 which was not trained on code that much.
DeepSeek v3: 671B parameters on total, and 37B activated sounds very good even though impossible to run locally.
Question if some people happen to know: For each query it activates just that many of parameters, 37B, and no more?