Live
Black Hat USAAI BusinessBlack Hat AsiaAI BusinessPakistan’s peace plan a ‘critical opportunity’ for US-Iran talks ahead of Trump deadlineSCMP Tech (Asia AI)Why Microservices Struggle With AI SystemsHackernoon AIAgentic AI Vision System: Object Segmentation with SAM 3 and QwenPyImageSearchWhy APEX Matters for MoE Coding Models and why it's NOT the same as K quantsReddit r/LocalLLaMAAt least 80 different Microsoft Copilot products have been mapped out by expert, but there may be more than 100 — Microsoft doesn't have a singular list available, so AI consultant mapped out the myriad products - Tom's HardwareGNews AI MicrosoftGoogle Study: AI Benchmarks Use Too Few Raters to Be Reliable - WinBuzzerGNews AI benchmarkNvidia Stock Rises. This Issue Could Hamper Its Next-Generation AI Chips. - Barron'sGNews AI NVIDIABroadcom's CEO Has Line of Sight to $100 Billion in AI Chip Revenue. Is the Stock a Buy? - The Motley FoolGoogle News: AI‘This is 160-million-year-old Jurassic clay’: inside Es Devlin’s bid to reshape AI ethics – through potteryThe Guardian AI‘This is 160-million-year-old Jurassic clay’: inside Es Devlin’s bid to reshape AI ethics – through pottery - The GuardianGNews AI ethicsI gave Claude Code our entire codebase. Our customers noticed. | Al Chen (Galileo)lennysnewsletter.comGoogle DeepMind and Agile Robotics Combine Robotics Platforms - Automation WorldGoogle News: DeepMindBlack Hat USAAI BusinessBlack Hat AsiaAI BusinessPakistan’s peace plan a ‘critical opportunity’ for US-Iran talks ahead of Trump deadlineSCMP Tech (Asia AI)Why Microservices Struggle With AI SystemsHackernoon AIAgentic AI Vision System: Object Segmentation with SAM 3 and QwenPyImageSearchWhy APEX Matters for MoE Coding Models and why it's NOT the same as K quantsReddit r/LocalLLaMAAt least 80 different Microsoft Copilot products have been mapped out by expert, but there may be more than 100 — Microsoft doesn't have a singular list available, so AI consultant mapped out the myriad products - Tom's HardwareGNews AI MicrosoftGoogle Study: AI Benchmarks Use Too Few Raters to Be Reliable - WinBuzzerGNews AI benchmarkNvidia Stock Rises. This Issue Could Hamper Its Next-Generation AI Chips. - Barron'sGNews AI NVIDIABroadcom's CEO Has Line of Sight to $100 Billion in AI Chip Revenue. Is the Stock a Buy? - The Motley FoolGoogle News: AI‘This is 160-million-year-old Jurassic clay’: inside Es Devlin’s bid to reshape AI ethics – through potteryThe Guardian AI‘This is 160-million-year-old Jurassic clay’: inside Es Devlin’s bid to reshape AI ethics – through pottery - The GuardianGNews AI ethicsI gave Claude Code our entire codebase. Our customers noticed. | Al Chen (Galileo)lennysnewsletter.comGoogle DeepMind and Agile Robotics Combine Robotics Platforms - Automation WorldGoogle News: DeepMind
AI NEWS HUBbyEIGENVECTOREigenvector

Bench 2xMI50 Qwen3.5 27b vs Gemma4 31B (vllm-gfx906-mobydick)

Reddit r/LocalLLaMAby /u/ai-infos https://www.reddit.com/user/ai-infosApril 5, 20263 min read0 views
Source Quiz

Inference engine used (vllm fork) : https://github.com/ai-infos/vllm-gfx906-mobydick/tree/main Huggingface Quants used: QuantTrio/Qwen3.5-27B-AWQ vs cyankiwi/gemma-4-31B-it-AWQ-4bit Relevant commands to run : docker run -it --name vllm-gfx906-mobydick -v ~/llm/models:/models --network host --device=/dev/kfd --device=/dev/dri --group-add video --group-add $(getent group render | cut -d: -f3) --ipc=host aiinfos/vllm-gfx906-mobydick:latest FLASH_ATTENTION_TRITON_AMD_ENABLE="TRUE" OMP_NUM_THREADS=4 VLLM_LOGGING_LEVEL=DEBUG vllm serve \ /models/gemma-4-31B-it-AWQ-4bit \ --served-model-name gemma-4-31B-it-AWQ-4bit \ --dtype float16 \ --max-model-len auto \ --gpu-memory-utilization 0.95 \ --enable-auto-tool-choice \ --tool-call-parser gemma4 \ --reasoning-parser gemma4 \ --mm-processor-cache-gb 1

Could not retrieve the full article text.

Read on Reddit r/LocalLLaMA →
Was this article helpful?

Sign in to highlight and annotate this article

AI
Ask AI about this article
Powered by Eigenvector · full article context loaded
Ready

Conversation starters

Ask anything about this article…

Daily AI Digest

Get the top 5 AI stories delivered to your inbox every morning.

More about

modelbenchmarkreasoning

Knowledge Map

Knowledge Map
TopicsEntitiesSource
Bench 2xMI5…modelbenchmarkreasoningagenticagentgithubReddit r/Lo…

Connected Articles — Knowledge Graph

This article is connected to other articles through shared AI topics and tags.

Knowledge Graph100 articles · 197 connections
Scroll to zoom · drag to pan · click to open

Discussion

Sign in to join the discussion

No comments yet — be the first to share your thoughts!

More in Models