setVeryLoud(true);

He / They

Software Developer

  • 5 Posts
  • 828 Comments
Joined 3 years ago
cake
Cake day: April 19th, 2023

help-circle














  • I actually did exactly that previously! I had both an RX 6800 XT and an RX 6600 in my system and I used the 6600 for video output. Unfortunately, this cuts my RX 6800 XT from PCIe 4 16x to PCIe 4 8x and severely slows down model loading for llama-swap. Joys of the X570!

    And yes, I do have it running right now with a bit of occupied VRAM, but I need to limit my model to 14 GB to leave 2 GB free for GNOME Shell. I really want one of those 64 GB UMA Mac Mini, I heard they work really well because the GPU has direct access to system RAM.





  • How has tool use been for you? I struggled a lot with tool use with Gemma and Qwen, to the point where I needed to build a healing layer.

    Regarding the coding harness, I was looking for something CLI-based or JetBrains-based, and I haven’t had much luck getting my local llama.cpp models playing ball with OpenCode. They keep losing context and misusing tools.

    I’m not too familiar with Apple containers as I’m running a full Linux stack, but I’ll give Pi a try, seems interesting! Does it work for coding tasks or is it strictly an “orchestrator”?