• 8 Posts
  • 756 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle



  • I love fully local stuff, but the LLM part seems very expensive. Even for such a simple thing as managing our lights and music.

    So, the simple things should just be voice command (not recognition, limited vocabulary), <wakeword> kitchen light on, what’s todays weather (assuming you automatically download it and have it there, needs TTS like whisper). That can run on a recent potato. Somewhat more complex things need full voice recognition e.g. Play me x songname by y artist. Laptop CPU should handle that fine. You’ll only need a full LLM for more general inquiries, e.g.<wakeword> What mess has Trump made today?, which will need web access seeing as small LLMs don’t have a lot of world knowledge (although you can self host Wikipedia and point it at that).

    If you have a desktop computer and a GPU with 16+GB VRAM (or even smaller using RAM and a MoE LLM) that’ll do the job with reasonable smarts e.g. Qwen 3.6 35BA3B, Gemma 4 12B or 26BA4B. You may have to wait a minute or two. Mine pulls around 55W at idle (7800XT) and does double duty as a NAS and gaming rig, or you can have it sleep most of the time and let the laptop wake it up when necessary.





  • Just because it’s not your use case (and maybe you’d find a use if it was available), doesn’t mean it’s not a common one. Market opportunities get missed all the time. People have been making do with offline HDDs, which is a waste of their potential, and they’re now victim to the RAMpocalypse price hike too.

    Just as an example, people would probably take a lot more video with their oh so capable phones if there was somewhere to put TBs of it cheaply.


  • Honestly, the world is crying out for a new consumer offline backup media, hasn’t been anything since BluRay. Even just whacking a USB3 onto LTO5/6 and not having it cost the earth for drive or media would sell like hot cakes.

    Something a lot nicer is cooking in China (Microsoft has been sitting on it as Project Silica for years now). That looks to be a 360TB CD sized disc good for 10000+ years cold storage, currently $30K writer and $6K reader, but that was the case with CDs too.

    I can see why our corpo lords and masters wouldn’t want us having a device that let us fit all of Netflix in the palm of our hands, but whoever mass produced it and brought the price down to reasonable would make an absolute fortune. Here’s hoping the latter greed wins out, it doesn’t seem like the drives are made with any sort of unobtianium. A new era of CD burners would be highly welcome.



  • “hundreds of millions of dollars annually for a creative fund”

    How gullible would you have to be to believe the kleptocrats would keep their word on that? Their pattern is steal what’s not nailed down (or likely to sue), hide it as best possible, and pay only if called out and then forced to under law. If this happens, those responsible for Australia’s decision are either utter morons or bribed to the gills (much cheaper than a creative fund) or both.

    Follows the pattern of just giving resources away to any company that asks, for a pittance, but social resources this time.





  • Thing is, most of those data centers and video GPGPUs are dark and will likely stay so. The cards will be obsolete by the time the centers are built, the power sourced, they’ve been sued and injunctioned etc., etc. . (reminiscent of all that dark fiber left after the OG internet bubble pop)

    The RAMpocalypse is mostly market manipulation courtesy of SamA grabbing 40% of global supply (of wafers, not even chips yet) from two of the three memory companies on the same day. In a better (the old) world he’d have been put in jail for this and fraud already.

    Which is to say, the demand is mostly fictional and FOMO and the big three memory companies are leaning hard into it because profit, and, the price never comes all the way back down, also a win for them. They make huge profit (hence making 5 year price fix deals with stupid people) until the Chinese companies ramp up production and the game ends. The actual RAM may or may not actually end up used, depending on when the bubble pops and what happens after.

    Just rampant greed doing what it does, SNAFU.




  • Yup, there’s a cycle between centralization and edge. Started with centralization, mainframes, went edge with the first PCs (and game consoles) and ever since corpos have been trying to pull it back to the center in waves. Thin client, cloud compute, arguably phones (as apps processing in the cloud), Geforce Now, AI. So far it’s always gone back to the edge for most of the population, except for niche cases (or not in the case of phones). As internet gets faster and more reliable the chance of it sticking longer in the central zone increases (IMO).