• 2 Posts
  • 117 Comments
Joined 3 years ago
cake
Cake day: July 1st, 2023

help-circle


  • Very cool!

    The voice sattelites like this and the Home Assistant voice from a year back are great and I love seeing more options for self hosted fully local devices like this.

    But honestly, what I find is missing for a fully local smart home setup is a “brain” for running the assistant LLM in a simple way.

    There is plenty of cloud solutions for connecting AI, but true local hosting requires a rig with a proper GPU if you want timely responses. I wonder if it is possible to build a purpose built “brain box” for Home Assistant that is small, not too expensive, fully local.

    It just sucks needing a rig with a RTX 3090 or whatever to get the full chain to run fully local. Small LLMs are stupid as dirt and our Home Assistant Green is in no way equipped to handle any form of LLM. I managed to get my gaming rig to host an LLM that was good enough, but I don’t want my gaming rig to be always on, and if I start gaming on it, the GPU has to deallocate all LLM business.

    I love fully local stuff, but the LLM part seems very expensive. Even for such a simple thing as managing our lights and music.