What’s up, what’s down and what are you not sure about?

Let us know what you set up lately, what kind of problems you currently think about or are running into, what new device you added to your homelab or what interesting service or article you found.

Personally I’m finally reaping the fruits of my labour and enjoy my stable homelab without doing much. One node went down recently and the other took over until I restarted so I was not in a hurry to fix things. Enjoying family time and only running updates that aren’t automated (yet). I’m about to dig a bit deeper into logging, probably setting up central log collection like Loki at some point, but not yet.

  • Bluefruit@lemmy.worldEnglish
    8·
    2 days ago

    Getting ready to move from out of the woods and back to civilization with my partner.

    Not looking forward to having neighbors above or below me but I’m very excited to have internet that doesnt fucking suck.

    Once were moved and a bit more settled, I’m gonna start really digging into to selfhosting things. I have the hardware, a couple HP mini PCs that will run home assistant and probably a server for various docker things. Nextcloud and immich seem to be the things I’ve found i wanna use so far. I already have a NAS set up, but was having am issue with it not booting if a monitor isnt plugged in. I bought a dummy plug for it but haven’t tried it out yet.

    Will also be setting up an AI server for local LLM use. Hope to train one to fit my needs once I pull the trigger on 3060 12GB card but need to figure out what other parts I’ll use. Might upgrade my main rig and use the parts from that, or maybe I’ll buy a old dell and fix it up. Not sure yet.

    Lots of ideas, so little time lol.

    • Jakeroxs@sh.itjust.worksEnglish
      2·
      19 hours ago

      Might want a bigger GPU, I have a 3080ti and the 12gb is pretty limiting in terms of how large a model you can use, or like one thing I was hoping to do was essentially replace Google Assistant/Gemini and can’t realistically run a good model and the STT/TTS off the one gpu.

      • Bluefruit@lemmy.worldEnglish
        2·
        17 hours ago

        Thats why i was considering training my own model if possible. Ive been toying around with kobold.CPP and gpt4all which both have RAG implementations.

        My idea is to essentially chat with documentation and as a separate use case, have it potentially be a AI search engine but locally hosted. I do still prefer to search myself, but fuck man, searches have gotten so bad, and the kobold.CPP web lookup feature was pretty neat IMO.

        So yea you’re not wrong, I’m just hoping that if in train it and or give it documentation it can reference when answering, it will be suitable. Mostly AI has been good for me as kind of a rubber ducky when troubleshooting and helping me search for things when I have some specific question and in don’t want “top 5 things vaguely related to your question” results.

        • Jakeroxs@sh.itjust.worksEnglish
          2·
          16 hours ago

          Interesting, I mainly have used text generation webui which has a search support plug in, kinda nifty to use my searxng instance for it. It’s a bit finicky though.

          Another thing to keep in mind then (apologies if this is just repeating info you already know), you’d also want to keep in mind your total potential context size in relation to the model size, since both take up VRAM. Reading search results/pages can eat up a lot

          • Bluefruit@lemmy.worldEnglish
            2·
            15 hours ago

            Yea I’m aware but I appreciate the insight :) so far my local ai experience has been lack luster so I’m hoping that training and RAG will make up for the context size at least a little. Ifnit can answer accurately in the first place, it may not need as big of a context window.

            If you haven’t tried using RAG in some form, I would recommend giving it a go. Its pretty cool stuff, helps make models answer more accurately based on the documentation you give them though in my case, ive had limited success. Tbh, chatgpt has become my last resort when I just wanna get something done but I don’t like using it due to the privacy concerns, not to mention the ethical issues I have with ai training in general from big tech.

            How is searxng BTW? Would you say its good to host or do you use a normal search engine more often? Or do you just use it for the AI search plugin?

            Ive actually been thinking about using it rather than duckduckgo but was also hopeful the search index they are working on would be enough to satisfy my needs, or that a self hosted AI enabled search engine would work well enough when I need it.

            • Jakeroxs@sh.itjust.worksEnglish
              2·
              15 hours ago

              I’ve completely replaced my searching with searxng, it is a little slower and ofc if I have an outage or something at home I have to go back to a different search temporarily but overall I like it a lot.

              It was one of the first things I set up last year with my homelab because I am attempting to degoogle a fair amount, the Ai search stuff was just a fun test