communism

@communism@lemmy.ml

• ⁨Comment⁩ on ⁨[AIT] what is your selfhosted local LLM stack?⁩ ⁨⁨4⁩ ⁨days⁩ ago⁩:

  Literally just having ollama installed and some models that work well with the amount of VRAM I have. I don’t auto-start the daemon because I rarely use it. Occasionally I use it for proof-reading/feedback on things like cover letters, or debugging. I don’t let it write anything. If it has something useful to say it can tell me in the chat and I can implement it if it’s good. Usually I don’t need it though so it just sits on my disk for the most part.

• ⁨Comment⁩ on ⁨How do you monitor your servers?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  Yes, it can be a bit involved. I’d say I can spin up a mail server in 1–2 days though provided the OS is already installed and I have a domain name and port 25 isn’t blocked (that’s the main one that tends to be blocked). It may take longer for people who aren’t familiar with mail servers but it’s not some huge operation.

  You can just try spinning up a mail server for fun. If it goes badly then just don’t move over your dentist appointments to it. It’s only critical infra if you use it for critical things, which you probably shouldn’t till it’s stable.

• ⁨Comment⁩ on ⁨How do you monitor your servers?⁩ ⁨⁨2⁩ ⁨weeks⁩ ago⁩:

  I self-host my own email, have done so for years. I’ve not gotten pwned. Use an SSH key and a firewall, only expose the ports that should be public, have reasonable access controls, etc, just practise general cybersecurity—it applies to servers too.

  I know everyone complains about hosting email. But I’ve not run into major problems. My emails get through spam filters and I have good uptime. It’s also a fun learning experience.