Comment on Is anyone self-hosting their own AI?
rumba@lemmy.zip 1 week ago
Ollama container, 2070 super, 3-7b, meta and qwen
I use it to ingest tons of news rss, find dupes, summarize, and then post that output to mesh a mesh network.
I also use it for a self hosted mealie to figure out recipe->ingredient.
I also use it for receipts. I translate pdf -> txt, feed the text to qwen, ask it who is the vendor, what is the price, what it a reasonable memo for the receipt, and give a dozen categories, which is best.
StateMachine@lemmy.zip 1 week ago
I am actually very interested in the bed feed thing. How do you find duplicates in the news feeds? Wouldn’t you have to pull the actual articles and then have MB of data to shift through which would exceed the window? Do you use a vector DB? Do you just have it use curl or do you pay for an Internet API service?
rumba@lemmy.zip 1 week ago
Pardon the mess, this turned out to be quite a lot of text.
No vector DB, no paid API, no full articles. Plain Python pulling free RSS feeds, plus Ollama running on my own 2070 Super. RSS already gives you the headline and a short blurb, so there are never megabytes to sift through. Every Ollama call is one story at a time, so the context window isn’t an issue. Llama3.1:8b for summaries and scoring. Gemma4:e4b for clickbate work, nomic-embed-text for headline duplicate vectors.
Find duplicates:
Python: strip each headline down to keywords (drop words like “the”, “says”, “update”) and only compare headlines that share an uncommon word. Two headlines match if they share enough keywords.
Python: sends batches of 128 headlines to Ollama
Ollama: turns each headline into a vector using nomic-embed-text
Python: compares the vectors, and anything over the config threshold counts as the same story. The vectors just live in memory for the run, no database needed. If Ollama is down, the keyword match still works on its own.
Regen short summaries:
Ollama: Generate a 2 sentence summary from the feed text if available
Get rid of clickbait:
Ollama: Detect clickbait from the headline only. Classify this news headline. “informative”: it states the news itself (who did what, what happened, what was found). Dramatic or emotional wording is fine if the fact is stated. “teaser”: it withholds the key fact to make you click: e.g. “The Surprising Reason…”, “The Class That…”, “Why X…”, “How X became…”, “What we know about…”, “Is X about to…?”, “Then things got weird.”
Unwrap clickbait and rewrite the headline if possible, otherwise drop the story
Ollama: A clickbait headline hides a key fact. Question it leaves open: {withheld}
Does the text below explicitly answer that exact question? Mentioning the topi talking around it, or answering a different question does not count.
“answer”: the answer quoted or closely paraphrased from the text, or “” if the text doesn’t give it. “headline”: if answered, a plain, factual replacement headline (at most 90 cha states that answer, using only facts stated in the text. Keep the text’s certainty: …
Respond only with JSON: {“answer”: “”, “headline”: “”}
Headline: {title} Text: {text}
Score each headline
Ollama: return a summary and an importance score in the same call:
SCALE = (“1-3 niche, local or trivial; 4-6 notable; 7-8 major national or inte “9-10 historic events or emergencies affecting millions right now.”)
Python: rank = importance + a bonus for more outlets carrying it - points for age
Don’t put US articles in World unless they have non-US countries involved
Ollama: Which countries is this news story about? List the country where it takes place and every other country that is a real party to it (acting, targeted, negotiating, fighting, trading, sanctioned). Use common English country names, e.g. “United States”, “United Kingdom”. Ignore countries only mentioned in passing or as background. Always name at least one country: if the text doesn’t say wh give the country where it most likely takes place.
Respond only with JSON: {“countries”: [”…"]}
Headline: {title} Text: {text}
Python: if the only country is the US, it’s left out of World
Everything gets cached per story, so each one only goes through Ollama once.
StateMachine@lemmy.zip 1 week ago
Wow, this is an absolutely wonderful write-up, thank you!