Had the same problem and someone guided me to the hugging face documents/tutorials.
They are quite nice to get a local model up and running, play around with it, how to fine tune it and connect it with agents

Haven’t tried much, but the articles were exactly what I was looking for
Hope it helps you as well