How to selfhost a llm?
How to selfhost a llm?
I'm trying to figure out how to host one myself. I'm trying to use barvarder and localai. But I am failing due to not enough knowledge and missing instructions. Any advice? did someone succeed with anything? I'd be happy to make other smaller steps at first as well. As long as I get somewhere.
You're viewing a single thread.
8
comments
If you want to be able to get into the nitty gritty or play with options besides just a chat, I recommend Text Generation WebUI.
Installing is pretty easy, then you just download your desired model from Hugging Face.
Or if you want to use it for roleplay or adventure style games, KoboldCPP is easy to set up.
You've viewed 8 comments.