Also check out LLM Studio and GPT4all. Both of these let you run private ChatGPT alternatives from Hugging Face and run them off your ram and processor (can also offload to GPU).
I’d also recommend Oobabooga if you’re already familiar with Automatic1111 for Stable diffusion. I have found being able to write the first part of the bots response gets much better results and seems to make up false info much less.
Mistral is thought to be almost as good. I’ve used the latest version of mistral and found it more or less identical in quality of output.
It’s not as fast though as I am running it off of 16gb of ram and an old GTX 1060 card.
If you use LLM Studio I’d say it’s actually better because you can give it a pre-prompt so that all of its answers are within predefined guardrails (ex: you are glorb the cheese pirate and you have a passion for mink fur coats).
There’s also the benefit of being able to load in uncensored models if you would like questionable content created (erotica, sketchy instructions on how to synthesize crystal meth, etc).
I can’t find a way to run any of these on my homeserver and access it over http.
It looks like it is possible but you need a gui to install it in the first place.
Also check out LLM Studio and GPT4all. Both of these let you run private ChatGPT alternatives from Hugging Face and run them off your ram and processor (can also offload to GPU).
I’d also recommend Oobabooga if you’re already familiar with Automatic1111 for Stable diffusion. I have found being able to write the first part of the bots response gets much better results and seems to make up false info much less.
There’s also koboldcpp, which is fairly newbie friendly.
And llama file, which is a chat bot in a single executable file.
Are they as good as chatgpt?
Mistral is thought to be almost as good. I’ve used the latest version of mistral and found it more or less identical in quality of output.
It’s not as fast though as I am running it off of 16gb of ram and an old GTX 1060 card.
If you use LLM Studio I’d say it’s actually better because you can give it a pre-prompt so that all of its answers are within predefined guardrails (ex: you are glorb the cheese pirate and you have a passion for mink fur coats).
There’s also the benefit of being able to load in uncensored models if you would like questionable content created (erotica, sketchy instructions on how to synthesize crystal meth, etc).
I’m sure that meth is for personal use right? Right?
Absolutely. Synthesizing hard drugs is time consuming and a lot of hard work. Only I get to enjoy it.
No one gets my mushrooms either ;)
Can you provide links for those? I see a few and don’t trust search results
No.
Something i am really missing is a breakdown of How good these models actually are compared to eachother.
A demo on hugging face couldnt tell me the boiling point of water while the authors own example prompt asked the boiling point for some chemical.
I can’t find a way to run any of these on my homeserver and access it over http. It looks like it is possible but you need a gui to install it in the first place.