What do you use for offline text generation?
Added 2024-06-06 20:55:59 +0000 UTCI am gearing up to provide offline local LLM support and i'd like to support the current most popular tools for: text generation and hopefully voice generation.
Comments
I've used Convai.com - which allows you to upload almost unlimited text documents to train a character on. It is made for game creators so I was wondering if that's what you're using? I think it works really well (at least by itself) and has an emotion engine built in. I haven't attached it to any sort of game character, but eventually I'd like to. Or at least some kind of nice interface. The character I made is for a stop motion series where the character resembles an AI but she's animated by hand. The Convai.com character is trained on the back story as well as lots of real world stop motion how-to information. If you're interested this is a link to the chatbot online. It's something you might want to experiment with if you don't already know about it: https://convai.com/shared-character?id=98f1a66c-2372-11ee-b429-42010a400002
Marc
2024-07-29 23:34:53 +0000 UTChttps://discord.com/invite/X7pxCef6QB
sgthale
2024-07-07 04:47:20 +0000 UTCcan I have the discord link?
Ege
2024-07-07 04:45:27 +0000 UTCI use all of the ones you mention, but NovelAI allows for API use for my own dev projects so it's the one I've been using mostly, though not local per se, they encrypt the data and give you that API use bonus, so I'd be using it most likely with MyRobot or other companions you might be making in the future. The version 2 voice generation NAI API has is pretty great too, but no settings. You give it a seed and hope it's the right voice, though you can mix seeds and separate into 3 voice qualities in the payload and build a dictionary of seeds to get the right gender and quality. Also, you may want to have a different text for what they say vs printed because inflection is controlled in a weird way with special characters, punctuation, and capitalization, so I have sometimes write the text said as a printable version and the version I use to control inflection, but even breath, mumbling, and stuttering is supported. Just an example of my case use, I can get somewhat decent [conversation/read time] streamed results with Mini Orca and Phi-3 Mini Instruct on my RTX 2060 and have "meh" to fair, limited 3D performance, pretty good in between generations but it bogs things down during gen. Maybe it would be doable on a more modern GPU on my part, but I don't think mini Orca and phi-3 mini would be good unless you had a specifically trained version of them as they are tiny compared to LLM that can make immersive, steerable(?) results, while giving decent 3D. I might be missing something though. Sorry, I know no one asked but this particular thing is both interesting and important to me in products I have interest in and my own current project, LULZ. Pardon my rambling.
Benjamin Allen
2024-06-07 11:53:12 +0000 UTClm studio is pretty good for story stuff with an easy install process, ollama has become insanely easy to use with a lot of people using it as a base
VR AnimeTed
2024-06-06 21:11:56 +0000 UTCYeah the challenging part is just the voice gen. Really considering adding a chirp or Undertale style talking.
sgthale
2024-06-06 21:09:55 +0000 UTCOnly muddled around for a bit with NovelAI but seen many results from others using it that I forgot that other models exists out there. Whoops lol. Curious to see what you'll go with and how well it will integrate with the game.
DatGai
2024-06-06 21:01:12 +0000 UTC
