In the past couple of weeks, I’ve been working on a project to make Steins;Gate characters come to life through AI – the usage of certain machine learning programs. So far, I’ve looked into GPT-2 (AI to generate somewhat coherent text) and finetuned the 117M model from OpenAI for GPT-2 to the ingame lines of Faris, Okabe and Kurisu, creating a new model for each character that enables GPT-2 to behave like that character. GPT-2 and the programs to retrain it can be found on GitHub. The models can be tested through a Discord bot, whose source code is available on GitHub. It is currently running on my Discord server. If you’d like to test it on my Discord server, please send me a message on Reddit (DaruHacks) or Twitter (@kaitocross)

Here are some samples of text lines, that have been generated by the GPT-2 models using the discord bot and the command-line training tool. It’s an Imgur Image Album full of crazy lines generated by GPT-2.

However, there’s more that can be done. Using Tacotron2, it seems possible that we can create a Text-To-Speech AI that uses the voice we choose and create! It has to be trained on many pairs of textlines & voicerecordings of the desired voice. The Steins;Gate visual novel provides these pairs for their characters, making a Kurisu Text to Speech AI possible. This GibHub project of mine outlines how to make the game content readable for GPT-2 and Tacotron2. The text-voice pairs generated for Tacotron2 have to be proofread, proofheard and corrected before training the Tacotron2 on that dataset. To help do so, I have created TacoTranscribe. I still have to write a README for it to explain how to use it correctly.

However, since Kurisu has 2,8k text-voice pairs, I ask for your help in proofreading, proofhearing and correcting the pairs. If you’d like to help, please send me a message on Reddit (DaruHacks) or Twitter (@kaitocross)

If we manage to couple the output of a Speech to Text system with GPT-2, and then couple GPT-2’s output with Tacotron2, we can make an Amadeus Kurisu (that is funnier than the Android App).

Leave a Reply

Your email address will not be published. Required fields are marked *

I confirm

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.