Using LoRA to Impersonate Friends
Or "how I GPT-ed" a chatroom
What if you could take a chat room with multiple strong personalities in it and trained it against a language model? Would you see the same personalities stick out? Or would it be a coherent mess?
Since LLaMa’s release, many people have been fine tuning LoRA models on top of it. With the help of other open source platforms, this is now easier than ever. The results are hilarious.
If I were to summarize:
It accurately captured each person’s personality, with enough data it can confidently say something that sounds like what that person would have said
The more the person talked in the chat, the more often they would show up
It doesn’t always answer the prompt correctly, but it has responses that could have resulted in a conversation about the topic and flow of conversation
It makes up a lot, for example posting a link that never really existed, or talking about a topic that no one ever talked about
It’s important to structure your source data set that’s very clean, mine ended up with too many artifacts
Before showing example prompts and responses, let’s go through what I did if you actually want to do this yourself.
The first step was deciding on what I wanted to train data on, and where that data would come from. In my previous experiments, having too little data does not move the needle enough to sway LLMs to the information you are trying to teach it. My first instinct was to use my 888 Investment Chat, which runs all day and would have tens of thousands of messages. It also happened to be that Facebook allows message exports! To get the messages from Facebook Messenger, I followed the instructions at https://www.facebook.com/dyi/ and selected export on Messages. Surprisingly the export had massive files but the first block of messages contained the text of the last 3 years:
Once I had a folder of messages, I needed to combine them into one file, however this would be a massive copy and paste effort. The answer was to use code, but what if coding it took just as long? Enter other AI platforms...
For the actual coding part, I used Codeium on PyCharm IDE which was only a matter of typing “#Loop through folder and combine json into a text file” and within a few minutes…I was done? My mind was blown, but I still needed to fix a few formatting errors. Enter ChatGPT, all I did was ask, “Can you fix this code?” and bam, it found my errors and gave me last last bit of code. I know in the latest version of Codeium, they have chat built in on VS Code, but surprisingly, this entire process only took a few minutes. We are living in a crazy world now, but okay, back to the project…
I ran the program and it worked! I have to say, my mind was blown that with all these AI tools I could quickly put some code together and get the task done and thus, I had an output that was both structured and quite big (42 MB):
Next up, I needed to run a training model with this on LLaMa. To do this, I plugged this file into Oobabooga (there is actually a one click installer now). You can run this platform on your computer but I recommend a beefy GPU card. I currently run an Nvidia 4090 and even this 42MB file will take a long time to process.
You’ll also need a Llama file, for this experiment I used this model - which can be downloaded via the download-model.bat file.
Finally to run the training, you’ll need to place the json or txt file into the \text-generation-webui\training\datasets folder. When you run Oogabooa (or Text Generation UI), select the Llama model and the “Training” tab.
You can see my settings below, I Just changed LoRA Rank and LoRA Alpha to 256 and 512. If you are using a JSON, you can select from the Dataset menu at the bottom, or if a large raw text file, click on the “Raw Text File” option at the bottom. If your file is in the folder click refresh and you should see it in the drop down.
Now the long part, after clicking on “Start LoRA Training” be prepared to wait a while if it’s a long file
Still waiting…
Alright! After you are done training the model, you can click on the “Model” tab and select the LoRA you created.
I decided to put this to work by asking it to just simulate a chatroom:
Surprisingly, it got some of the personalities correct, Hanson in general asks a lot of questions and asked what “simulate” meant, it also did a long LOLLLLL which is also his style
I totally expected the next question to be answered by Albert, which was “Should I buy japanese whiskey”:
Immediately it was Yes sir and wow damn, two responses I’d expect and it even got Albert correct.
Someone named Ben asked what the bot would say in his place:
Funny enough, Ben’s most common interactions are emojis, so this is indeed what his response would have been
I decided to ask a more direct question, which messed up the LLM a bit:
Michael’s response “Just tell her to buy eth” is hilariously appropriate in the context as well as something he would say. Also, Hanson asked more questions.
I then asked “What crypto should I buy?”
Again, this accurately captured many of the personalities in the chat including their style of responses, though the question was not really answered. It went on a conversation by itself between the chat members on crypto tax and how the government would track you…which is actually something that probably would have happened in that chatroom.
With these LLMs, you also have the option to let them keep going too, especially in a chatroom type dataset, however, I already get that in real life…
All in all, this was a fun experiment, there are some other observations, but ultimately my next try needs better formatted data. Let me know if you have any suggestions and I’d love to try them out. It’s only a matter of time before LoRA type fine tuning will be everywhere with company and personal data.
so think is that how to integrate it into project