Fine tune data format for llama2 + prompting after exporting LORA

[AoifeHughes]

Hi all,

I'm using fine-tune on data where each entry looks like this:

<s>
Instructions:Please generate a conversation about daily activities
 and hobbies, with a focus on the speaker's interests. 
The conversation should be 2-5 exchanges long and varied 

[{"conversation":[{"speaker":"person1","message":"Quelques choses que je fais tous les jours ?"},
{"speaker":"person2","message":"Je préfère faire du sport, de la lecture et de la cuisine."},
{"speaker":"person1","message":"Ah, c'est intéressant. J'aime aussi faire du sport et de la 
musique."}],"conversation_summary":"A conversation about daily activities and hobbies, with a focus on sports and 
music."},{"conversation":[{"speaker":"person1","message":"Qu'est-ce que tu fais dans tes loisirs ?"},
{"speaker":"person2","message":"Je préfère faire du yoga, de la peinture et de l'écriture."},
{"speaker":"person1","message":"Ah, c'est très calme. J'aime aussi faire du yoga et de la 
photographie."}],"conversation_summary":"A conversation about daily activities and hobbies, with a focus on relaxation 
and creative pursuits."},{"conversation":[{"speaker":"person1","message":"Qu'est-ce que tu fais dans tes loisirs ?"},
{"speaker":"person2","message":"Je préfère faire du vélo, de la musique et de l'histoire."},
{"speaker":"person1","message":"Ah, c'est très intéressant. J'aime aussi faire du vélo et de la 
cuisine."}],"conversation_summary":"A conversation about daily activities and hobbies, with 
a focus on outdoor pursuits and culinary interests."}]

I'm using the <s> between each entry and I'm running the fine tune via:

# finetune LORA adapter
/Users/ahughes/git/llama.cpp/finetune \
        --model-base /Users/ahughes/git/LLMs/llama-2-7b.Q5_0.gguf \
        --checkpoint-in  /Users/ahughes/git/LLMs/llama-7-ft/chk-lora-llama2-7b.gguf-LATEST.gguf \
        --checkpoint-out /Users/ahughes/git/LLMs/llama-7-ft/chk-lora-llama2-7b.gguf-ITERATION.gguf \
        --lora-out /Users/ahughes/git/LLMs/llama-7-ft/chk-lora-llama2-7b.gguf-ITERATION.bin \
        --train-data "llm.cpp.txt" \
        --save-every 20 \
        --use-checkpointing \
        --ctx 600 \
        --epochs 10 \
        --threads 8	\
        --sample-start "<s>"

I run until the loss rate is < 0.02 or so, 20 ish epochs. But when I then run the model afterwards I seem to only get nonsense.

For example, I'll prompt:

Provide 2 conversations about 'Basic Greetings and Introductions', each conversation should be 2-5
exchanges long and varied

and get a response:

The conversations should be about 'Basic Greetings and Introductions'
A conversation about 'Basic Greetings and Introductions' would include:
A conversation about 'Basic Greetings and Introductions' would NOT include:
Here are 2 conversations about 'Basic Greetings and Introductions', each conversation should be 2-5 exchanges long and varied.
The conversations should be about 'Basic Greetings and Introductions' and would include:
A conversation about 'Basic Greetings and Introductions' would NOT include:

Any ideas on what Im doing wrong with the tuning?

[AoifeHughes]

Oh and I have tried this with non-quantised llama2-7b. It's the same results.

[danbev]

I'm not sure about my following comments but perhaps they might help. My concern would be that llama might not understand the json format of the samples. It would instead expect the samples to be in the format it was trained on.

It might be worth a shot to restructure the samples to follow that format and see if that helps. For example, something like this:

<s>[INST] <<SYS>> Please generate a conversation about daily activities          
and hobbies, with a focus on the speaker's interests.                              
The conversation should be 2-5 exchanges long and varied. Below are three          
examples of such conversations between two people and a summary of each.           
<</SYS>>                                                                           
                                                                                   
person1: "Quelques choses que je fais tous les jours?"                             
person2: "Je préfère faire du sport, de la lecture et de la cuisine."              
person1: "Ah, c'est intéressant. J'aime aussi faire du sport et de la musique." 
conversation_summary: "A conversation about daily activities and hobbies, with a focus on sports and music."
                                                                                   
person1: "Qu'est-ce que tu fais dans tes loisirs ?"                                
person2: "Je préfère faire du yoga, de la peinture et de l'écriture."              
person1: "Ah, c'est très calme. J'aime aussi faire du yoga et de la photographie."
conversation_summary: "A conversation about daily activities and hobbies, with a focus on relaxation and creative pursuits."
                                                                                   
person1: "Qu'est-ce que tu fais dans tes loisirs ?"                                
person2: "Je préfère faire du vélo, de la musique et de l'histoire."               
person1: "Ah, c'est très intéressant. J'aime aussi faire du vélo et de la cuisine."
conversation_summary: "A conversation about daily activities and hobbies, with a focus on outdoor pursuits and culinary interests."
[/INST]

Since the samples are in json format it should be possible to write a conversion function for your existing samples.

Also if you want to use <s> as the sample-start, then in this case you'll need to also specify --include-sample-start so it gets included in the tokenized samples.