Question: I am trying to fine-tune the Mistral-7B-Instruct-v0.1-GPTQ model using SFTTrainer from trl. However, when running my script in Google Colab, I encounter the following error:

TypeError: SFTTrainer.__init__() got an unexpected keyword argument 'tokenizer'

Here is the relevant portion of my code:

import torch
from datasets import load_dataset, Dataset
from peft import LoraConfig, AutoPeftModelForCausalLM, prepare_model_for_kbit_training, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig, TrainingArguments
from trl import SFTTrainer, SFTConfig
import os

# Load and preprocess dataset
data = load_dataset("tatsu-lab/alpaca", split="train")
data_df = data.to_pandas()
data_df = data_df[:5000]
data_df["text"] = data_df[["input", "instruction", "output"]].apply(
    lambda x: "###Human: " + x["instruction"] + " " + x["input"] + " ###Assistant: " + x["output"], axis=1
)
data = Dataset.from_pandas(data_df)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GPTQ")
tokenizer.pad_token = tokenizer.eos_token

# Load and prepare model
quantization_config_loading = GPTQConfig(bits=4, disable_exllama=True, tokenizer=tokenizer)
model = AutoModelForCausalLM.from_pretrained(
    "TheBloke/Mistral-7B-Instruct-v0.1-GPTQ",
    device_map="auto"
)
model.config.use_cache = False
model.config.pretraining_tp = 1
model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)

# LoRA configuration
peft_config = LoraConfig(
    r=16, lora_alpha=16, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", target_modules=["q_proj", "v_proj"]
)
model = get_peft_model(model, peft_config)

# Training configuration
training_arguments = SFTConfig(
    output_dir="mistral-finetuned-alpaca",
    per_device_train_batch_size=8,
    gradient_accumulation_steps=1,
    optim="paged_adamw_32bit",
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    save_strategy="epoch",
    logging_steps=100,
    num_train_epochs=1,
    max_steps=250,
    fp16=True,
    packing=False,
    max_seq_length=512,
    dataset_text_field="text",
    push_to_hub=True
)

# Initialize trainer
trainer = SFTTrainer(
    model=model,
    train_dataset=data,
    peft_config=peft_config,
    args=training_arguments,
    tokenizer=tokenizer,  # This causes the error
)

trainer.train()

What I Have Tried:

• I checked the TRL documentation but couldn't find a tokenizer parameter in the SFTTrainer class.

• Removing tokenizer=tokenizer avoids the error, but then I get a different error about tokenization during training.

• I tried passing tokenizer inside training_arguments, but that didn't work either.

Question: Is SFTTrainer expecting the tokenizer to be handled differently in the latest versions of trl? How should I correctly pass the tokenizer to ensure training works?

I would appreciate any insights!

Question: I am trying to fine-tune the Mistral-7B-Instruct-v0.1-GPTQ model using SFTTrainer from trl. However, when running my script in Google Colab, I encounter the following error:

TypeError: SFTTrainer.__init__() got an unexpected keyword argument 'tokenizer'

Here is the relevant portion of my code:

import torch
from datasets import load_dataset, Dataset
from peft import LoraConfig, AutoPeftModelForCausalLM, prepare_model_for_kbit_training, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig, TrainingArguments
from trl import SFTTrainer, SFTConfig
import os

# Load and preprocess dataset
data = load_dataset("tatsu-lab/alpaca", split="train")
data_df = data.to_pandas()
data_df = data_df[:5000]
data_df["text"] = data_df[["input", "instruction", "output"]].apply(
    lambda x: "###Human: " + x["instruction"] + " " + x["input"] + " ###Assistant: " + x["output"], axis=1
)
data = Dataset.from_pandas(data_df)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("TheBloke/Mistral-7B-Instruct-v0.1-GPTQ")
tokenizer.pad_token = tokenizer.eos_token

# Load and prepare model
quantization_config_loading = GPTQConfig(bits=4, disable_exllama=True, tokenizer=tokenizer)
model = AutoModelForCausalLM.from_pretrained(
    "TheBloke/Mistral-7B-Instruct-v0.1-GPTQ",
    device_map="auto"
)
model.config.use_cache = False
model.config.pretraining_tp = 1
model.gradient_checkpointing_enable()
model = prepare_model_for_kbit_training(model)

# LoRA configuration
peft_config = LoraConfig(
    r=16, lora_alpha=16, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", target_modules=["q_proj", "v_proj"]
)
model = get_peft_model(model, peft_config)

# Training configuration
training_arguments = SFTConfig(
    output_dir="mistral-finetuned-alpaca",
    per_device_train_batch_size=8,
    gradient_accumulation_steps=1,
    optim="paged_adamw_32bit",
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    save_strategy="epoch",
    logging_steps=100,
    num_train_epochs=1,
    max_steps=250,
    fp16=True,
    packing=False,
    max_seq_length=512,
    dataset_text_field="text",
    push_to_hub=True
)

# Initialize trainer
trainer = SFTTrainer(
    model=model,
    train_dataset=data,
    peft_config=peft_config,
    args=training_arguments,
    tokenizer=tokenizer,  # This causes the error
)

trainer.train()

What I Have Tried:

• I checked the TRL documentation but couldn't find a tokenizer parameter in the SFTTrainer class.

• Removing tokenizer=tokenizer avoids the error, but then I get a different error about tokenization during training.

• I tried passing tokenizer inside training_arguments, but that didn't work either.

Question: Is SFTTrainer expecting the tokenizer to be handled differently in the latest versions of trl? How should I correctly pass the tokenizer to ensure training works?

I would appreciate any insights!

• python
• google-colaboratory
• huggingface-transformers
• mistral-7b
• peft

1 Answer 1

In the 0.12.0 release it is explained that the tokenzier argument is now called the processing_class parameter.

You should be able to run your code as before by replacing tokenizer with processing_class:

trainer = SFTTrainer(
    model=model,
    train_dataset=data,
    peft_config=peft_config,
    args=training_arguments,
    processing_class=tokenizer,  # This causes the error
)