Skip to content

Add Qwen3 (1.7B) classification notebook for many labels - #425

Open
arihanv wants to merge 1 commit into
unslothai:mainfrom
arihanv:feat/many-class-classification
Open

arihanv wants to merge 1 commit into
unslothai:mainfrom
arihanv:feat/many-class-classification

Conversation

@arihanv

@arihanv arihanv commented Nov 10, 2026

Copy link
Copy Markdown

Closes #192

#192 asks for a classification notebook that works with many classes (the reporter has about 400). The text classification notebook linked in the README trims the LM head down to single tokens like "0" to "9", which breaks once labels need more than one token.

This adds Qwen3 (1.7B) Classification, which uses a sequence-classification head instead: FastLanguageModel.from_pretrained(..., num_labels, id2label, label2id), one output per class, trained with cross-entropy. Label names never go through the tokenizer, so the same code works for 77 or 400+ classes.

What the notebook covers

  • Banking77: 77 customer-support intents, ~10k train / 3,080 test.
  • A short "using your own data" section with class_encode_column, for string labels like the reporter's.
  • LoRA via get_peft_model. Unsloth detects the classification model and trains the new score head too.
  • Accuracy and macro F1 during and after training, and top-k predictions with probabilities.

Results (free Colab T4, full Run all)

  • Accuracy 0.936, macro F1 0.936 on the 3,080 test questions after 2 epochs
  • Peak reserved memory about 2.3 GB (15.5% of the T4)

Files

  • original_template/Qwen3_(1.7B)-Classification.ipynb, plus the Colab, Kaggle and AMD notebooks and the python_scripts/ and fsdp2_scripts/ copies generated by update_all_notebooks.py
  • README: one row each in the Colab, Kaggle and AMD Qwen tables

When I ran update_all_notebooks.py locally, the Hugging Face metadata lookup failed for every model, which reordered most README tables and touched unrelated notebooks. I kept only the files for this notebook and its README rows.

Fine-tunes Qwen3 (1.7B) with a sequence-classification head on Banking77
(77 intents) instead of trimming the LM head to single-token labels, so the
same recipe scales to hundreds of classes and multi-token label names.
Shows class_encode_column for string labels, tracks accuracy and macro F1,
and prints top-k predictions.

Closes unslothai#192
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request for Text classification with Unsloth notebook for 10+ classes

1 participant