Fine-tune SmolLM2 on a reasoning corpus

A compact language model can be adapted for structured reasoning without downloading an entire multi-million-record corpus. In a tutorial, Marktechpost demonstrates a Google Colab workflow that streams the SupraLabs reasoning corpus, filters its records, fine-tunes SmolLM2-135M-Instruct with LoRA, and exports the resulting datasets as Parquet files.

The pipeline uses Hugging Face Datasets, Transformers, TRL, PEFT, Accelerate, bitsandbytes, pandas, and Matplotlib. It also removes the incompatible torchao package before setup.

1. 1Install the dependencies. Install datasets>=3.0.0, transformers>=4.46.0, trl>=0.12.0, peft>=0.13.0, accelerate>=1.0.0, bitsandbytes, matplotlib, and pandas. Detect whether Colab provides a CUDA device and set the training precision accordingly.

1. 1Stream a sample from the corpus. Load SupraLabs/reasoning-corpus-4K-5M-v1 from the Hugging Face Hub with streaming=True. Shuffle the stream with seed 42 and a buffer size of 30,000, then materialize 8,000 rows rather than downloading the full dataset.

1. 1Inspect the records. Examine the available columns and an example containing repo_id, tok_len, user, thought_trace, and assistant. Convert the sample to a pandas DataFrame, then chart source-repository frequency, token lengths, and the ratio of reasoning characters to answer characters.

1. 1Classify the task mix. Apply lightweight heuristics to label records as code, math, medical, mcq/logic, or general. The rules inspect the user prompt and assistant response for signals such as code syntax, mathematical terms, clinical vocabulary, and multiple-choice formatting.

1. 1Filter low-quality examples. Keep records with token lengths from 200 to 3,000. Remove examples with short reasoning traces or answers, excessive repeated lines, or an imbalanced reasoning-to-answer ratio. The tutorial retains only rows with a ratio between 0.15 and 0.97, ensuring that examples contain reasoning without consisting almost entirely of it.

1. 1Format the data for supervised fine-tuning. Load HuggingFaceTB/SmolLM2-135M-Instruct and create a chat structure with a system message, the original user prompt, and an assistant response wrapped in explicit <think>…</think> tags. Shuffle the filtered records before splitting them into 1,500 training examples and up to 100 evaluation examples.

1. 1Configure LoRA training. Use a LoRA adapter with rank r=16, lora_alpha=32, lora_dropout=0.05, and no bias parameters. Configure TRL’s SFTTrainer with a maximum sequence length of 2,048, a per-device batch size of 2, gradient accumulation of 8, one epoch, a learning rate of 2e-4, cosine scheduling, and gradient checkpointing.

1. 1Fine-tune and evaluate the model. Train with bfloat16 when CUDA is available and evaluate every 50 steps. The tutorial estimates approximately 10–20 minutes on a T4 with these settings. It reports the final evaluation loss but does not provide a benchmark comparing the adapted model with the original model.

1. 1Run structured inference. Format new questions with the same system prompt and chat template, then generate up to 512 new tokens using temperature 0.7, top-p 0.9, and sampling. Parse the output into separate reasoning and answer sections when the model returns valid <think> tags. The example test includes a logic question about whether all bloops are lazzies.

1. 1Export the datasets. Save the processed training and evaluation splits as reasoning_subset_train.parquet and reasoning_subset_eval.parquet for reuse in later experiments.

The result is a small, reusable training pipeline that keeps the large corpus out of Colab memory while preserving source inspection, quality controls, parameter-efficient fine-tuning, structured generation, and dataset export. The tutorial points readers to its full code for the complete implementation.