PoCSmith fine-tunes codellama/CodeLlama-7b-hf with QLoRA, and the configuration is a study in fitting a 7B model onto a 6GB GPU. The base is loaded in 4-bit through bitsandbytes with NF4 quantization, double quantization, and bfloat16 compute dtype, which cuts the footprint from ~13GB to ~4GB. On top of the frozen quantized base, LoRA adapters are trained with rank 16, alpha 32, targeting the q_proj and v_proj attention projections, 0.05 dropout, no bias, causal-LM task. The result is 8,388,608 trainable parameters, 0.12% of the model, saved as a 33MB adapter_model.safetensors.

The training hyperparameters complete the memory picture: batch size 1 with gradient accumulation 4 (effective batch 4), learning rate 2e-4, 3 epochs, paged_adamw_8bit optimizer, bfloat16 precision, max sequence length 1024, and gradient checkpointing enabled. Each of these is a memory choice: batch 1 and accumulation trade wall-clock for VRAM, the paged 8-bit optimizer offloads state to RAM, checkpointing recomputes activations instead of storing them, and the 1024 cap bounds activation memory. Together they hold the run at 5.9/6.0GB. This is the reference recipe for consumer-hardware 7B fine-tuning, and PoCSmith documents it end to end.