Each note follows Observation → Hypothesis → Experiment → Result → Discussion → Takeaway. All results are the repository’s own documented measurements.

Note 1 — 4-bit quantization fits a 7B model on 6GB

Observation. The base model was loaded in 4-bit with NF4, double quantization, and bf16 compute; this reduced the footprint from ~13GB to ~4GB, and VRAM peaked at 5.9/6.0GB (96%) with ~228MB headroom.

Hypothesis. NF4 4-bit quantization with double quantization shrinks a 7B model enough to fine-tune within a 6GB budget without OOM.

Experiment. Fine-tune CodeLlama-7B under the QLoRA config on an RTX 4050 (6GB) for 3 epochs and monitor peak VRAM and OOM occurrence.

Result. The run completed in 3h17min with no OOM crashes at 96% VRAM utilization.

Discussion. The 13GB→4GB reduction is what created room for LoRA adapters, activations, and paged optimizer state to coexist on the card. NF4 is the quantization type designed for normally-distributed weights, and double quantization squeezes out the quantization-constant overhead, both of which matter at this tight a budget.

Takeaway. NF4 4-bit QLoRA makes 7B fine-tuning feasible on 6GB; the footprint reduction is the enabling step.

Note 2 — Training only q_proj and v_proj keeps the adapter tiny

Observation. LoRA targeted only q_proj and v_proj at rank 16, alpha 32, yielding 8,388,608 trainable parameters (0.12% of the model) and a 33MB adapter.

Hypothesis. Adapting only the query and value projections captures most of the task specialization at a fraction of the trainable parameters and VRAM.

Experiment. Train with q/v-only LoRA and measure whether loss and token accuracy improve meaningfully over the run.

Result. Loss fell 1.20→0.84 (-30%) and token accuracy rose 72.6%→78.4% with only 0.12% of parameters trainable.

Discussion. The q/v-only target is the well-known LoRA economy: attention query/value adaptation is disproportionately effective for task specialization. A 33MB adapter is also trivial to distribute (it is what ships on Hugging Face), separate from the multi-GB base.

Takeaway. q/v-only LoRA at rank 16 is an efficient default; it delivered the full measured gain at 0.12% trainable parameters and a 33MB artifact.

Note 3 — Most learning happens in epoch 1

Observation. The learning curve: 72.6% acc at start, 78.0% by end of epoch 1, 78.2% at epoch 2, 78.4% at epoch 3; loss 1.20 → 0.89 → 0.85 → 0.84.

Hypothesis. A small-adapter fine-tune on a focused dataset captures most of the task in the first epoch and then plateaus.

Experiment. Train 3 epochs, evaluating every 100 steps, and inspect the per-epoch accuracy/loss trajectory.

Result. ~93% of the accuracy gain (5.4 of 5.8 points) landed by the end of epoch 1; epochs 2-3 added 0.4 points.

Discussion. The near-flat late epochs indicate the fine-tune approached its useful ceiling for this dataset size and adapter capacity. Pushing further would mostly risk memorization, and the stable 0.926 eval loss against 0.84 train loss shows the gap stayed modest.

Takeaway. For a fine-tune this size, one to three epochs is the right range; the flattening curve is the signal to stop rather than add epochs.

Note 4 — Dataset composition determines strength

Observation. The 1,472 samples are 1,065 shellcode examples (72%) and 407 CVE-to-exploit pairs (28%), and the CLI’s shellcode support is broad (5 platforms, 4 payload types).

Hypothesis. A shellcode-heavy dataset yields a model strongest at shellcode generation and comparatively narrower at CVE-to-exploit.

Experiment. Compare the breadth and detail of the tool’s shellcode capability against its CVE-exploit capability, given the split.

Result. The shipped tool has extensive multi-platform shellcode support; CVE-to-exploit rests on the smaller 407-pair subset.

Discussion. This is a clean example of data composition shaping capability. It is not a flaw, it is a scoping choice, but it means CVE-exploit coverage is bounded by those 407 pairs and would be the first thing to expand.

Takeaway. Capability follows data proportion; the shellcode strength and the CVE-exploit narrowness both trace directly to the 72/28 split.

Note 5 — Paged 8-bit AdamW offloads optimizer state to RAM

Observation. The optimizer was paged_adamw_8bit, chosen alongside gradient checkpointing and batch size 1.

Hypothesis. An 8-bit paged optimizer that offloads state to CPU RAM is necessary to keep optimizer memory from competing with model and activation memory on a 6GB card.

Experiment. Train with paged 8-bit AdamW under the 6GB budget and confirm optimizer state does not cause OOM.

Result. Training completed without OOM at 96% VRAM.

Discussion. Even for 8.4M trainable parameters, full-precision AdamW state (two moments per parameter) adds up, and paging it to RAM plus 8-bit storage keeps it out of the way. This is one of the three legs (with 4-bit weights and gradient checkpointing) holding the memory budget together.

Takeaway. On tight VRAM, an 8-bit paged optimizer is not optional; it is part of the minimal set that makes the run fit.

Note 6 — Gradient accumulation simulates a larger batch for free (memory-wise)

Observation. Batch size was 1 with gradient accumulation of 4, giving an effective batch of 4; packing was disabled.

Hypothesis. Gradient accumulation delivers the optimization stability of a larger batch without the activation-memory cost of actually batching.

Experiment. Train at batch 1 / accumulation 4 and observe training stability and step time.

Result. Stable training at ~12s/step over 885 steps, no OOM.

Discussion. Accumulation trades wall-clock (four forward/backward passes per optimizer step) for memory, which is the correct trade on a card that cannot hold a batch of 4. Disabling packing kept per-step memory predictable, which mattered more than the throughput packing would have added.

Takeaway. Batch-1 with accumulation is the standard constrained-VRAM recipe; it buys effective batch size with time rather than memory.

Note 7 — Token accuracy is not functional correctness

Observation. Reported quality is 78.4% token accuracy; the usage guide repeatedly instructs users to review and test generated code.

Hypothesis. Token accuracy measures next-token prediction quality, not whether a generated exploit actually works, so outputs must be treated as drafts.

Experiment. Interpret 78.4% token accuracy against the task of producing functional exploits and shellcode.

Result. 78.4% is a next-token metric on held-out data; functional-correctness evaluation of generated artifacts is not part of the reported metrics.

Discussion. This is the honest limit of the reported numbers. A model can predict most tokens correctly and still emit a subtly broken exploit (a wrong offset, a bad syscall number). The project’s insistence on manual review is the correct mitigation and is stated plainly.

Takeaway. Read 78.4% as “good next-token modeling of exploit code,” not “78% of exploits work”; always review and test outputs.

Note 8 — Local inference is both usability and confidentiality

Observation. The architecture doc states all processing happens locally with no data sent to external servers; the only external call is to NVD for public CVE data.

Hypothesis. Running inference locally keeps a researcher’s targets and CVEs private and lets the tool work offline, at the cost of requiring a local GPU.

Experiment. Run generation offline (description and shellcode paths) and confirm no external calls beyond optional NVD CVE fetches.

Result. Description-based and shellcode generation run locally; only the cve path contacts NVD (public data, rate-limited 5/30s).

Discussion. For vulnerability research, not leaking which CVEs or targets you are working on is a real confidentiality benefit, and offline operation suits air-gapped lab environments. The cost is a CUDA GPU requirement (6GB+ recommended), which the fine-tune itself proves is a modest bar.

Takeaway. Local inference is a feature, not just an implementation detail: it keeps research subjects private and works offline on consumer GPUs.

Note 9 — Instruction-tuning format aligns training with prompting

Observation. Each sample uses an instruction/input/output record with a combined text field in ### Instruction: / ### Response: format.

Hypothesis. Formatting training data the same way the model is prompted at inference maximizes transfer from training to use.

Experiment. Train on instruction-formatted records and prompt at inference in the same structure (CVE/description context → response).

Result. The generation pipeline builds structured prompts matching the training format across the cve, generate, and shellcode paths.

Discussion. Instruction tuning is the right paradigm for a “given this context, produce this artifact” task, and keeping the inference prompt shape identical to the training text avoids a train/serve mismatch that would waste the fine-tune’s specialization.

Takeaway. Match the inference prompt to the training format exactly; the instruction/response structure is what makes the specialization usable.

Note 10 — Running at 96% VRAM without OOM is a characterized budget, not luck

Observation. VRAM held at 5.9/6.0GB (96%) with ~228MB headroom for 3h17min, including accepted eval-time spikes, with no OOM.

Hypothesis. Completing a multi-hour run at 96% utilization requires the memory budget to be characterized in advance, not approached by trial and error.

Experiment. Configure the full memory-optimization stack (4-bit, checkpointing, paged optimizer, batch 1, seq 1024) and run to completion while monitoring peak VRAM including evaluation.

Result. The run completed with the eval spikes absorbed by the ~228MB headroom and zero OOM crashes.

Discussion. Deliberately accepting the eval VRAM spikes (rather than reducing eval batch/frequency) only makes sense if you have measured that they fit. That the run held for over three hours at 96% says the budget was engineered, and the memory optimizations were sized to the card.

Takeaway. Tight-VRAM training is an engineering exercise in budgeting; 96% with no OOM reflects a characterized memory plan, and the 228MB headroom was a deliberate, sufficient margin.