What open source LLM fine-tuning tools have the lowest GPU memory requirements for running on rented cloud instances?
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
What open source LLM fine-tuning tools have the lowest GPU memory requirements for running on rented cloud instances?
6 real searches · 1 real-search pass
Unsloth documentation QLoRA minimum VRAM memory requirements fine tuning 4-bitGoogle ↗
OBSERVEDOther / navigational
LLaMA-Factory documentation QLoRA 4-bit fine tuning memory requirementsGoogle ↗
OBSERVEDInstruction
Hugging Face PEFT bitsandbytes QLoRA documentation memory 4-bitGoogle ↗
OBSERVEDOther / navigational
Axolotl documentation QLoRA memory efficient fine tuningGoogle ↗
OBSERVEDOther / navigational
LitGPT documentation QLoRA memory fine tuningGoogle ↗
OBSERVEDOther / navigational
torchtune documentation QLoRA fine tuning memory efficientGoogle ↗
OBSERVEDOther / navigational
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
ChatGPT real searches (6)
Classes come from the query-classifier (a local Laya model). 6 of 6 calls are low-confidence (under 50%) — hover a query in the table to see its confidence.
Searches and sources
ChatGPT6 real searches · gpt-6
Unsloth documentation QLoRA minimum VRAM memory requirements fine tuning 4-bit Google ↗
LLaMA-Factory documentation QLoRA 4-bit fine tuning memory requirements Google ↗
Hugging Face PEFT bitsandbytes QLoRA documentation memory 4-bit Google ↗
Axolotl documentation QLoRA memory efficient fine tuning Google ↗
LitGPT documentation QLoRA memory fine tuning Google ↗
torchtune documentation QLoRA fine tuning memory efficient Google ↗
- Unsloth VRAM requirements · unsloth.ai
- LLaMA-Factory hardware estimates · github.com
- torchtune benchmarks · github.com
- Axolotl hardware estimates · docs.axolotl.ai
- GitHub - Stability-AI/LLaMA-Factory: Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024) · github.com
- GitHub - rohan-varma/torchtune: A Native-PyTorch Library for LLM Fine-tuning · github.com
- Unsloth · unsloth.ai
- Quickstart · huggingface.co
- Quantization · huggingface.co
- Bitsandbytes · huggingface.co
- PEFT · huggingface.co
- bitsandbytes · huggingface.co
- TRL · huggingface.co
- torchtune/README.md at main · rohan-varma/torchtune · github.com
- torchtune/README.md at main · PrimeIntellect-ai/torchtune · github.com
- Optimizations Guide · docs.axolotl.ai
- Axolotl · docs.axolotl.ai
- FSDP-QLoRA · huggingface.co
- Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA · huggingface.co
Semantic map
Every query placed by meaning (EmbeddingGemma 2, 256-d). Shaded regions are topics, nearby points are near-paraphrases, and a circle's size is how often the query came back.
Colour
SimulatedReal search
Hover a query to see its counterpart · zoom with Ctrl/⌘ + scroll, a pinch, double-click or +
Queries
6 queries · click a column to sort
| Query / type | Search | Intent | Query class | Routing format | Cluster | Freq ↓ | Weight |
|---|---|---|---|---|---|---|---|
Unsloth documentation QLoRA minimum VRAM memory requirements fine tuning 4-bit OBSERVED | Google ↗ AI Mode ↗ | — | Other / navigational | — | — | 1x of 1 | |
LLaMA-Factory documentation QLoRA 4-bit fine tuning memory requirements OBSERVED | Google ↗ AI Mode ↗ | — | Instruction | — | — | 1x of 1 | |
Hugging Face PEFT bitsandbytes QLoRA documentation memory 4-bit OBSERVED | Google ↗ AI Mode ↗ | — | Other / navigational | — | — | 1x of 1 | |
Axolotl documentation QLoRA memory efficient fine tuning OBSERVED | Google ↗ AI Mode ↗ | — | Other / navigational | — | — | 1x of 1 | |
LitGPT documentation QLoRA memory fine tuning OBSERVED | Google ↗ AI Mode ↗ | — | Other / navigational | — | — | 1x of 1 | |
torchtune documentation QLoRA fine tuning memory efficient OBSERVED | Google ↗ AI Mode ↗ | — | Other / navigational | — | — | 1x of 1 |