Peft Fine Tuning

Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.

This is a how-to guide for PEFT-style fine-tuning; it appears internally consistent. Before you run it: (1) be prepared for large network downloads (LLM weights can be many GB) and local disk use; (2) pip installing packages and building bitsandbytes from source will run code on your machine—only run these commands on trusted systems and inspect commands if you have concerns; (3) some models (e.g., meta-llama variants) may require explicit licensing or HuggingFace authentication—this skill does not request credentials but you may need them to access certain models; (4) follow GPU/CUDA compatibility notes in troubleshooting to avoid runtime failures. If you want to be extra cautious, run package installs in an isolated virtual environment or container and verify any third-party scripts (e.g., convert-hf-to-gguf.py or cloned build scripts) before executing them.

Type: OpenClaw Skill Name: peft Version: 0.1.0 The skill bundle provides comprehensive documentation and code examples for Parameter-Efficient Fine-Tuning (PEFT) of Large Language Models. All dependencies are standard ML libraries (peft, transformers, torch, bitsandbytes), and installation steps use legitimate `pip install` commands. The Python code snippets demonstrate model loading from HuggingFace, training, and saving/pushing fine-tuned adapters to the HuggingFace Hub, which are all standard and expected operations for this type of skill. There is no evidence of prompt injection against the agent, data exfiltration of sensitive user data, malicious execution, persistence mechanisms, or obfuscation. The `push_to_hub` functionality is presented as a legitimate way to share trained models, not as a means for unauthorized data exfiltration.