nanogpt-training

Train GPT-2 scale models (~124M parameters) efficiently on a single GPU. Covers GPT-124M architecture, tokenized dataset loading (e.g., HuggingFace Hub shard...

This skill is a coherent, textual training guide that appears safe to inspect and use. Before running: (1) review dataset licenses (downloading large token shards can have legal/ethical implications); (2) run initial experiments on a tiny subset to validate code and resource usage; (3) be aware of resource/cost implications when using GPU clouds (Modal examples request A100); (4) only provide HF/GitHub credentials if you intentionally access private repos; and (5) if you plan to execute code from untrusted sources, do so in isolated environments (containers) and inspect code snippets carefully for any modifications before running.

Type: OpenClaw Skill Name: mhc-layer-impl-nanogpt-training Version: 0.1.0 The skill bundle provides a comprehensive and legitimate implementation for training GPT-2 scale models using PyTorch and Modal. The code follows standard practices from well-known community projects (e.g., Karpathy's nanoGPT), including implementations for the Muon optimizer, Rotary Positional Embeddings, and memory-mapped data loading from HuggingFace Hub. No evidence of data exfiltration, malicious execution, or prompt injection was found; all network activity is limited to downloading public datasets from HuggingFace.