AI Tip from @karpathy

Description

Automatically generated AI learning skill from curated web and social media sources.

Steps

1. Tip: When training neural nets, always start with a tiny dataset (e.g. 10 examples) and overfit it completely before scaling up. This validates your entire pipeline works end-to-end.

2. Test your training loop

3. def test_overfit():
4. model = YourModel()
5. tiny_dataset = get_tiny_dataset(10)
6. train(model, tiny_dataset, epochs=1000)
7. assert loss \x3C 0.01

Code Examples

# Test your training loop
def test_overfit():
    model = YourModel()
    tiny_dataset = get_tiny_dataset(10)
    train(model, tiny_dataset, epochs=1000)
    assert loss \x3C 0.01