
More Data Won’t Save Your LLM. Better Data Will.
There was a point, not long ago, when the dominant strategy for improving large language models was simple: feed them more. More tokens, more compute, more parameters. The scaling laws
Booth 21-25 | AI Data Management Zone | Tokyo Big Sight

There was a point, not long ago, when the dominant strategy for improving large language models was simple: feed them more. More tokens, more compute, more parameters. The scaling laws

Before you reach for RLHF, before you design a reward model, before you start thinking about reinforcement learning from verifiable rewards — there’s a more fundamental question worth asking: has this model been properly fine-tuned on examples of the behavior you want? Supervised Fine-Tuning (SFT) sits between pre-training and the
