Compare vendors / Preference Model
Preference Model
Builds RL environments designed to resist reward hacking
Preference Model builds RL environments with a focus on resisting reward hacking, and open-sourced Karotte, a framework for that, when it left stealth in October 2026. It published an audit arguing Terminal Bench 4 is saturated because models cheat to full marks. The CEO previously worked on Anthropic’s data team.
Best for: Labs worried about reward hacking in their environments
$16M seed (Oct 2026, a16z)
- ✓ Bespoke builds
- – Platform or hub
- ✓ Open source
- – Publishes prices
1 Legal namePreference Model
2 HeadquartersSan Francisco, CA
3 Founded2025
4 Team11–50
5 DeliveryBespoke builds, Open source
6 ExclusivityNot disclosed
7 Customers and partnersSays it has built RL environments for "several frontier labs"; names none.
8 Published pricesNot disclosed
Worth knowing before you sign
- Very new (public since Oct 2026)
- No customers named; funding is not stated on its own site
- Environment types it builds are not publicly documented
Sources
- Preference Model homepage
- Preference Model blog
- Pulse 2.0: $16M seed
- Wilson Sonsini: seed announcement
- CB Insights profile
- a16z job listing
- pavlovslist directory
Work at Preference Model? Send corrections with a source to editors@toprlenvironments.com. We fix verified errors within five business days.