FAR.AI co-founder and CEO Adam Gleave joins Nathan to discuss FAR.AI’s AI Security Leaderboard, the first systematic head-to-head evaluation of the misuse safeguards frontier developers actually ship. The findings expose a major measurement gap: while Claude Fable 5 and GPT-5.6 Sol withstood FAR.AI’s suite, Grok 4.5 and Gemini 3.1 Pro yielded hundreds of universal jailbreaks at low cost. Adam explains why many effective attacks look more like social engineering than advanced ML, why “jailbreak tax” should not be relied on for safety, and how FAR.AI scores whether a model is genuinely helping an attacker. The episode’s stakes are whether AI developers can measure and harden real deployed defenses before threat actors make routine use of increasingly capable systems.
- FAR.AI AI Security Leaderboard: http://leaderboard.far.ai/
- People can e-mail owsa@far.ai if they're interested in the open-weight safety accelerator grantmaking program.
For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/is-offense-or-defense-dominant-far-ai-s-adam-gleave-on-the-ai-security-leaderboard/
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(03:22) AI security leaderboard
(07:56) Universal jailbreaks explained
(16:26) Finding social jailbreaks (Part 1)
(16:31) Sponsor: Claude
(18:01) Finding social jailbreaks (Part 2)
(30:48) Layered safeguard defenses
(42:25) Uneven frontier safeguards
(51:10) Sharing safety standards
(01:00:30) Offense versus defense
(01:08:50) Open-weight model safety
(01:17:25) Control failure warnings
(01:30:05) Coordination and risk
(01:39:24) Episode Outro
(01:42:52) Outro
PRODUCED BY:
https://aipodcast.ing
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
