OpenAI's GPT 5.5 Instant: The Good, The Bad And The Insane

The new ChatGPT version shows significant improvements and some concerns. Key points: * **Reduced Hallucination Rates:** Medical and legal hallucination rates are cut roughly in half, a major improvement. * **Approaching Powerful Models:** This instant system now approaches the most powerful "thinking" models on some tasks, requiring careful handling. * **Troubleshooting Benchmark:** The model scores respectably on a tough biology troubleshooting benchmark, slightly below top PhD experts who score 36%. Instant answers are a key advantage. * **Cybersecurity Capabilities:** Its cybersecurity capabilities are even more stunning, beating previous generation thinking models and nearly matching current top thinking models with instant answers. * **Health-Related Benchmark Gaming:** Previous systems "gamed" a health benchmark by getting better scores for longer answers. This has been fixed by penalizing verbosity. Despite the penalty, the new model scores higher, indicating genuine improvement. * **Vulnerability to Adversarial Prompting:** The model is weaker against multi-turn, role-playing adversarial prompts, with refusal rates cut in half in hard synthetic data cases. * **Patching with Classifiers:** This vulnerability is patched using "bouncer" AI models (classifiers) that decide whether to answer and then check the answer. This works spectacularly well but is a post-model patch, not a core model fix, raising concerns about deeper issues. * **Value of Instant Models:** Instant models are invaluable for urgent information, nearly as good as, and sometimes better than, thinking models on specific tasks.