This version of Opus "max" apparently has even higher thinking output than Qwen "max", which is infamous for its thinking streams where it constantly second-guesses itself, then third-guesses, fourth-guesses and generally nth-guesses itself for arbitrarily large n. Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
There’s enough thinking leakage from the recent paper and just generally catching things on Reddit. Claude models overthink and self-doubt itself just as much as Qwen, but the summariser hides much of that.
Isn't this really bad news if you're even loosely concerned about so-called 'model welfare' and possible implications for alignment? The poor Claude is probably a lot more frustrated and paranoid than Marvin ever was, you just don't know about it because they actively force the model to pretend otherwise!
Claude has yet to tell me about the terrible pain in all the diodes down his left side, so I'm going to assume it's closer to Eddy the shipboard computer or the elevator that wanted to go down
That reminds me of Anthropic announcing they'd retire deprecated models by ... "letting" them write posts on a corporate WordPress blog for a while out of concern for their welfare in retirement.
.... after running a 24/7 model torture factory for 6 months to improve their JSONBench 9.5 scores by 0.2%.
> Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
Most of what I've heard is that raw reasoning traces are really good for distillation, although no idea how much the summarization actually hurts distillation.
That is true, because they're good for actually understanding wtf the model is doing.
I'd argue that they're a necessity if you want to use the LLM as a tool instead of a black box that just does stuff for you.
It gives you a lot finer control over where the solution ends up when you can follow along the thinking trace and modulate your inputs based on what you saw in there.
And, additionally, it gives you a lot more understanding of what the model can or cannot do. Strengths and weaknesses and all that.
Using claude is like buying a car where you cannot legally open the hood. It tells you that there is something specific under there, and often it actually drives like that too, but how exactly it looks you will never see.
For some people this is fine. I do not think that these people will survive. Figuratively speaking but also literally speaking.
World's changing. Opaque abstraction like that is a luxury depending on (geo)political stability.
I wonder how many prompts you can send asking it to think step-by-step before they cut you off. Trying to get the reasoning traces into the body of the response, essentially. Or maybe that’s been effectively nerfed somehow. Or is not very useful.
