deciduous seems like a great way to ensure there is no space left in context for useful work.

Problem with all these automatic memory managment tools is that they can't reliably classify if something is useful or not. Once it start using some tool it will heavily become biased towards using that tool during that session. First you will have decision to use postgres as a database, by the end of session it will start saving "decided to use SELECT query for this method because that is how you pull data out of database".

Not how it goes in practice at all, would encourage you to actually try it out.

If you could post an issue if you still have the error around that would be awesome.

Could you explain what happens when you try to shoehorn a 2.4T parameter model into a 24gb m4 mac?

extreme divergence would be my guess

LLMFit tells you what can run on something. I built something quite similar to their search into Shoehorn now.

A really great place to reach for this is Elixir/Phoenix now that the set theoretic type system is in place with LiveView. I have been doing some pretty large/complex codebases gradually working towards fully integrating working this way for the past 5 years and it has been great.

Yeah… I’ve been a professional Elixir developer for 5+ years and additional 4 as a hobby. The quality of the ecosystem and the practices are nowhere near those in OCaml. Sad to say it. People just want to write imperative procedural code and monkey patch in tests so they can test anything at all.

I have been having pretty good success with Qwen 3.5 9B for "nontrivial but not challenging work all things considered" -- it runs great on my 24gb unified memory m4 pro MacBook Pro. What do the baseline specs look like Mac-wise for getting this model to run? Am I looking at a 96gb? 128? 256?

I posted this elsewhere, but Unsloth says the 27B model should run in 18GB. That leaves little RAM for other tasks, but it depends on your tolerance for slowness I suppose. I haven’t tried it in 24GB so report back if you do.

https://unsloth.ai/docs/models/qwen3.6

You might be interested in Ornith 1.0 9B, which is a new intriguing post-training of Qwen 3.5 9B.

Qwen 3.6 27B will run in full offload with a 4-bit quantisation in 64GB on an M1 Max. It is quite slow.

I don't know about 48GB but 64GB should be enough.

I've been trying Ornith 1.0 35B, I'm pretty impressed with it: https://simonwillison.net/2026/Jun/29/ornith/

It's the one I have loaded right now.

It got rather tangled up when I tried it with one of my coding tests, which is a simple wordpress plugin, but I frustrate the model by asking it to write code for older PHP, break WP coding conventions and use a rather bespoke method for arranging code in objects. So it is sort of a hybrid of a green field and brown field task; a bit muddy.

It did not do as well as Qwen 3.6 35B, but the way it worked through its thoughts was interesting.

TBH I struggled to understand what DeepReinforce are doing that is materially different; the explanation of their training technique goes over my head at this point.

It is also available with Ollama now and I am equally impressed too.

Thanks! I was thinking of doing the 128gb to have some future proofing. I figure at this point, it's akin to a mechanic keeping great tools around, when it comes to having this sort of homelab and exposing it for your own uses. And great practice for building the next era of user facing computing that will be around as this proliferates.

I would not buy a 64GB model again, probably, if this were to remain particularly important to me. But I gather memory bandwidth is pretty important here.

So for example I'd favour a used M1 Max over a used M2 Pro, at least based on my naïve understanding. Not quite sure where the balance changes.

There appear to be some hardware improvements with the M3 and up regarding the Apple Neural Engine which I'd hope would show up in MLX performance; I remember seeing some optimisations in image generation models that are only possible on later hardware.

The GPU cores are progressively better I believe, but the memory bandwidth is lower. Though perhaps the M4 can get closer to actually saturating said bandwidth.

(And I must reiterate that my understanding of this stuff is pretty naïve.)

Used M1 max is still a good choice because its memory bandwidth only got surpassed by generation m4 and later (except with ultra variants which are more expensive). Its prefill speed is not great though, and that is an issue for running larger contexts, which only substantially improved with m5. Moreover, up to m3 they only have thunderbolt 4, not 5, which means that they lack RDMA support which would make stacking machines more effective. So unless you go higher price for m4+ max, or any m ultra, m1 max is pretty decent still compared to m2 and m3 max, definitely better than pro variants, if you can find in a decent price and want to experiment without caring much about time to first token and large contexts.

A very useful resource for characteristics and comparative performance of all M variants, if anybody is interested, is https://github.com/ggml-org/llama.cpp/discussions/4167?sort=...

Its sister discussion for nvidia gpus is https://github.com/ggml-org/llama.cpp/discussions/15013

Note the drop in performance for the base (binned) m3 max version. You are better off with full m1 max than the binned m3 max, even price aside.

The issue I have with my m1 max is that with 64gb you cannot run really decent MoE models, ie the ones you can run like qwen 35B-A3B have only 3b active parameters and are much less capable than qwen 27b in my testing. So I end up running the 27b one, but it runs relatively slow (though still usable at 10-20 tok/s) and I would have been better off a used nvidia gpu setup for dense models. I assume 35B-A3B has its use cases, eg as subagents, just that I cannot find them. With a higher amount of ram I could probably run bigger MoE models which could be more comparable, though prefill would still be an issue (and prob a bigger one). The only hopeful thing is that there are performance hacks appearing (speculative decoding and prefill) that seem to start improving inference speed once getting implemented, so I am mildly hopeful.

(I must also iterate that my understanding is not very deep either)

Good reply, those two links are v. useful and I had missed them.

I use deciduous every day while working with LLMs to create more of my own tools/projects. It is a living memory for agents stored in a simple, portable manner. https://deciduous.dev

It might be helpful to rotate the books on the frontpage so that that you can read them by binding without tilting your head.

I regularly have agents communicate with each other this way using my tools Deciduous (https://deciduous.dev). It keeps all decisions in a DAG and the other agents, when configured, read from it constantly and use it to inform their new decisions. Extra entries in the same space from another agent come to light and they can begin to work together like this.

Valid, but you lose the lived history that comes with the audit log of it being actual review back and forth and CI runs vs lost to a developers machine and only a relic in the commit log. I can see both sides, though.

Can you elaborate about the practical value of having the history of back and forth, in a PR or even in the commit log? In my 20ish years of experience, I can’t recall a single instance where I’ve solved something thanks to having this work-in-progress state persisted in the repo history.

It’s exclusively been the other way around where having a smaller number of larger squished commits (post merge) that’s made the project be more maintainable.

It's not about having it in the commit history. I've seen a few cases where the back and forth revealed that the AI reviewer was offering bad advice (and a few others where I suspect bad local AI advice is why people keep sending me the same category of mistake).

Actually not, it is similar debate like rebase or merge.

e.g. I don't squash, I prefer to see full history, not redacted one.