On my m5 max 27b model does 75tps on 256k ctx and starts at 80 on the 8k ctx when you add https://huggingface.co/collections/z-lab/dflash-2 to it. So yeah base might be 30tps (I used iq4) but mtp or dflash help a lot and should be used when checking what is useful and what is not for running models as it is not fare to judge without them.
This looms nice and I starred but you just want actors and there are other options for that you might want to compare, I like rika actors, or temporal (not actors exactly but durable state and workflows). This is a space where a few very similar technologies after looked from different lenses sell themselves widely differently. So if you know what you want and the names of the things you are searching for you can get a thing that looks 90% like te rest, sorry a bit rambly. But yeah it would be nice to add some comparisons against these technologies as well.
You are most likely correct but that sounds less fun. Also they can do it much smarter way. Just ask these companies for content so they can detect it and prevent copyright infringement
reply