Hi, I'm one of the founders of Evidence. We're in the process of overhauling our docs and I'd love to hear any issues you ran into so we can make them better. Any feedback appreciated!
1. It will take much longer to understand the output of the machine that it takes to prompt and create it.
2. The only? best? one? way to /verify/ that you /in fact/ understand the output of the machine is to explain it to someone else.
So there will be a machine generating koans which need to be meditated upon and discussed with human social back-pressure validating understanding. I think this could be much more cooperative and at a minimum this will be a way different math social construct.
I think AI CEOs are dumb but AI 'enterprise decision-makers' could be very very interesting.
Imagine how quickly your meetings could get resolved if you had teams submit prompts / contexts, had a clear, documented set of overarching objectives, etc.?
I am here to tell you HELL YES and TAKE MY MONEY. I am such a fan of the idea of using AI to help give personalized and structured AI lessons in the hands of students and let them cook!
- It looks like you are gating family access to K-3 for now and I think that's right. I wouldn't really be comfortable giving my first-grader a live chatbot. Maybe I would think about whether there are other non-persona modalities that could still be self-directed (i.e. I am uncomfortable with a chatbot interface on this for a six year old but gamified flash cards with options could be different).
- I think the other issue is with motivation. I have various duct-tape versions of these types of agents and the thing about it is if you're doing the learning right it can be HARD. So I would think about using motivational interviewing or other techniques to help keep the user coming back and motivated.
- I would really think about the assessments here too. Many people are worried about LLMs ruining student evaluations, but if you could bake in reliable, flexible exams that gauge user progress (even for something like a "Did you read this" quiz) I would bet teachers would like it. There is likely so much you could do on student progress observability and e.g. structuring team-based projects or having targeted student working groups to hash out hard concepts in a targeted way, etc.
This is such an interesting market and use case too because the educational system might be very structurally set up to the current pedagogical staffing model (think about the incentives for teacher's unions and administrators). If you think it will be hard to change that system as quickly as you want I would also try to have an offering direct to families / home-schoolers. I think there is also a cottage industry of tutors that might benefit. Maybe partnering with the textbook publishers? I'm sure there are some "Teach your kids better" influencers that would get you into some feeds?
On motivation: we have something called Bloomy Bucks. Kids earn these for getting questions right, mastering skills, and being consistent. They can then redeem them for privileges that the teacher/school/parent sets (e.g., homework pass, game time, whatever). This is part of the effortful dopamine that I, personally, think is the right approach to motivation. A lot of people will disagree with me on this, but I think that some amount of extrinsic motivation (something we've already been doing for quite a long time with letter grades) can cultivate intrinsic motivation. One specific application of this is that students can earn up to 1 Bloomy Buck for a meaningful interaction with BloomyBot on a given question.
On assessments: imagine if we could get rid of assessments and just have live diagnosis happen all the time? With enough data from practice and mastery, that should be possible.
We have an offering for homeschoolers, indeed. Bloomy is reimbursable in ~15 states right now through ESA-type scholarships. Great ideas!
- I think the other mistake I see here is trying to over-engineer a deterministic learner path instead of giving the AI more free reign on best next interaction and a set of goals it needs to accomplish through the session; it can feel more responsive and free-form that way from the learner's POV.
- If you had voice here you could also make the screen optional. In my experience typing out long answers to questions can take a while too - so a voice mode might be helpful for learners. It would also be cool if people could take a 'photo of their work' for e.g. math equations done by hand.
- To the earlier point on family end-market, an interesting idea is modeling bloomy - have some grown-up oriented courses so you can learn with / side-by-side with your child? Just an idea.
IME LLMs are kind of like a projection of your current expertise - your prompting and guidance etc. biases LLM plans kind of 'in the direction' of your thinking. I think this is one reason why it seems like senior engineers get more lift vs. juniors.
What I am exploring is another step to the classic 'research / plan / implement' pattern: 'research / plan / LEARN / implement' where LEARN involves the human doing AI tutoring sessions to ensure a deep understanding the concepts etc. that the LLM is planning to implement so you can refine / iterate on plans and direct the LLM in ever more effective ways. My idea is that this then compounds your human capital and reduces the occurance of 'sounds smart, doesn't work' pattern.
Error: Error during compaction: API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup).
Commoditize your complement - I expect to see this most in consumer AI (after that starts actually working...)
It will be important for Apple to have good enough, cheap local LLM models that run on-device.
If the barrier to performance shifts from fundamental model capability to context collection and management I would expect to see folks focused on that problem continuing to drive open-weight LLM model development in some shape or form.
reply