I don't believe at all in the idea that cheaper LLMs will somehow "democratize LLMs". Rich people will always be able to buy far more compute than poor people, and we know how to turn more compute into more capability (bigger models, more reasoning and more agents).

The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.

You have a fair point. I am financially secure so it's not a great sacrifice.

It was psychologically difficult to give up the opportunity to make more money at a fun job, though.

I didn't, and still don't, think Google was unambiguously good or evil. They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things. I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.

It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).

> They did dirty stuff with Chrome, they do addiction optimization, but they also provide great free services, fight endless wars against spammers, and provide reliable secure infrastructure to a lot of other people, and lots of other good things.

As mconnor liked to say, there is no free lunch. :)

> I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.

Ultimately, I don't think you can divide your niche from the corporation's primary goals (especially if you think it was your duty to be a good employee), so I disagree with your rationalization but kudos for at least having a moral compass and trying not to lose sight of it.

> It's pretty hard to live in the world and be completely disconnected from anything unethical.

Sure, but I don't think that should be an excuse for being fairly well connected.

> Are you aware tech bros have been saying this same thing about every AI advancement since the 1950s?

Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.

FWIW I was in the GDM GenAI unit. Not exactly working on AI directly, or I would have quit earlier, but in the middle of it.

so you don't think realized goods like AlphaFold don't out weigh unrealized fears of AI doom? so you quit?

It's not that simple; the goods of AlphaFold are mostly not yet realized, and some negative effects of AI are being realized.

More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.

Yeah, it's an age-old question. My high-level answer is, God's plan will be done regardless of what I do, but my actions determine (at least from my point of view) whether I'm aligned with the plan or not. And I want to be aligned with the plan.

Edit: removing my comment

(OP here) FWIW I despise the addictiveness, er engagement optimization that Google and others do. I have said so inside and outside the company many, many times. I would never have agreed to work on that stuff myself. One of the hobby projects I would like to work on is exploring how to use personal AI to beat those algorithms.

I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.

Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.

Thanks for sharing this.

I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.

I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.

Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.

> how to use personal AI to beat those algorithms

+1000

So it is ok to believe (even despise) it is bad but as long as I don't work directly on it but my super big paycheck gets funded by it, all good! Cheers!!

Google (unlike Meta say) makes a lot of money from a lot of things that have nothing to do with the addictiveness optimization.

I could have explained that better, but it was already getting too long.

Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.

"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.

Indeed, that message was for my coworkers. HR gets a form.

Unrelated to the thread of discussion, but thank you for posting it on your personal site, and not on Twitter/X.

I'd also be interested in how much your opinion on the existential threat of AI changes during your ten day backpacking trip, assuming it'll be a pretty offline experience.

Whilst I am involved in technology in my work and some of my hobbies, I do enough things 'away from the desk' that it feels to me that a lot of the doomerism is from people who have too much of their own personality and self-worth tangled up with the internet; the serially-online. I don't know where you naturally exist on that scale, but you already said you're outside the bubble and aren't the typical AI-rationalist-tech-bro (and surely the natural beauty of New Zealand is impossible to ignore).

I personally think that the old analogue world is big enough that some kind of AI-pocalypose wouldn't be anywhere near a civilisationally existential threat.

At risk of coming across an arrogant prick in telling you to do something: Consider your opinion now, and record it somehow. Then revisit it when you get back from your trip and see how far you've strayed from that as a baseline.

Thanks for the thoughtful comment. I go on this sort of trip pretty regularly so I don't expect this one to reorient me dramatically. They are good for my mental health though, for sure.

I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.

> and also host more instances of itself

Where? Frontier LLM-based AI requires very powerful computers to run. You cannot hide it in a toaster.

That would be ironic! But there's no chance I'll work on anything that accelerates AI. If I was OK with that, I would not have left Google.

I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...

> But there's no chance I'll work on anything that accelerates AI.

Fair, but I don’t mean you in particular.

> I do expect to use AI in my work

So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.

I think unintuitively all of us are capable of holding contradictions in our head (or word) without any conflict.