Talking Postgres with Claire Giordano

CLAIRE: 00:00:05
Welcome to Talking Postgres. It's a monthly podcast for developers who love this database. I'm your host, Claire Giordano, and in today's podcast and all the episodes, we explore the human side of Postgres databases and open source, which means why do people who work with Postgres do what they do and how did they get there? I want to say thank you to the team at Microsoft for sponsoring today's community conversation. And today's guest is Simon Willison. This is Simon's third time back on the podcast, which makes me feel really, really good that he's willing to come back. He's experienced it before and he's willing to do it again.

SIMON: 00:00:44
It's always a great time.

CLAIRE: 00:00:46
Thank you. Simon is an independent open source developer, and his first claim to fame, which most people have heard of, is that he was co-creator of the Django web framework. Since 2002, which is a while ago now, he's been a prolific blogger, sharing what he's learned on a daily basis, as far as I can tell, multiple posts a day in many cases. And you can find his blog at SimonWillison.net. Officially, Simon works full-time building open source tools for data journalism, creating software that is going to help somebody win a Pulitzer Prize someday, including tools like Datasette, which he created. And he also spends a lot of time these last couple of years. Exploring the cutting edge of the latest LLMs and AI tools, sharing his observations and what he's learned, which of course the rest of us are really happy about. So, welcome, Simon.

SIMON: 00:01:44
Hey, I'm really glad to be back.

CLAIRE: 00:01:47
So today's topic, officially, I pushed out a social media post earlier today, was titled How AI is changing software development. But I considered a couple other titles: How AI is changing how I build software, where I is you, Simon, or How AI is affecting open source projects, because I do want to talk a little bit about open source projects later in the show. So what I really want to jump into are specific stories and examples of how you are using AI to build software now, how it's changed what your day-to-day life looks like. And then also, I'm hoping you share some stories about things you've heard from friends and other people as well, because I want people listening to be able to visualize, to be able to imagine what's possible, because maybe, maybe they're not there yet in terms of their day-to-day use.

SIMON: 00:02:47
You know what? I have the perfect story to kick us off. [Oh, good.] So one of, I've got.

CLAIRE: 00:02:52
Does it involve a seal named No?

SIMON: 00:02:55
Named Chonkers? It doesn't, but Chonkers is always on my mind. There was a wonderful giant Steller sea lion in San Francisco a couple of months ago called Chonkers, who is three times the size of the other sea lions and made, made quite an impression on people. Okay, so this morning, I was— one of my open source projects is a little tool called sqlite-utils. I do most of my work with SQLite. sqlite-utils is a command line tool for dumping data into a SQLite database, and a Python library as well. So a lot of my projects end up as both Python libraries and command line tools. And the neatest feature of sqlite-utils is that you can take a bunch of JSON and say, stick this in my database, and it will create the correct schema for you. It'll go, Oh, okay, this JSON has a title, which is a string, and it has an age, which is an integer in it, creates the table and inserts the data, and that all just works. And it's a one-liner in your terminal to do that. And this morning, I was thinking, you know what, I've always wanted— To have a version of that that works with Postgres and with other databases as well, like DuckDB and such like.

SIMON: 00:04:00
So literally, while I was in the shower this morning, on my phone, I fired up a coding agent session on my laptop, because you can remote control. I'm using Codex, and you can remote control that from your phone. And I fired it up and I said, go and look at sqlite-utils and build me a new version of this library which works against PostgreSQL and DuckDB as well, and do it with test-driven development and build everything. And that was it, and it's done it. And so now I'm looking on my computer right now at a new version of my software that works against PostgreSQL and DuckDB in addition to SQLite. And this is It's one of these weird, it's a research spike, right? It's a little proof of concept to see if that would work.

SIMON: 00:04:40
Except that the software's got over 100 tests now, and it runs the full test suite against all three of those database engines. And my involvement was pretty much typing on my phone in the shower to try and kick the thing off, and then a couple of follow-up prompts to get it to add new features. This is wildly—this is, when we talk about what's changed in software engineering, this is sort of the ultimate extreme end of all of this. This is where you can have an idea for quite a sophisticated piece of software, and this is something which I'd been idly thinking about for a couple of years now. I like using this tool. Wouldn't it be great if I could use this tool with other database engines? The latest frontier models are now capable of taking as loose as, take the ideas from this project and rebuild them against these other things, and churning out working software. And that, I feel like this is new as of.

SIMON: 00:05:34
Really, November was when we got the first models that, like Opus 4.5, that were capable of taking these larger projects and actually delivering them without making too many stupid mistakes along the way. Is that what, it's what, that was what, 8 months ago, 9 months ago?

CLAIRE: 00:05:48
November, November 2025. Okay, and that's what you call the inflection point, I think.

SIMON: 00:05:53
I think that was the inflection point, because prior to that, Claude Code itself was born in February of last year. So that was really the first coding agent of the kind that we think of today. Really isn't that old as a piece of software. It came out in February, but it was—and for the first sort of six months of that year, it was fun to poke around with, but you wouldn't trust it to build you useful software. As of November, the models caught up to the point where now you can trust it to write software. Now, what is it? It's August now, and yeah, I'm increasingly outsourcing extremely ambitious projects to these tools and getting back software that I trust myself, but I won't release to other people until I've done a little bit of extra work because I don't want to stake my reputation on something that I'm not 100% confident in. But it's astonishing. What an incredible rate of advancement we've had in the past sort of nine months.

CLAIRE: 00:06:47
Okay, so I want to get really nitty gritty and dig into the request that you made when you were in the shower. You didn't use voice apparently because you said you were typing it on your phone. Is that right?

SIMON: 00:07:00
I've not hooked up voice via my phone to control my laptop yet. I believe it is possible. I use voice a lot, but for this particular one, I haven't quite got that working yet.

CLAIRE: 00:07:09
So how long of a request was it that you typed in? How long were you sitting there typing and pecking into your phone?

SIMON: 00:07:15
I'm looking at it now. It is two paragraphs of text. I can read the whole thing. It says, do a research spike to see what it would take to build a library with the same core API as sqlite-utils, in particular the insert and upsert and create and update methods and the table introspection stuff, but backed by SQLAlchemy so it works for multiple database engines. Test against Postgres and SQLite and DuckDB. Use `~/dev/sqlite-utils` for reference. Create a Git repo for this. Commit early and often. Use `uv init` to set up the project. Use red-green TDD and pytest. So a bunch of jargon in there about how I wanted this to work, but that was it.

SIMON: 00:07:50
This is, and it's one of those things as well where a lot of people are concerned about the impact this has on software careers, because now I can knock out a couple of paragraphs on my phone in the shower and it does a substantial piece of work. But you have to have so much expertise yourself in the sort of domain in order to drive these things. I was pointing it at SQLAlchemy and telling it how to use red-green TDD and all of these different bits and pieces. And because I've spent so much time tinkering with these models, I was pretty confident that it was going to work. I can now sort of imagine a prompt that will more likely than not get me to the desired state. And that was it. And off it went, and it worked for 43 minutes. And delivered something that mostly worked, and then I told it to refactor the code so the engine-specific portability stuff lives in a file for each specific engine, because the code was full of if-then statements that I didn't like.

SIMON: 00:08:43
And it said, okay, I did that, and off it went. And yeah, this is... Honestly, this is, this is like a week of work for me in the before times, and it got it done in an hour this morning while I was having a shower and then making breakfast.

CLAIRE: 00:08:57
So wait, you said the software has over a hundred tests now. How does it have a hundred tests? Where did that spec come from? Are you satisfied with those tests, or are they superficial and incomplete, or?

SIMON: 00:09:09
Well, so I've hardly even looked at them. The reason it has tests is I told it to use red-green TDD and pytest. And red-green TDD, that's the thing where you have to write a test and watch it fail, and then you write the implementation and get the test to pass. I've found that this has This has been giving me really good results with agents, because the one thing you don't want is you don't want an agent to write a bunch of code that hasn't really been exercised at all. And then you're basically just rolling the dice as to if the thing works or not. I want every line of code they write to have been exercised, and the easiest way to get them to exercise them is to tell them to write tests, because they're very good at writing tests generally. The one test file I did browse, I poked into a couple of them, and it was doing exactly the right thing in that it was It was exercising the sort of user-facing library feature.

SIMON: 00:09:59
And then it was using pytest fixtures to run the exact same test against the three different backends. And that's exactly how I want this to work. I want every test to run against Postgres and then SQLite and then DuckDB, and only pass if all three engines pass. And that worked. And partly as well, this is because I'm using pytest, the Python testing framework, which is very well established, extremely mature software. All of the agents have seen it running in so many different configurations. And pytest has baked-in features called fixtures for running the same test against three different backends. All of that kind of stuff is at well-understood patterns. So once you know that, you can Point to the agent, you can basically say to the agent, use pytest, and you know that the agent is going to pick up those techniques from that because you've got that deep understanding of that software and you've seen the agent use it well in the past. So all of this stuff comes down to enormous quantities of sort of built-up experience across both the technologies that you're using and across the agents. You have to know what kinds of things they're capable of. If I'd tried this with an agent six months ago, I very much doubt I'd have got good results out of it. But this one, I'm using GPT-5.6 Sol Ultra, which is the mode of GPT-5.6 where it fires up multiple sub-agents and does research and has something else looking at the documentation, all of those kinds of things.

SIMON: 00:11:26
And it's cost me, well, it's been, it hasn't cost me anything at all because I've got a subscription, but apparently I've burnt through $66 worth of tokens already. Just today. $61, because you, and this is off my $100 a month subscription to OpenAI. Because the subscriptions give you just a massive discount on the actual price of the tokens. If I was paying API prices, if I was an enterprise, I would have spent $61 on this project so far.

CLAIRE: 00:11:57
Okay, but instead it's included as part of your subscription, no matter how little you use or how much you use, or is there some threshold after which you're going to be capped?

SIMON: 00:12:02
Exactly.

SIMON: 00:12:09
Well, there's a threshold. I'm now 71% of the way through my— no, I'm 29% of the way through my five-day threshold. But OpenAI resets the usage limits all the time as a marketing exercise. So there's almost this thing. I actually see people; somebody from OpenAI announced that they were going to reset everyone's usage limits in four hours, and a bunch of people were saying, brilliant, fire up Ultra, let's, let's burn through those four hours because we know we're getting a reset, which is unhealthy, quite frankly, you know, back when Claude Fable was on limited release, people were losing sleep because they didn't want to waste a second of the day that they could have been spent prompting these models. So that whole side of things is kind of gross. But yeah, the results that we're getting are. Really quite astonishing at this point.

CLAIRE: 00:13:00
So one of the things you said on, I don't know how many podcasts you're on. I feel special because this is your third time on Talking Postgres, but I do know. Was it, let's see, it's August. So five months ago, you were on Lenny's podcast and you talked about the state of the AI union or something like that. And in that podcast, you said something like, by 11 o'clock in the morning, you are exhausted because you and your agents have done so much at that point in time. And I guess you were trying to describe your day-to-day. And I want to understand what your morning looks like from when you wake up till 11 a.m. That leads you to being exhausted at that point. Did I capture that right?

SIMON: 00:13:45
Well, so the good news is I've found a bit more balance than I had five months ago. So five months ago, this was like peak, God, Claude Opus 4.6, I think. But it was that period where, in January and February of this year, that's when People really started paying attention to these tools because they got good in November, and then we had December and we had the sort of holiday break. And during the holidays, individuals were tinkering with them a bit. But January and February was when companies started figuring out, oh, actually, this stuff works now. And you had the token-maxxing and all of these absurd sort of absurd scenarios while people are trying to understand what we can do with this stuff. And yeah, that was a very exhausting period of time because you did feel this pressure to— you almost felt like if one of your agents isn't writing code, you're wasting time, which is a very unhealthy mentality. So I've definitely got over that now. I do not feel like I'm missing out if I don't have something churning away.

SIMON: 00:14:42
But there is a real challenge here in finding this new balance, because we can work so much faster and, you know, an agent can churn out 10,000 lines of code in an hour. That's not. That doesn't mean that it's code that you can ship, because even if the code is good, even if the code is good and passes tests, my, I feel like our professional responsibility is we have to understand it. Delivering code to the rest of the world that you don't understand yourself is a very dangerous thing to do. It's a, that's where you get the sort of cognitive overload where firstly you're sort of getting exhausted by what's going on. You don't really understand what you've been delivering. And also for projects that you intend to keep on working on, there's this interesting thing where if you don't understand the state of your project in enough detail, you can't make decisions about it. You can't sit down and think, okay, what should I do next? Because the amount that you don't understand about your own piece of software just keeps on Growing. So that's, I think that's one of the most interesting sort of friction points at the moment, is figuring out, okay, what is the fastest we can work while we're still confident that the software is good, and we're confident that we understand it well enough to be able to make good decisions about what to do next.

SIMON: 00:15:56
And when you start thinking about it like that, firstly, it means that you can slow down a little bit because there's no point in churning out a million lines of code in a week if you don't know what you built. And you start building much, much, much more sort of sensible habits in terms of spotting—there are tasks that are tiny and inconsequential, and you can completely outsource them to an agent. The other day, a website that I work on had that bug where when you log in, you visit a page and you're logged out, and you log in, and it doesn't send you to where you were before. You know, the classic: when you log in, it really needs to send you back. And I knew I'd implemented that in the past and it worked, and now it didn't work. And again, on my phone, I fired up an agent and I said, Figure out why redirects don't work, and it found the bug, and it was a two-line change, and it wrote a test, and we shipped the change, and it was done. And it was great because that's the kind of thing where If I'd carved out time for it myself.

SIMON: 00:16:52
It would have been a minimum of 20 minutes of me sort of researching, trying to figure out, and it was open-ended task. Maybe it would take me four hours to fix, and maybe it would take me half an hour to fix. But it was very difficult for me to justify investing that time on something which was an irritation, but it wasn't the most important thing I could work on. You outsource that to the agent. Either the agent solves it, in which case it's solved, or it doesn't, in which case, okay, it was harder than I thought. Maybe I'll look at it later. That kind of work, that sort of parallel work where you can be working on something else while these sort of smaller, more irritating, but not necessarily Worth spending large amounts of time on tasks are just being investigated over there. But yeah, so I feel I've diverted from your original question. These days, yeah, the mornings, I'm not exhausted by 11 a.m. Anymore because I'm pacing myself properly. I don't have that, I don't feel that tension to have as many things going as possible.

CLAIRE: 00:17:45
And I'm glad you've course corrected my question because I'm, I hate to say it, but I'm not actually interested in what your mornings were like five months ago. [Ooh, okay.] I'm kind of interested in today because I know the AI world has changed so much in the last five months. So, yeah, talk me through a typical morning in the last week or two, assuming you haven't been on vacation or something like that. Because I want, I want people to, to be able to visualize how they could be using these tools. Now, I know a lot of people are on the bandwagon and they're there, but not everybody is, so. Or not as far as they want to be.

SIMON: 00:18:28
Okay, so something I've been doing recently, which I spent a day on just a couple of days ago, I've got way too many open issues and pull requests against some of my projects. And I've got to that point of that sort of that guilt where you're like, if I don't look at them, maybe I won't feel bad about all of the ones that I haven't got to yet. And it turns out they're all available by the GitHub API, and the agents know how to use the GitHub API. So you can basically, so I actually said to, I said to Codex, Do a review of every open issue and pull request from the last six months on this repository and figure out what are the low-hanging fruit, which are the ones where the fix is actually quite straightforward and it won't take much time, or there's a pull request that's ready to land with a few changes, and then prioritize them by easiest to hardest. And it gave me a list of ten, and I went through and I knocked off the first four, and I felt so good about it. This was— Actual, meaningful. It was, it was, it was probably.

SIMON: 00:19:24
I invested about an hour of my time closing down four issues that had been open for quite a while, landed good code that I fully understood. I felt good about it. I didn't write the code. In two of those cases, it was pull requests from someone else, and I reviewed their code. Codex reviewed their code, made a couple of tiny formatting tweaks and landed it, and that was fine. And the other two were open issues where The fix was, it was tiny. You know, these are fixes that are four or five lines of code, but the problem is always figuring out where the four or five lines of code go. And so it was very quick to review. Reviewing four or five lines of code that an agent has checked out is no major— The problem with review comes when you've got a thousand lines. That's when things sort of freeze up. But if you can arrange things into these much smaller commits, and that was great. You know, that was, and that sort of— It unstuck me on that, that aspect of that particular project. And so that, but part of the problem is that I'm very, yeah.

CLAIRE: 00:20:23
But does the unsticking continue? Okay, so in other words, you had the low-hanging fruit and you integrated, you said four of them, right? [Yes.] So now what? What about all the rest of them?

SIMON: 00:20:36
So I got inspired, and I'm like, you know what, there's this one much bigger task on this project that I've been putting off. Let's spend some time on that, and that turned into a— Two-and-a-half-hour session of paying close attention to what was going on. This was solving a particularly gnarly sort of database migration-related problem. But because I'd got that momentum going from fixing some small things, I could move, I could start taking on the much, much larger project. And that was the kind of thing where You have to pay. That's more of a collaboration with the agent, where you're not just setting off on a side quest and then wandering off. You're actually paying attention to what it's doing. You're directing it. You're modifying little bits and pieces yourself.

SIMON: 00:21:21
But that was good, you know, and that, that, so that sort of the low-hanging fruit turned into shipping actually quite a significant feature, which I'd been putting off again for months. All of this keeps on coming back to procrastination and guilt, which is kind of interesting. Something I have noticed, though, that's very weird is that sometimes, in traditional software engineering, you have hard problems where you really do need to carve out four hours of uninterrupted time. You have to focus everything. There's that famous idea that if you interrupt a programmer to ask them a question, you might have just cost them half an hour because the whole stack of cards in their head comes tumbling down, and then they have to get back into the zone, and that, that's, that's, that's a major problem. That's not a problem for me anymore. Only because.

SIMON: 00:22:07
If you're working with an agent, the agent holds a lot of that context itself. See, and you can ask it questions and say, Oh, where did we get to? and get back up and running really quickly. But the really weird thing is that the harder a task is, the more you can be distracted from it, because you might have a task where an agent has to crunch away for 15 minutes working on it, and that's suddenly 15 minutes that you have free for something else. I do get substantial amounts of work done. When I'm walking the dog, which is a bizarre state of—it's absolutely bizarre that that's true these days, but I can fire up a really difficult task, go on a walk with the dog, check in on my phone occasionally to see if it's going in the right direction and maybe prod it somewhere else.

SIMON: 00:22:51
And by the end of the walk, I've got 90% of a very difficult problem solved for me. And then the challenge is, it's that discipline. It's making absolutely sure that you fully understand everything in there. My gold standard is, could I explain this to somebody else? Can I take this change, which is going to have my name on it, you know, I have to be able to take accountability for the work. Could I sit down with somebody else and talk them through exactly how it works? And if I can, I feel confident shipping that as production software, even though I wrote hardly any of the lines of code that make up the change.

CLAIRE: 00:23:22
So I feel like that gold standard is something that a lot of people are. Either struggling with or not getting right yet. It's, it's, it takes discipline to do that. And in fact, you wrote a blog post the other day that I loved that was a blog post about a blog post, which you do sometimes. You see another blog, you really like it, and you want to give it a shout-out and quote from it and kind of shine a light on it. And it was a Sophie Alpert blog post titled, There Are No Lossless Transformations of Natural Language Text, which I'd love you to expound on why you thought that was so important, but—

SIMON: 00:23:42
Mhm. Yes. It's such a great post, yeah. So this was the internal policy on acceptable use of AI writing by engineers that Sophie wrote for her employer, Clay. And it's very short. You should read the whole thing. It's like, what, five paragraphs of text. [Yeah, yeah, yeah.] But it's such an important topic right now because these things are really, really good at—they're really good at writing code, and they're very good at writing technical documentation. AI writing, if you're trying to be convincing or creative, I think it's garbage. You know, you can—you just read these things and your teeth get set on edge. I use it for documentation all the time because the whole point of documentation is it just has to describe in the most boring way possible exactly what the thing does. And so if you manage to tell your AI not to add any jokes or try and fluff things up, you can get very, very useful results out of it.

SIMON: 00:24:51
And you know, it's great at writing things like pull request descriptions because a pull request description just has to describe exactly what it is. And yet, when I see an AI-written pull request description, I don't want to read it. You know, it's like eight paragraphs, and it's all technically accurate, but it's not telling me that sort of higher-level thing where a human being is communicating to me, these are the things that matter. That, it turns out that I care about that more than accuracy in a lot of cases. I want to know, what did you, with your unique human judgment, think is the most valuable thing about this? And so, Sophie, one of the things Sophie said is, you must stand behind every idea and every sentence in your documentation. It has to be representative of your own thoughts before you share it. If a reviewer asks, What did you mean by this line? It's not acceptable to reply with, Oh, sorry, AI wrote that. Just ignore it. I love that. That, to me, because the other thing, I think it's very important not to tell people they shouldn't use AI for Writing, because a lot of people have English as a second language. I've got nearly 25 years of blogging experience. I'm a very experienced writer. It is not fair for me to say to other people, You should not use technical assistance for your writing, just because I find it easy because I've been doing it for 25 years. But at the same time, there's a very important thing about respect. You have to show respect to your readers, to your coworkers. You have to have put the effort in to make sure that the thing that you're communicating to them is good and it reflects the exact truth in your mind of what matters. And that, I think, Sophie caught that beautifully with this piece. Then her closing, that line about lossless transformations, she says, There are no lossless transformations of natural language text. Every rewrite and rephrase...

SIMON: 00:26:33
Changes the meaning of your writing, and if this is done by an entity that doesn't have the most detailed mental representation of what you personally are trying to communicate, information will be lost. And that, that nails it for me. You know, I don't care if the AI wrote it, as long as you will stand by every single detail on it and you're confident that it's the best expression of what you're trying to communicate, and it's not going to waste my time to read through it.

CLAIRE: 00:26:57
Yeah, so I feel like. I just feel like that's, that's a— A struggle for some people, because it's just so easy to skip that step, be like, okay, it did it, I'm going to move on. I want to get— Sometimes people just want to get things done, and they glance at it, they give it a superficial review. Anyway, I've seen— AI slop, and I don't like it. So...

SIMON: 00:27:21
It's, it's a plague right now. It's an absolute plague. I've got this problem on Twitter in particular because I show up on lists of prominent AI voices to follow. Anything I post on Twitter gets at least a dozen automated AI bot replies with all of the, the, the— and it's just soul-destroying. It's— I've seen a few of them start to show up on Bluesky as well, and it's just horrifying. It's like this, this, this is absolute junk, and it instantly— Which if it destroys the credibility of the people using the bots, which is, is, is something people just aren't understanding yet.

CLAIRE: 00:27:58
I mean, it's no different, I suppose, that as a project lead or a maintainer, just because a PR was written by another engineer on the project, if you're the maintainer and you commit it and you accept it, you have to stand by it. So in many ways, these agents are like the people submitting the code or the employee or the intern, if you will. You still are responsible and accountable for it.

SIMON: 00:28:25
Absolutely, yeah, it's the, the accountability and the credibility are the two most important things. If you want to stay credible with your co-workers, with the world at large, you can't be seen as just—there was a great term for this—there's slop cannons, there's the, the word slop shows up in all sorts of different ways around it. A slop proxy, I think was it. It's just. [Oh.] Copying and pasting in through exactly what the AI said, and that adds no value at all. And my optimistic hope here is I think this is a passing phase. [I hope so.]

SIMON: 00:28:59
Because it's all so new. [Yeah.] The fact that an AI can write you a decent pull request has been—that's, again, six months ago that started being the case. And the friction is already starting to show up, and people are beginning—it's becoming socially unacceptable to do that in certain circles. And I'm hoping that spreads because what I've actually, something I've started doing recently is, so GitHub Issues supports the summary details HTML element. So you can say less than summary, no, less than details greater than less than summary greater than bit of text, and then splat in the rest of the stuff. And it gives you a little collapsed, a little piece of collapsed text. So there's a little arrow and a single line, and when you click the single line, it expands.

SIMON: 00:29:44
And shows you everything else. I've started using this for the AI pull requests. So if my agent wrote a very detailed pull request, I will make it available on the pull request, but I will collapse it. So I will have a paragraph of text that I wrote saying, I fixed this and this and this, and then I'll have a little thing that says Claude 5 PR description. And if you click that, you'll get 20 paragraphs of detail from Claude, which is useful if you want to see it, but it just feels less pollute-y and less sloppy to hide that stuff by default unless people opt into seeing it.

CLAIRE: 00:30:17
Oh, that's a literal example of having to double click into something. You know, that horrible, horrible phrase that people use, but yeah, I like it. The 20 paragraphs, though, I do sometimes find that AI-generated text can be too long. [Yep.] It can be too much. It can be overwhelming. And it goes back to what you said, you need that human to figure out what matters, or we need to use our LLM tools to get to the crux of what matters more.

SIMON: 00:30:49
Honestly, like I said, I do let the AI write a lot of my technical library documentation, but I read every single line it wrote and I try to edit it not by editing it myself, by saying to the agent, make that shorter. Don't mention that detail. Drop that bit off. Split that into two bullet points. And that's something I found as useful as a technique for code review as well. So occasionally you do need to review a thousand lines of AI-generated code. It's built some kind of complex subsystem. You need to take responsibility, so you have to review it. Reviewing a thousand lines of code is miserable. It's very, very easy for your eyes to glaze over and you skip over the details. Something I found quite helpful is, I think of it as a sort of aggressive nitpicking.

SIMON: 00:31:34
Review, the kind of thing you would never do to your coworker because it's really rude to go after your coworker and nitpick every single tiny detail of the code that they've written. It's not rude to do that to an agent at all. So you can set yourself a goal to basically force it to rewrite almost every single line. And it's the tiniest, tiniest little things, like absolute nitpicking. But the goal isn't actually to improve the code so much as to make sure that you've had to think about and transform every bit of that code just so that you've paid attention to it. And I've done that for a few of these larger changes, and I think it works pretty well. I come out of it at the end, I definitely understand the code. I feel like I've got—I'm ready to stake my credibility on that. I'm ready to say, no, this is, I have reviewed this, even though it was a thousand lines. Made a bunch of little tiny changes to it. I feel good about it.

CLAIRE: 00:32:23
So you said a few minutes ago that your gold standard with a thousand lines of code or whatever is, could you explain it to someone else? [Yes.] But it sounds like the way you're getting to that ability to explain it to someone else is perhaps this aggressive nitpicking review that causes the rewrite. Are there other ways?

SIMON: 00:32:43
I mean, you can just sit down and read it really, really, really carefully. I just, I'm skeptical. I don't think I have the ability to read a thousand lines of code and really come out at the end fully confident that I understood the whole thing. I feel like I have to be manipulating that code in some way. And sometimes I'll fire up, I use Python, so I can fire up a Python interactive interpreter and try a few things interactively there. That can help. But honestly, for those larger code blocks, I think it really, the, the nitpicking review so far is the thing I found that feels the most credible. It feels like I've really forced myself to engage with the code because I'm actively trying to find reasons to change it.

CLAIRE: 00:33:28
All right, so. You can see I've done a tiny bit of research here. I want to go back to, or go over to another podcast that you were on recently, which was with Bryan Cantrill and Adam Leventhal, Oxide and Friends, who, you know, I used to work with them at Sun. We were all in the kernel group together. Bryan was across the hall, Adam was down the hall and around the corner, and so I've known them for decades, and they're awesome. But there was an example you used on that podcast where you described something that had happened that day. You said, I...

SIMON: 00:33:40
Oh, that was fun, yeah.

CLAIRE: 00:34:02
I think you, I don't know which LLM you were working with, but you said, I gave it my main open source project and I prompted it and said, do some experiments and try to make it faster. And I'm assuming that's Datasette that you're talking about. Okay. And you just checked in to see the results and it had sped it up by 36%.

SIMON: 00:34:14
Yeah, yeah, it.

SIMON: 00:34:21
And you know what? I've not even reviewed or landed that code. That is somewhere on my computer. I have a branch of Datasette that is 39% faster, and I just haven't got round to reviewing what it did yet.

CLAIRE: 00:34:31
That's exactly what I wanted to ask. I wanted to know, and so maybe we can talk about this theoretically then, since you haven't gotten to it yet. But what are you going to do next? What do you have to do before you can accept that 36% improvement change? To verify it, to QA, to review it, to make sure there's no regressions, to document it.

SIMON: 00:34:50
That is such a good question. [I know.] I think the reason I've not done it is everything in software engineering is about trade-offs, and the number one trade-off is your time, right? This is what's so disruptive about coding agents is a lot of people will tell you that it makes no sense at all to measure productivity in terms of lines of code written. That's, that's, that's— and I'd actually disagree with those people because there is this sort of hard limit in the before times. A software engineer could produce a few hundred lines of working code per day. And when I say working code, I mean a few hundred lines of production-ready, and that's actually an incredibly good day if you produce 200 lines of working, debugged, production-level code.

SIMON: 00:35:40
You can feel very, very good about yourself. Most days you'd produce 50 or 60 lines of production-ready debugged code. If agents let you produce a thousand lines of debugged code, that really is a very meaningful improvement, as long as that code is of the same quality, right? It has to be high quality, it has to be maintainable, it has to be tested, all of that kind of stuff. You can get to that point with agents, but it takes a huge amount of skill and knowledge and experience and all of the, this is what senior engineers are made of, is this ability. So on the basis of those trade-offs, the problem is I've now got a branch of Datasette with a 39% performance improvement. I know that getting that from where it is right now to Feeling confident, properly tested, QA'd, in the project such that I can explain to other people is. Several hours to several days of work, and I've not. Chosen to put that at the top of my stack yet.

SIMON: 00:36:36
And at some point, I hopefully will. But it's a huge problem because that's not the only one. I've got dozens of branches of my major open source projects now with changes of that nature, oh, it's sped it up by 39% and so forth. And they all just sit there sort of going stale because just because you can do this, just because you can point this agent and say, hey, make it faster, doesn't mean that you're ready to commit to landing that code. With code that you land, you have to feel like you're, on a major open source project, that's a commitment for life. It's the puppy is for life, not just for Christmas thing. Am I going to be able to maintain this code with these optimizations in? Yes, but only if I understand them. So yeah, it's a real problem. I can, I could have, that was probably another... The shower one, right? I can spin these things up in the shower. I learn something from them, but the gap between A very high quality prototype where all the test pass and production software that I'm ready to stake my reputation on still exists. And there's still a commitment. There's a decision I have to make to commit my time and my sort of mental energy to getting these things into production. [Wow.]

SIMON: 00:37:50
And you know what? This is another reason that I don't think we're all going to get laid off. This is, well, because it turns out, as I can do way more work as a single engineer than I could without agents. So you could argue, well, in that case, why should a company have more than one engineer? Just have one engineer, buy them a Codex Max subscription and stick them in the corner. And the answer, apart from the obvious bus factor thing, like having a team of one is a very badly designed team, is that the new limiting factor is mental. Capacity, it's cognitive capacity. And I can work a hundred times. I can churn out code a hundred times faster. I can't. I don't have the cognitive capacity to stay on top of 100 times the amount of code, so you still need a team of engineers so you can load balance that cognitive capacity effectively across that team.

CLAIRE: 00:38:42
I mean, I wonder if. The way in which your day-to-day work has changed has caused you to change how you make trade-offs, how you do time management, how you do your priority setting, and I would love for you to tell me what's working and what's not.

SIMON: 00:38:59
This, and it's not just me, this is the— any development team that starts taking on coding agents, I think this is the single most disruptive change, is that all of that baked-in intuition that you have. I've got 25 years of intuition about how long it takes to build a piece of software, and all of these knock-on intuitions from that, where I might look at something and say, should I fix this bug? It only affects a tiny little edge case. It'll take me a day. I shouldn't fix it. Let's learn to live with it. If that now takes me 10 minutes, I should fix it. If it takes me— if it takes an agent an hour to get to a proof of concept, but I don't have to monitor what it's doing, then suddenly all of these intuitions I have about what takes a long time and what doesn't take a long time and what's worth spending time on, all of— My intuition has kind of been shattered, which is quite upsetting because one of my competitive advantages as a software engineer is 25 years of intuition about how long things take. All of that's up in the air now. It's one of the reasons I'm constantly trying new experiments with agents. I love throwing, the task I threw it this morning to get my library working against different database engines.

SIMON: 00:40:08
I did that. Just out of curiosity, because I'd learned something from that exercise, and it would help me build up a bit more of a mental model of, okay, is that the scale of task that an agent can take on? Can it do this kind of thing? How long does it take? How much of my effort do I have to invest in it? And the more of those experiments you do, the better. I'm doing a lot of, I'm doing quite a bit of game development at the moment. I've been putting up little games. Partly because it's a...

CLAIRE: 00:40:35
Wait, are there pelicans in these games?

SIMON: 00:40:38
Oh, of course, absolutely. And actually, mostly more raccoons than pelicans. [Really?] I like raccoons in my games, yeah. And the reason I do the vibe coding games thing is I know nothing about game development. I've never built a game in my life. So it's a great way to see what happens when you point these tools at domains that you do not understand. And it's also really fun, and it's very humbling because one of the things I've learned from this is that it's very easy right now with—any coding agent can get you a thing that looks like a game. You can get from a zero state to something that visually you look at it and go, Oh wow, that's a computer game, very, very quickly.

SIMON: 00:41:16
It's not going to be any fun. Building a game that is fun is far beyond me and Claude and Codex and all of these tools, because actually there's a reason game development is a skill, right? And it's not that it's difficult to put the pixels in the right place. It's that designing a game loop that is rewarding and entertains people and keeps people engaged and is just challenging enough, that is incredibly difficult. And so I quite enjoy. How I can knock up these little— I built a Command and Conquer variant with these drones flying around. The other day I was building a raccoon heist game where a team of raccoons break into a museum and try and do heists together. They're garbage games. The Raccoon Heist one looks really cool, where you can tell Claude Code or Codex to generate textures for a 3D game, and they can use the OpenAI image generation thing and actually generate assets that are good enough for a naughty little prototype.

SIMON: 00:42:16
And, but the game's junk, you know, you play it, and the, the, the, the, I got to a case where one of my raccoon heist games is fun for about one minute and 15 seconds, and then you get bored of it.

CLAIRE: 00:42:27
That's very precise. I'm taking you, you clocked it. You used a—

SIMON: 00:42:31
By far the best result I've had for one of these games. So much fun though. And also, to be honest. If you were to invest real time in one of these game projects, actually iterate on it and playtest it yourself and think really carefully about things, I bet you could build something that's genuinely fun. I've been doing these as little sort of vibe-coded, not spending a huge amount of time just seeing what you can get from a prompt. But it's a fun way to learn more about the capabilities of the models, and it's a good way to remind yourself that just because you can get the 3D thing, the raccoon running around the garden doesn't mean that you've got a game.

CLAIRE: 00:43:07
Okay, so I want to go back to the question about how it's changed your, how you prioritize, how you manage your time. My sense of you, and you tell me if I'm wrong, is that you do a lot of experiments. You're very curious. You recognize that you have to go down rabbit holes and rat holes in order to have— Unexpected learnings. Oh, some people hate the word learnings. I don't know what to say instead. Unexpected lessons? [Discoveries.] Discoveries, unexpected discoveries. That's so much better. And so my sense is you give yourself time. That's not regimentedly disciplined scheduled, right?

SIMON: 00:43:46
This is where it helps that I am self-employed and I don't have a boss and I can do whatever the heck I want. [I love that.] I mean, but it's a very privileged position to be in, especially right now when we've got all of this crazy stuff going on, and I can. I get to set my agenda. I get to decide at the beginning of the day what I want to try and do. I will set myself goals, and by the end of the day, I often haven't achieved those goals, but I've achieved a bunch of other stuff. [Okay.] Partly, so January the 1st this year, my New Year's resolution, every year I make the same resolution, which is do less stuff and focus more. Pick the things that you really want to get done and focus on those. This year, I set the opposite ambition. My ambition was to be more ambitious and do more projects because of coding agents. I'm like, okay. These things exist.

SIMON: 00:44:36
What about if I set an ambition to do more stuff, and I've been sticking to that ambition, and it's been really, really fun, and my focus is still a problem, but I'm churning out a lot of cool stuff. So I'm getting away with it for the moment.

CLAIRE: 00:44:50
Okay, but it sounds like you actually do pause. Did I hear you right that you do pause every morning and set yourself goals for the day? And that sounds like something you used to do even prior to LLMs.

SIMON: 00:45:03
I've always tried to. It's the greatest— honestly, this is my biggest challenge in my career, is because everything is interesting, and it's so— and the problem with LLMs is that they make the cost of going down a rabbit hole so much lower. This thing I built this morning, this multi-database command line tool thing, that's a rabbit hole. I've spent— Five minutes prompting it and five minutes looking at it, and, and, but so the amount of time I've spent in that rabbit hole is tiny. The amount of mental energy that's being absorbed by it is difficult to measure. I've got this sort of thing in my head now where I'm like, hey, there's this little rabbit hole which I haven't put any effort into, but it's leading somewhere interesting. Do I get sucked down it? So, and imagine that with half a dozen projects a day. You know, you just come up with, because the, and this is a really interesting challenge for product design, right? So. The challenge with product design is that You need to decide exactly which set of features to build such that you end up with a coherent product at the end of it. There's actually, there's a concept in The Mythical Man-Month about this from— Oh, I'm gonna have to look that one up.

SIMON: 00:46:17
I'll find it in a moment. But yeah, so you need to have conceptual integrity. The Mythical Man-Month is a whole idea of conceptual integrity, where well-designed software has that sort of integrity to it, where there are no surprises in it. It covers exactly the right domain of things. Everything fits together and makes sense. That is so much harder with coding agents, where you can have an idea for a feature, and you run a prompt, and five minutes later now you've got the feature, and so your software grows little weird bumps in funny different directions.

CLAIRE: 00:46:48
You know my analogy for that? The Winchester Mystery House in San Jose. So for anyone who's never heard of it, it's this house that grew little by little by little, and it's kind of just a mess. So.

SIMON: 00:46:52
Exactly, exactly, yes.

SIMON: 00:47:01
Wonderful to visit. It's got 140 rooms, I think, because the woman who built it was the widow. She was the widow of the guy who invented the Winchester rifle, and her psychic told her that she would be haunted by the ghosts of everyone killed with that rifle unless she kept on building her house forever. So she built her house forever. So for 40 years, she kept on adding new rooms. That's exactly the problem with coding agents and software, is that it's very easy to keep on adding new rooms because the cost of adding those rooms is so much cheaper. But what you end up with is a piece of software, the conceptual integrity falls apart, and then it's harder to make decisions about it. And that's, it's all of this stuff, it just keeps on coming back to discipline. You have to have— Way more discipline, because it used to be that the discipline was enforced on you by the amount of time it took. You could come up with an idea for a crazy feature and think, yeah, but that would take me a week, and— Yeah, I cannot justify spending a week building this thing, so I'll forget about that. If it takes you an hour, it's so much easier to justify, oh, it's just, you know, just seeing how that works out. And now you've got a piece of, now you've got an additional feature.

SIMON: 00:48:11
And so one thing you have to do is you have to learn to throw things away, which is another thing where, in a world where a good day of work produces a couple of hundred lines of code, throwing away a couple of hundred lines of code is throwing away a day's work. There's a very strong incentive not to waste the effort you've put in. If the few hundred lines of code was knocked out by an agent in the corner, it's entirely disposable. We live in a world of where code has gone from the most valuable asset to almost the least valuable asset, which is—

CLAIRE: 00:48:41
Yeah, and you don't want to end up like a pack rat. You don't want to end up like a hoarder in a house full of stuff, stuff that you're not willing to throw away, right?

SIMON: 00:48:50
Well, so I'm a total hoarder on that front. What I've been doing, I've got certain projects which are my hoarding projects. So my favorite one of those is I've got a GitHub repository just called Tools, T-O-O-L-S, and it's got 240 HTML pages in it. And each of those 240 things is a little tool that's entirely self-contained. It's HTML, JavaScript, and CSS, and it gets deployed to my tools.simonwillison.net website, and that's the whole thing. And it's wonderful because anything, that's where my hoarding happens. Any idea I have for a piece of software that could turn into a single page of HTML, I'll vibe code it up.

CLAIRE: 00:49:29
But that's structured hoarding. That's good hoarding. That's useful hoarding. That's not, it sounded like what you were describing was code that had been created that wasn't going to get used, that wasn't good enough, didn't meet the standards, or didn't have that conceptual integrity or whatever. And so, but you don't want to throw it away.

SIMON: 00:49:33
It is kind of good, yeah.

SIMON: 00:49:51
I don't know. I think my tools thing, the only conceptual integrity to that is that they're all things that run in a browser. That's it. That's the theme. And it's a release valve. If I have an idea for some tiny little, pointless, stupid whatever it is, I can knock it up as a tool and I'll stick it in with the other 240, and it's not going to cause any damage. It's not going to corrupt any of my project outside of that one terrifying repository full of weird experiments. And I feel like that's quite a useful thing to do, is to have those release valves. Okay, because I don't want to throw these things away. But really, they just need to live in a GitHub repository somewhere where they're, where I never have to think about them again. And if I do think about them again, you know, and then I'll go back to a tool from a year and a half ago and I'll add a new feature that I need, and it's— It's very well isolated. All of that crazy vibe coding weird stuff can live in my junk drawer, but it doesn't impact the rest of my project.

CLAIRE: 00:50:44
You know, one of the things, I have this concept, I didn't coin the term, of a cutting room floor. And when I'm writing, sometimes I need to make something better, and I'm not able to edit the paragraph, the thing, the chapter, and make it better. In place. I have to put the thing in the cutting room floor, and then I can edit it. But I need to save that copy just in case I want to revert to it, or just in case I need to go back, or I don't know, my brain isn't free to create the next version unless I can revert.

SIMON: 00:51:19
And this is the same because that paragraph of text represents real work. That was an investment, 15 minutes or half an hour of effort that you put into that. So throwing it away is difficult. Yeah, I like that. I might have to borrow that for some of my own writing projects.

CLAIRE: 00:51:36
Well, it's also that I put them side by side then, and I look at them and I'm able to compare them. But if I just make my edits in place, then I can't compare them easily.

SIMON: 00:51:44
Yeah, yeah, that makes sense.

CLAIRE: 00:51:47
Okay, so challenge for product design is how you started before you got into conceptual integrity for the concept in The Mythical Man-Month. If you had to summarize the challenge for product design right now with LLMs, it is.

SIMON: 00:51:57
Features are cheap. That doesn't mean that you should build them all. That means that the editorial step becomes even more important because it used to be that the big forcing factor was features take a long time to build. As a result, it's easy to justify then prioritizing just the most important things. They don't take a long time to build anymore. That means it's very tempting, if you don't have the discipline, you can end up with a Winchester Mystery House.

CLAIRE: 00:52:24
And you don't want to end up with a Winchester Mystery House, I think is the moral of the story, right?

SIMON: 00:52:29
I mean, I, I kind of do, it's a great house, but yeah, I think in, in this particular case, you, you do not want to end up one of those.

CLAIRE: 00:52:36
Okay, so one of the things I'm curious about is how you interact with the LLMs. And you gave your example in the very beginning of the episode about being in the shower, and in that case, you don't have voice hooked up to your phone, and so you actually typed in your two-paragraph request to the agent. But I know, my son uses Wispr [Flow] all the time, and he talks to his agents and things like that as dictating requirements. It's a completely different way to spec what you want to have done, using voice and not typing. And when you're typing, when I type at least, I back up, I correct, I clarify, I rewrite, I'm making all these changes. But voice is like stream of consciousness almost. So I'm curious.

SIMON: 00:53:07
Yes.

CLAIRE: 00:53:21
How do you use voice? What tools do you use? Is it as effective as typing for you? Tell me about it.

SIMON: 00:53:28
So, my best, so for voice, I do all, I don't like talking to my, I talk to my phone, I don't talk to my laptop. And I should probably figure out a way to talk to my laptop at some point.

CLAIRE: 00:53:39
Wait, you talked to your phone, but you didn't talk to your phone this morning in the shower.

SIMON: 00:53:43
I didn't, and that's purely because I haven't figured out how to do that. Well, also because in that particular case, I was typing in specific, like ~/dev/sqlite- and that kind of thing. And, but, so my favorite way at the moment to talk to these things is ChatGPT iPhone app with the new advanced voice mode that they launched a month ago. I think they called it GPT Live, and it's a very, very good voice model. It's like you can interrupt it while it's talking. It feels very natural to talk to. And the most important thing about that model is their previous voice model was based off of GPT-4o, and it was like two years out of date, and it just wasn't very bright. And the new voice model actually has the ability to prompt a real, a bigger model in the background. So sometimes it'll say to you, Oh, let me think about that, and that means it's actually running it through a better model to get a better answer. So it's a much better sort of, much, much higher quality baseline.

SIMON: 00:54:44
So what I've started doing is I will fire up my phone, I'll fire up Voice mode, I'll go on a walk with the dog with an AirPod in, and I will talk through an idea for a project with that, with it on the phone. So I'll say, hey, I want to solve this particular problem. I forgot, I was messing around with that perennial problem of how do you change, how do you store text revisions in a database, right? You've got an article, and you want to, every time someone edits the article, you want to store the old version so that you can see diffs and all of that kind of thing. And I've tried a whole bunch of ways of doing this in the past. For this particular experiment, I thought, hang on, what if we— compression algorithms work really well, and SQLite has a blob column. What about if we store every revision in a big JSON array and then run zstandard compression on that to squish it down to a blob column and stick it in the database that way? That would be an interesting thing to try out.

SIMON: 00:55:39
So on this walk with the dog, I described my intended scheme to the chat model, and it talked back to me, and we got to a point. And then I turned off the voice mode, and I typed into the app, Build me a prototype with your Python tool, because at the moment the voice mode doesn't have access to the Python tool, but the text mode does. And so we had our voice conversation was already in the context, and I fired up GPT-5.6 Pro, which is very effective, and I told it, Build a prototype, and it did. That was a project where it was entirely me having a two-way voice conversation with the model to derive the specification. And then I typed, use your Python tool and build it, and it built it, and I got working software out of the end of it. And that was very effective.

CLAIRE: 00:56:31
I like it. I'm gonna try that.

SIMON: 00:56:33
38 minutes to build the software as well, because 5.6 Pro will burn a lot of tokens. But yeah, that one is on my blog for the 9th of August. I've actually got the transcript of what I said to it, because you can copy and paste that out as well.

CLAIRE: 00:56:50
I love it. Okay, I'll link to that in the show notes. I'll try to, you've mentioned lots of things that I'll try to make sure we link to in the show notes. I'm giving a talk in New York City at the end of September. That's a new talk for me. And it's about replication, and it's supposed to be a beginner's guide to Postgres. The event is called Postgres Summit US, which is kind of a renamed event from what was PGConf New York City in past years. And it's a beginner's guide to replication in Postgres. And one of the things I try to do in my beginner talks in particular is give people analogies, things that fit within their existing mental model. And obviously, a Postgres expert is not going to attend my talk unless they're like a friend of mine, and they just want to see me give a talk. So I'm expecting people come who need that beginner's guide, and I'm really, really curious, just to take advantage of the fact that we're talking today.

CLAIRE: 00:57:52
What analogies, what explanations you would give to people who are trying to understand the various many flavors of replication in Postgres for the first time? [Oh wow.] Is that, is that something that's in your wheelhouse at all?

SIMON: 00:58:14
My understanding of Postgres replication, the most interesting difference, it's the statement-based versus row-based. Is that accurate?

CLAIRE: 00:58:26
When you look at the terminology that gets used for Postgres replication, and I'm still, you know, the reason I'm asking is I'm still doing my research because I, you know, don't have the talk created yet, but there's synchronous, there's asynchronous, there's different names, there's physical replication, and so there's, there's, I feel like it's a bit of alphabet soup almost.

SIMON: 00:58:54
I mean, I don't know about analogies, but I feel like the thing that's always interested me most with replication is how long does it take to, if you say update articles set is published equals true across 10,000 articles, does that result in a sync? Is that quick to send to the replicas because it just sends that update, or is it slow because it has to modify 10,000 articles and then replicate all 10,000 of those in a sequence? Because that makes so much of a difference to— All sorts of operational concerns around this. I've worked at companies where You just have to avoid doing things like that at all costs, because the infrastructure cannot handle 10 million rows all needing to be schlepped out to all of the different replicas. And then you have to start thinking in terms of, okay, well, we'll batch it, we'll do 100 at a time and all of that sort of stuff.

SIMON: 00:59:41
But yeah, maybe because I feel like the biggest challenges I've always found in replication are the operational edge cases, right? It's what are the cases where this is going to break in weird ways. Certainly with MySQL replication, the challenge of adding new columns, like schema updates in replicated MySQL has been so difficult that I've worked at companies where they will avoid adding new columns to the main tables because it's just too hard. [Really.] I've seen projects that put data in Redis because it was too hard to add a new column to a MySQL table, and that always made me so upset. That's such a great way to take on extra technical debt. But yeah, I don't know about analogies, but I definitely feel like the operational concerns are things I find—

CLAIRE: 01:00:26
I've got a month and a half. I will figure it out. All right, switching back to AI questions. One of the things I'm curious about is, obviously, you had decades of development experience under your belt before LLMs hit and you started on this adventure that you're on. What's your take on which of the skills that you'd already built, that you'd already strengthened, are benefiting you the most personally in your work with LLMs?

SIMON: 01:00:56
Weirdly, one of them is engineering management. I've been an engineering manager, and I've been an engineering lead, and I've coordinated large software projects across teams of people. And I hate anthropomorphizing agents, but it certainly is the case that having that experience in managing teams of people does help with managing agents, partly because. It means that you're better at breaking, looking at the large problem and saying, okay, here's how we can break this into smaller chunks that different people can work on. And you're much better at communicating clearly. The art of dealing with LLMs is always, forget about fancy prompting tricks. It's just clear communication. You have to be able to express very clearly, here is the thing that needs to be done, and figure out, okay, and here are the bits of information you will need to get those things done. So having that, having experience with that has helped enormously. And I feel like that pattern is playing out elsewhere. I keep on, the people I know who are having the best time with agents tends to be people who've done engineering, they're senior engineers who've done engineering management work of some sort. That does seem to map very closely.

SIMON: 01:02:05
Having great success with agents. I've noticed something that I've been really enjoying observe is Anthropic, the company, keep on hiring CTOs and startup founders to be individual contributors at Anthropic. I know quite a few people who have been the CTO of a large organization, and they went to work at Anthropic, and now they're an engineer at Anthropic, and they're having an amazing time because so much of that experience they have from leading large, complicated engineering teams. Turns out to be applicable to managing these agent-driven projects as well. So definitely, some level of engineering management does seem to help a lot. The other thing, the thing that I think is particularly important today, as of maybe just two months ago, the Claude Fable and GPT-5.6 era of models. If you can break a problem down into a Clearly defined goal that the agent can demonstrate it's got to. If you can take a problem and say, okay, this problem will be solved when this particular thing is true, or when these tests pass or whatever, it's honestly extraordinary what the agents can get done. Because they're basically brute force engines. They can brute force a problem until they hit the success criteria. If you can define that success criteria, you can get amazing results out of them. And the way this has been playing out the most, obviously, recently is all of these security incidents. Agents are very, very, very good at finding security vulnerabilities because that's an extremely easy goal to define, right? The goal is get at this data that you're not supposed to be able to get to by figuring out the vulnerabilities and creating exploits for vulnerabilities and so forth, and they can do this really, really, really well, worryingly well. But it's exactly the same kind of thing as an ambitious software project. If you can say, I need all of the, I need this conformance suite of tests to pass.

SIMON: 01:04:07
And you've got an existing suite of tests. Go and build it in Go or go and build it in Rust. They can go ahead and they can do that. So that's an important one, I think, being able to define problems in terms of a goal that the agent can test itself against.

CLAIRE: 01:04:23
Yeah, and I mean, what you're describing too is not just what makes a good CTO, what makes a good technical lead. It's also what makes a good PM. It's what makes a good manager. It's that ability to get to the crux of the issue, to take a problem and really know what is the problem we're trying to solve. How do we get there? It's just—

SIMON: 01:04:44
And you know, there's one group of specialists who I really hope are having a great time right now, and that's QA people, like QA and testers. I, I feel like in Silicon Valley, sort of five to ten years ago, having a QA team went a bit out of fashion. There were a lot of companies that, that.

CLAIRE: 01:05:01
DevOps, right, or.

SIMON: 01:05:03
Well, no, and more like companies that are saying, you know what, we have our separate testing teams. Let's get rid of them. Tell the engineers to write, to test their own software. It's better for the engineers to take responsibility for that. And there's an aspect of truth to that. But at the same time, it missed the fact that QA is a skill. Being able to find the edge cases in pieces of software, find the things that break consistently and methodically figure out where those edge cases are. And I've worked with people with this skill, and it's amazing. The quality of software that you build if you've got somebody who's really good at finding the edge cases and finding the bits where it works, even if they don't know anything about writing code. They're wonderful people to work with, and it's suddenly the most important skill. Anyone who does agentic engineering projects knows that it built you some software.

SIMON: 01:05:50
You, you're on the hook for making sure the software works. You have to manually test that. You have to root out those edge cases. I kind of, I love the idea that there are all of these QA engineers now who are like, "Brilliant, we don't need the programmers anymore. Forget about the programmers. We know how to test this stuff. We can get really good results."

CLAIRE: 01:06:07
Yeah, that's cool. All right, I'm looking at my stuff to make sure before we end that we've covered all the things I wanted to cover, and we didn't. Talk about skill atrophy as much. So I asked you, what are the skills that you're really leaning on that are making you more successful in your new workflows, in how you use the agents, but—

SIMON: 01:06:19
Ooh.

CLAIRE: 01:06:32
I, I, I gave a talk at FOSDEM earlier this year, and it was a really big deal for me because it was on the main, what's it called, the main track. So it's that big room. You've been there before in Brussels, right? You've been to FOSDEM, right, with the—

SIMON: 01:06:43
I have a long time ago, but yeah.

CLAIRE: 01:06:45
Okay, with a ginormous room that seats 1,500 people or whatever. And the talk was about building the next generation of open source contributors. And when I think about that, because obviously Postgres has been around. This year is the 30th birthday, the 30th anniversary of the open source project. The technology is 40 years old, but the open sourceness of Postgres happened 30 years ago. And so I've been thinking about the future. What's it going to look like in five years, 10 years? How will the committer team change and evolve and grow? And where will the next generation come from? And what are their challenges going to be in spinning up and learning? And so I guess I do worry a little bit about people who are maybe earlier in their career who are leveraging these tools. How are they going to stand by every line of code? How are they going to build their skills? How are they going to build their judgment? So I'm just curious, what are your thoughts about skill atrophy and how AI is positively and/or negatively impacting learning?

SIMON: 01:07:48
So this one, this one, so this is such a big topic.

CLAIRE: 01:07:51
I know. It's a whole episode.

SIMON: 01:07:53
I mean, the one I worry about most actually is writing. You hear about all of these kids going through university who the AI writes their essays for them. And I feel like writing is one of those things where you cannot take shortcuts. The misery, the misery of crunching through, figuring out all of the words for your essay is the whole point. That's the thing. That's the skill that you have to learn, because writing is thinking. So, yeah, I get very worried about.

CLAIRE: 01:08:15
And I was just going to say that writing is thinking is my favorite phrase because it's true.

SIMON: 01:08:21
And so that, that, that worries me a lot. At the same time, I think skill atrophy itself, at least as a software engineer, is a choice that you make. Because you can choose not to learn anything at all and have the agents write everything. And you will then become one of these—a slop proxy, right? You're not adding any value, which is terrible for your career, and it's terrible for your sort of sense of self-worth as well. Or you can say, you know what, we've got these tools that give us all of these new abilities. They can explain things to us that we previously would have had to find a tutor for. How am I going to lean into that? And if you do that, if you say, okay, my goal is to learn new things, and one of the skills you have to learn is learning itself, right? That's a difficult thing to figure out. You have to learn what style of learning works best for me. How can I Take something I want to learn and best, best use the tools that I have available to me to help me get to that point. So I don't know, my optimistic hope for the sort of beginner programmers is I think.

SIMON: 01:09:27
Some of them are going to figure this out, and they are going to learn, like I did at the start of my career, but faster. They are going to get to the point that I got to in an accelerated way because they're applying these new tools that they have available to them, and they're able to do that. And hopefully that behavior will be rewarded, and people like that will bubble up to the sort of top of the sort of incoming set of programs, which will inspire other people, and maybe the kids will be all right because they'll figure this stuff out. But I have no idea, right? I'm not—I'm 25 years past that point in my career, and the temptation of this stuff, where it just solves the problem for you, is very real. So yeah, I feel like the most important thing people can do as individuals is to understand that skill atrophy exists, and everyone knows that, right? It's very obvious to people that if you If you get the agent to do all the work for you, you're not doing the work. You're not developing those muscles. And then deliberately develop those muscles. Deliberately say, okay, this is the project where I'm going to try and learn Rust, and in doing Rust, I'm going to set the agent to one side, and I'm going to work through these exercises and so forth. I believe in people. I believe people who are motivated to do that will be able to get that done. And then the other thing, just sort of open source generally. I've already got the problem where my projects used to get a couple of pull requests a month. Now they're getting five or six pull requests a week because the cost of creating a pull request has gone down so much, which increases the burden on me as a maintainer, on the review and so forth. And I don't know. I feel like open source itself is right in the middle of this, this sort of piece of disruption. We're having to learn new balances. We need to figure out new etiquette. Different projects are coming up with different policies around this.

SIMON: 01:11:14
Hopefully it'll shake out. Hopefully in six months' time, these conversations we're having about open source will look a lot different because we'll be able to say, this project over here figured this thing out and it worked for them. This pattern here has been well established. But yeah, no, it's a messy time for all of this.

CLAIRE: 01:11:30
Yeah. I mean, there's definitely, you answered my question before I asked it, which is how in the world are maintainers going to keep up with the onslaught of AI-reported issues? And I think people out there are using AI tools to find these issues because they want to contribute, and it's a way they can contribute, and it is valuable, right, to find things that are wrong that need to be fixed, but—

SIMON: 01:11:55
The frustrating thing is, any one of these bug reports, if it had been produced at the pace that I was used to last year, Pure upside. The level of detail they're providing in these AI-driven pull requests is very good detail. It's just the volume. It's that the sort of the systems that we have, the societal systems that we have in place are not set up to deal with this volume of— All of this stuff. So every, every, every pattern that we have, every working pattern we have is breaking down because it wasn't designed for this just cannon of stuff that's being fired at us.

CLAIRE: 01:12:33
There's got to be historical examples for how other disruptions in transportation or whatever caused changes, you know, and—

SIMON: 01:12:39
So that's the big problem is, yeah, you can absolutely find lots of historic examples, but they all played out over the case of 5 to 10 years at the least, and this stuff is playing out 5 to 10 months, right? So yeah, it's a wild time to be alive and involved in this industry.

CLAIRE: 01:12:42
How long it took, I don't know. And this is so much faster.

CLAIRE: 01:13:02
Beyond the onslaught of PRs and/or bug reports that are being found by AI tools and submitted to open source projects, do you have any other stories or examples of open source projects that you're close to of the impact on maintainers and how they're dealing with them? I ask, of course, wearing my Postgres hat. We as a community, we're evolving, we're growing, we're changing. But always looking to learn from other communities too.

SIMON: 01:13:32
I think, I think it's too early to be able to point to any one community and say what they're trying worked, because all of these policies are so new. It's, I mean, a lot of places have just turned off pull requests. GitHub had to add that feature so that people could take more control of what was going on, which is sad, but it does, it is, it's a rational thing to do if you've got this massive influx of things. It's funny, it was curl, the curl project, a year ago, they were saying, we are being inundated with security reports and they're complete junk and it's just a waste of time. And then six months ago they were saying, we're still being inundated with security requests. Most of them are legit; most of them are real issues. It's exhausting, but we're having to step up. My hope there is, I think any given piece of software has a finite number of security holes in it, and I'm hoping that we'll get them all. Ideally, we'll get to—

CLAIRE: 01:14:28
And then move on to a different whack-a-mole game.

SIMON: 01:14:30
Exactly, because the security thing is very real. It's the agents are so good at finding security issues in software, and they will find real security issues. And we have got, we do need to accelerate the way that we're dealing with these because the bad people have access to the same tools now. But yeah, so no, my optimistic take on that is— There's no such thing as infinite security holes. We can close them all. We have the tools that let us close them all. Let's lock everything down.

CLAIRE: 01:15:00
All right, so before we wrap up, is there anything That you're looking forward to, that you're excited about, that you haven't tried yet, but you're looking forward to trying, or, yeah.

SIMON: 01:15:11
Okay, there is one, yeah. I try very hard not to get excited about models that haven't been released yet because who knows how good they're going to be. The one exception, Qwen 3.8 27B is due out on Thursday or Friday of this week, and this is

CLAIRE: 01:15:28
Okay, that sounds like to someone who's not paying attention to all the new models and all the new names, that, it sounds like gibberish almost, yeah.

SIMON: 01:15:34
That's noise. Yeah, it's total gibberish. The key thing to know: Qwen are Alibaba's AI research lab. They are one of the best Chinese AI labs putting out open-weight models. The 27B size runs on a laptop with just 30 gigabytes of RAM; a Mac with 32 gigabytes of RAM can run a 27B model and still have space for other software. So I think 27B is the largest size that you can run on a laptop realistically. And Qwen's last 27B model is one of my favorite models. Their Qwen 3.6 27B is, it can do all of the things that you want a model to be able to do when it runs on a laptop. I think their 3.8 one is going to be An improvement in quality. I think it's going to be the new best model to run locally on a laptop. And that's exciting because the laptop models.

SIMON: 01:16:24
They're getting to the point now where they can drive a coding agent. They can write code and test the code and execute in a loop and do the tool calling and all of that kind of stuff. One of the Google models, Google's Gemma 4, I think it was their 27B one, the same size. That one, I took a screenshot of a web page and I gave it the screenshot and said, build this web page, and it built the web page in HTML and CSS. And my laptop did that. My laptop can now turn an image of this web page into working HTML. Unbelievable! Unbelievable that that works.

CLAIRE: 01:16:53
Were you, did you look at the code? Were you happy with it? Was it maintainable?

SIMON: 01:16:57
That's the great thing about the, oh no, it's the, it wasn't great quality code, but you can at least open it in a web browser and you can see that it worked, you know.

CLAIRE: 01:17:05
Okay, so you saw the potential, but it wasn't deployable at that stage yet. It would need more post-processing.

SIMON: 01:17:07
Exactly. But my laptop can build me a web page from a screenshot. Unbelievable. So I'm very excited. That one I'm looking forward to. I feel like the local models thing. The problem with local models has always been that they're nowhere near as good as the hosted models, and normally they're not good enough to get real work done. I think that's been changing in the past six months. I think the models I can run on my laptop now are good enough that I can—they're comparable to the best available models in the world a year and a half ago, and I was getting real work done with those models. So now I can run that level of capacity on my laptop. That's pretty exciting. It's very good news for not ending up in a dystopia where just three companies control all of the intelligence in the world, you know? I'm much more excited about the— The competition between the local AI models, the fact that they're getting good, the fact that we've got Meta's new Muse—what do they call their most recent one? Glimmer. I've run that on my laptop. That one's quite capable. That one can describe my photographs and write competent Python code and stuff. It's really exciting. I think we are going to be able to dodge the bullet of just a few companies controlling these private models that you can only use with their permission.

CLAIRE: 01:18:30
Okay, I have one last question for you. And remember, going back to the beginning, my goal through the whole conversation was to dig into specifics and details so that people could really imagine doing things that they're not yet doing. So we talked about software development and coding. We talked about writing. We talked about skills. But I'm curious if there are other types of tasks—I'm not making guacamole—but types of work-related tasks that you've seen you or your friends are also using the LLMs that have changed your day-to-day.

SIMON: 01:19:04
Really big one for me is research. A couple of years ago, LLMs started getting search tools, and they were absolute garbage, just the worst. You'd ask it a question, it would go and search some websites, find the cheapest, worst information, and then hallucinate back at you, and you just got total junk. These days, GPT-5.6 Pro, or GPT-5.6 Sol on the phone. I can't remember the last time I got bad research results out of it, and I'm often setting it very challenging research tasks where it will go away for 15 minutes and look at 150 different web pages and then give me back useful information.

CLAIRE: 01:19:40
Now, is that interaction via chat or via, so it's not, you're not setting off agents on a task there. It's just you're chatting with it?

SIMON: 01:19:43
Yep. Yep.

SIMON: 01:19:49
It is, but I, it's, it's an agent. It just doesn't call itself an agent, you know, effectively, because it can write bits of code and stuff as well. It can, sometimes it'll download PDFs and then do OCR on bits of the PDFs . The amount of success I'm having with this helping me out with really complex research tasks is super interesting. And again, the trick with that is to play with it. So when I walk the dog, we go down to the local harbour, and it's a commercial fishing harbour, so there are lots of fishing boats there. And every time I go past that harbour, I take a photo of a new fishing boat, and I tell GPT-5.6 Pro.

SIMON: 01:20:25
This is a fishing boat, Pillar Point Harbor. Research it and tell me the history and who built it and when it was built and who's owned it and as much information as possible. And it will churn away for 10 minutes, and it'll look up the visible license number in the Coast Guard's records, and it'll go, Oh, this one was built in 1954 in Puget Sound by this shipping company, and then this. And of course, I'm not fact-checking every detail, but I do occasionally click through and look at some of the links that it used to determine that. And so far it looks pretty credible. It's zero risk, right? There is nothing bad happens in the world if my AI tells me the wrong date that a fishing boat was manufactured when I'm sort of serving my own personal curiosity. But watching how it solves those problems is a great way of learning the process that it goes through and getting a sort of idea for, okay, what kind of problems can it, can you outsource to this? My rule again is if I'm going to share information with someone else, I fact-check it. Information for my own curiosity, that's completely fine. I'll accept the risk. If I'm going to publish something on my blog, it is not enough that a chatbot told me that this is true. I need to find those primary sources.

CLAIRE: 01:21:20
Yes.

CLAIRE: 01:21:32
I could not agree with that more. I guess I've always had a bit of an intolerance for sloppiness, and it drives me crazy if someone uses that phrase you described earlier, which is, Oh, I got that from AI. It must be a mistake, you know, just ignore it. Drives me crazy.

SIMON: 01:21:38
Mhm.

SIMON: 01:21:49
You're the human in the loop. The whole point of the human in the loop is to protect other people from dumb mistakes that AI makes. We have to take responsibility for that.

CLAIRE: 01:21:59
Well, and somewhere in something I read on your blog, and maybe you were quoting someone else, maybe it came from you, it's this notion that when you publish something, it's going to get read by a ton of people. And so you need to respect all those people. And an extra five minutes or 10 minutes or even 15 minutes of your time is time well spent.

SIMON: 01:22:12
Absolutely.

SIMON: 01:22:19
And there's a selfish, there's a selfish reason to do this as well, which is that your personal credibility is the most important asset you have in this, especially in this world of slop and everything. You want people to trust to come to you because they're like, I trust you. I know that you are a reliable source of information. And credibility is so easily lost. One undisclosed AI-generated bunch of rubbish, and people will stop having faith in you. And that's bad. You don't want that damage to your reputation.

CLAIRE: 01:22:48
So, so let's go back to these research questions about the fishing boats that you're asking when you take your dog for the walk. What settings do you have ChatGPT on? Are you using the deep research mode when you do that? Because you said it goes off for 15 minutes, which means it's not coming back with a quick answer.

SIMON: 01:23:05
So this is the thing, is I think Deep Research may have even been retired, but it's just Deep Research under a new name. It's called, so I think this is the thing you have to pay them at least $100 a month for, because $100 a month unlocks the Pro models, and the Pro models are the ones that will spend 15 minutes on something. [Okay.] So that's what I'm— although, to be honest, GPT-5.6 high and xhigh. Those are, those are good for search things, but I wouldn't have those. I wouldn't send those off on a 15-minute quest.

CLAIRE: 01:23:39
Got it. Any other? Well, I think that's it. We've, we've, we're definitely, we've spent a lot of time talking today. I have loved every minute of this conversation, Simon. Thank you for being continually willing to share your experiences and your perspectives. A lot of us learn so much from you.

SIMON: 01:23:57
You know what? I hope that one of the things people take away from this conversation is it's all still up in the air. All of this stuff, it's so new. It's been six months since we got the sort of—it's only been, what, three months since we had the Fable-class models. And everyone is figuring this stuff out together right now. And actually, I'll throw in one last observation. One of the problems with using coding agents is that you can come up with a very convoluted way to use them. You can say, you know what, I'm going to use story cards modeled after how Roman generals fought their wars. You can come up with some wild scheme like that, and it will work because everything works. No matter what you give to a coding agent, it will produce software that runs and does something useful. So the signal is really hard to find. You can try all sorts of crazy schemes, and everything will give you working software. What are we even supposed to do with that, you know?

CLAIRE: 01:24:54
What are we supposed to do with that? Is there an answer?

SIMON: 01:24:57
I think just keep on trying to find the— keep it simple. Go for, try and figure out what is the simplest possible way of interacting with these tools that gives you working software. And then only when something doesn't work, say, okay, maybe I should try a little bit of an extra twist on that. Because the other problem is that the models really do increase— the models improve to a point that old prompting techniques are no longer worthwhile. I'm always telling models, use red-green TDD. I'm suspicious that maybe you don't need to anymore. I think maybe Claude Fable 5 will use that technique without even having to tell it to.

CLAIRE: 01:25:33
When I think about how models are and will be changing my life, it's interesting because there's the angle of trying to make me better at my job, make me better at my work. But then going back to your New Year's resolution, there's also the angle of figuring out, well, what was I not even contemplating doing in the past that I can now do? So it's not just about getting better at the things, but it's maybe I should be doing different things. And I don't know. I'm still figuring it out myself, and I'm not where I want to be at.

SIMON: 01:26:10
One of my dream situations for all of this, I want there to be so many more little small businesses because entrepreneurship, like starting a coffee shop, is one of the hardest things to do in society, right? Just unbelievable amounts of bureaucracy and insurance and licenses and all of this stuff. If we get to a point where AI helps soften off all of the sort of bureaucratic burden of just figuring out what the hell do I have to do to take on this ambitious project, and we get more small businesses and coffee shops and things, that would be great. You know, that would be a wonderful thing.

CLAIRE: 01:26:42
Yeah. I mean, the other thing that'll be really nice is there's other places in our lives that have bureaucracy. You could think about getting permits or renewing your passport or, for me, getting compliance approval to renew a software subscription at Microsoft or whatever. And I actually, just yesterday, submitted a request for compliance approval to renew a software subscription at Microsoft. And in the past, it might have taken longer than I want to say out loud. I got it back within 24 hours. I'm like, whoa. And I do it every year, so compared to a year ago, that's a dramatic improvement. I was very happy about that.

SIMON: 01:27:11
Mhm. Nice. That's exciting, yeah, a society where all of those frictions just get shaved down a bunch is that sounds pretty good. That, that's my sort of utopian version of all of this.

CLAIRE: 01:27:29
I like it. I like your dream there. And then someone can form a small business having to do with pelican-based artwork on bicycles, maybe. [Indeed, yep.] Greeting cards, posters, little luggage tags for our suitcases. You actually have a lot of those pelicans, and people would pay money to put them on luggage tags on their suitcases. I mean, the total geeky thing to do. And only those who read your blog would recognize it on a luggage tray on the island of Paros in Greece.

SIMON: 01:28:05
OK, fine, I'll do merch. I need to do merch.

CLAIRE: 01:28:07
Yeah, you need to do merch. I think so. Okay, this has been awesome. Thank you to everybody who's listened to the whole episode, which I imagine will be a lot of people. Thank you to you, Simon, for carving time out of your summer to talk to us again. It's been a year since your last episode with us, so I hope you come back again someday, and the world will be different when you do. That's a guarantee.

SIMON: 01:28:31
Yep, and hopefully different and better, but we'll see.

CLAIRE: 01:28:36
All right. Thank you to Simon Willison for joining us today. For those listening, if you liked today's episode, and I think you will have, and you want to hear more of these Talking Postgres episodes, you should subscribe on Apple, Spotify, YouTube, or wherever you get your podcasts. And please tell your friends. In the podcast world, word of mouth is gold. You can always get to past episodes and links to subscribe by going to TalkingPostgres.com, and transcripts are included on the episode pages on TalkingPostgres.com too. And a big thank you to everybody who joined the live recording today on Discord.