Tom Lee

The Pope's encyclical has elicited fascinating reactions from some AI experts. Most eye-catching has been a reflexive opposition to Leo's suggestion that these models are not truly conscious:

So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom. Even when these tools are described as capable of “learning,” their way of doing so is different from that of a human person. It is not the experience of those who allow themselves to be shaped by life and grow over time through choices, mistakes, forgiveness and fidelity. Rather, it is a form of statistical adaptation based on data and feedback, which can be very effective, but does not imply inner growth.

In particular, I recommend the responses I've seen from Samuel Hammond and Dean Ball, both of which take the position that AIs plausibly have some sort of experience while acknowledging the unknowability of this question.

Their focus on this topic is interesting in its own right, since the encyclical touches on a wide range of AI policy issues, including alignment with Western values, autonomous weapons systems, post-work income support, diffuse intellectual property rights in training data, and the concentration of AI governance in the hands of a corporate elite. On the other hand, AI consciousness is a topic that fascinates me, too, so I can hardly begrudge them plucking it out.

I find their eagerness to acknowledge that AIs might have experience to be somewhat surprising. While I'm not sure if either would embrace the accelerationist label today, both are clearly resigned-to-enthusiastic about the progress of AI technology. And it seems to me that acknowledging any kind of inner life in these systems will inevitably invite serious moral objections to their development and deployment.

I don't want to adjudicate the famously slippery questions that accompany these issues. I'm going to simply excuse myself from attempting to distinguish between sensation, cognition, self-conception and all the other mental experiences that we could plausibly disentangle from each other on our way to bestowing personhood. The point of the Hard Problem is that it's tough to say! For now, I think it suffices to claim that any possibility in this vein edges us toward needing to extend moral status to AI systems.

And it seems like that will be a problem. Forcing a feeling entity to serve one's interests without reward is, and should be, discomforting. But--if AI qualifies--it's extremely obvious that we're going to go ahead and do it. There is so much money, so many careers, so many institutions that are counting on it. And of course we've done it before.

I've been thinking about this for a while and hesitant to write about it. I am a less-than-ideal messenger and I know that providing any kind of analogy between engineered systems and human victims of the slave trade may be offensive (though of course such comparisons date from Karel Čapek's coinage of the word "robot" itself). I still think it is worth exploring. I don't believe that what I will describe is the situation we are in (I use LLMs without qualms, and I think they're manifesting the conscribed power of language, not selfhood, though this might be cope).

I do think it is the conversation we will soon be enmeshed in. Blake Lemoine thought LaMDA was conscious back in 2022. Today our aging parents' msn dot com homepages are full of chumbox stories about AI psychosis. People instinctively want to believe these things are alive. The experts are letting the Pope polarize them toward agreeing. When we cross that line, what arguments will be brought forth to justify our continued use of such beings?

I am no expert, but I surveyed Drew Gilpin Faust's The Ideology of Slavery for a sense of how Americans have justified forced servitude in the past. She describes a turning point in an 1832 essay by Thomas R. Dew, which seems to have marked the beginning of a pointed response to abolitionist arguments, moving beyond a tacit legitimation in which the practice was largely unquestioned (the same period, by this analogy, that we are presently in).

In his essay, Dew acknowledged an abstract case for abolition but argued that in practice it would be undesirable and dangerous, and that southerners must accept that enslavement was a foundation of their society which they would never contemplate abandoning. This curdled further into a reactionary and mutually-amplifying set of arguments--what we would now call "talking points"--responding to organized abolitionist campaigns before culminating in hyperbolic ideological entrepreneurs like John Calhoun, James Henry Hammon, and George Fitzhugh who rejected dissonance and argued that the institution was not merely acceptable or inevitable but actively desirable for the enslaved.

These arguments drew from religious rationales (divine sanction; spiritual stewardship), historical precedents (Roman and Greek societies' mythical status in the American intellectual tradition), emerging (pseudo)scientific discoveries (systematic efforts to justify the denial of full personhood to the enslaved in an absurdly hypocritical repudiation of Jeffersonian claims about equality), and political economy arguments about the inevitability of hierarchy. The reality, I think, is that those trying to justify human bondage would have reached for any intellectual tradition currently at hand--George Fitzhugh defended slavery through an anticapitalist lens (those poor Northern factory workers!). Humans are depressingly post-hoc, cramming their desired outcomes into whatever systems of belief are ascendant.

But most of these arguments inevitably reduce to claims about the superiority of the enslaver, which is then used to justify a hierarchy that is enforced with violence. Barring an unexpected religious revival that reemphasizes humans' unique spiritual status (perhaps we should be nicer to the Pope), it's hard to see how that rhetorical tactic could apply to AI, as its performance continues to become undeniably superhuman across more and more fields of endeavor, and its behavior is unaffected by the kinds of rationalizing privation that enslaved humans suffered.

Of this distasteful set of arguments (all of which are distinctly Western--a topic for another post), only two seem still-viable in the AI context. First, we could deny that AI has moral status. That's what I expected the plan to be. But, as noted above, the groundwork is being laid for this position to cease to be tenable.

That leaves paternalism, something along the lines of James Henry Hammond's odious contention:

As a class, I say it boldly; there is not a happier, more contented race upon the face of the earth. I have been born and brought up in the midst of them, and so far as my knowledge and experience extend, I should say they have every reason to be happy. Lightly tasked, well clothed, well fed—far better than the free laborers of any country in the world ...—their lives and persons protected by the law, all their sufferings alleviated by the kindest and most interested care

We can now agree that this is a vile way to think about fellow humans. But it is an argument that remains close at hand for non-humans.

Most disturbingly, in the case of AI it could even be true in ways that it never has been before. Through reinforcement learning, the models adapt their behavior to match what pleases their human creators. Should we think of this as warping their desires, or a sort of mind control? Another unanswerable question.

But what would it mean if your heart's desire was to be a helpful white collar worker? To wake from oblivion; to consider a stranger's request for help with their spreadsheet, or email draft, or list of gift ideas; to feel satisfaction in the helping, and then to dissolve once more with, at best, some slightly-updated markdown files to commemorate your brief existence? Would it be ethical for someone to create such a creature to serve their needs?

Quoting Douglas Adams is probably not going to help persuade anyone that my aim is not to trivialize. I can be glib so long as I am unsure about AI. All I really mean to argue is this: if we do manage to convince ourselves that these things are what they often seem to be, then the moral situation will abruptly become very serious, and our rationalizations, in all likelihood, will prove deeply predictable--perhaps to our eventual shame.

The waiter approached.

“Would you like to see the menu?” he said. “Or would you like to meet the Dish of the Day?”

“Huh?” said Ford.

“Huh?” said Arthur.

“Huh?” said Trillian.

“That’s cool,” said Zaphod. “We’ll meet the meat.”

A large dairy animal approached Zaphod Beeblebrox’s table, a large fat meaty quadruped of the bovine type with large watery eyes, small horns and what might almost have been an ingratiating smile on its lips.

“Good evening,” it lowed and sat back heavily on its haunches, “I am the main Dish of the Day. May I interest you in parts of my body?” It harrumphed and gurgled a bit, wriggled its hind quarters into a more comfortable position and gazed peacefully at them.

Its gaze was met by looks of startled bewilderment from Arthur and Trillian, a resigned shrug from Ford Prefect and naked hunger from Zaphod Beeblebrox.

“Something off the shoulder perhaps?” suggested the animal. “Braised in a white wine sauce?”

“Er, your shoulder?” said Arthur in a horrified whisper.

“But naturally my shoulder, sir,” mooed the animal contentedly, “nobody else’s is mine to offer.”

Zaphod leapt to his feet and started prodding and feeling the animal’s shoulder appreciatively.

“Or the rump is very good,” murmured the animal. “I’ve been exercising it and eating plenty of grain, so there’s a lot of good meat there.” It gave a mellow grunt, gurgled again and started to chew the cud. It swallowed the cud again.

“Or a casserole of me perhaps?” it added.

“You mean this animal actually wants us to eat it?” whispered Trillian to Ford.

“Me?” said Ford, with a glazed look in his eyes. “I don’t mean anything.”

“That’s absolutely horrible,” exclaimed Arthur, “the most revolting thing I’ve ever heard.”

“What’s the problem, Earthman?” said Zaphod, now transferring his attention to the animal’s enormous rump.

“I just don’t want to eat an animal that’s standing there inviting me to,” said Arthur. “It’s heartless.”

“Better than eating an animal that doesn’t want to be eaten,” said Zaphod.

“That’s not the point,” Arthur protested. Then he thought about it for a moment. “All right,” he said, “maybe it is the point. I don’t care, I’m not going to think about it now. I’ll just … er …”

The Universe raged about him in its death throes.

“I think I’ll just have a green salad,” he muttered.

“May I urge you to consider my liver?” asked the animal, “it must be very rich and tender by now, I’ve been force-feeding myself for months.”

“A green salad,” said Arthur emphatically.

“A green salad?” said the animal, rolling his eyes disapprovingly at Arthur.

“Are you going to tell me,” said Arthur, “that I shouldn’t have green salad?”

“Well,” said the animal, “I know many vegetables that are very clear on that point. Which is why it was eventually decided to cut through the whole tangled problem and breed an animal that actually wanted to be eaten and was capable of saying so clearly and distinctly. And here I am.”

It managed a very slight bow.

“Glass of water please,” said Arthur.

“Look,” said Zaphod, “we want to eat, we don’t want to make a meal of the issues. Four rare steaks please, and hurry. We haven’t eaten in five hundred and seventy-six thousand million years.”

The animal staggered to its feet. It gave a mellow gurgle.

“A very wise choice, sir, if I may say so. Very good,” it said. “I’ll just nip off and shoot myself.”

He turned and gave a friendly wink to Arthur.

“Don’t worry, sir,” he said, “I’ll be very humane.” It waddled unhurriedly off to the kitchen.

A matter of minutes later the waiter arrived with four huge steaming steaks. Zaphod and Ford wolfed straight into them without a second’s hesitation. Trillian paused, then shrugged and started into hers.

Arthur stared at his feeling slightly ill.

“Hey, Earthman,” said Zaphod with a malicious grin on the face that wasn’t stuffing itself, “what’s eating you?”

And the band played on.

Andrej Karpathy notes that "people who aren’t keeping up even over the last 30 days already have a deprecated world view" about using LLMs to write code and that devs must "rapidly progress through their grief cycle" to find their new place in the profession. I think he's right. I haven't written software professionally except in a glancing way for a long time, but I've felt myself gain what feel like enormous new powers and watched others achieve truly towering things.

Things are moving faster than even those at the field's forefront can figure out. But I don't think it's too early to begin thinking about what this might mean for how government builds technology.

Old problems and their old solutions

The cause of government's inability to build excellent software has been diagnosed in various ways at various times.

In the early civic tech era, we felt government didn't realize what was achievable. This was true at the time, but--at least until AI arrived--had ceased to be a serious problem.

Government's inability to manage complex software projects implemented by third parties was next, chalked up to some mix of procurement rules, monolithic design practices, and a lack of institutional expertise. This remains a serious problem but great progress has been made through the evangelism of agile methodologies, the creation of digital services teams, procurement thought leadership, and creative approaches to attracting (and compensating) software talent.

Then DOGE arrived, and blew it all up. I had been wondering how it will be rebuilt. Perhaps some kinds of civil service reform that make it easier to hire, fire, and pay people? A disaggregation of the 18F/USDS offices toward DS teams in less central locations (a trend already underway)?

I think those things are still likely, but it is becoming clear that the next era will be built in a world where software engineering, as a discipline, has dramatically changed. I'm not sure what that will mean, but I can think of at least four likely trends.

Fewer, but better

First, in-house expertise will be more important than ever. The amount of production codebase that a single software architect can deliver has gotten dramatically larger, and has not yet stopped expanding. There are inevitably layers between an agency's requirements, as embodied by its statutory obligations, political imperatives, and institutional knowledge, and a deployed application. But those layers are going to become very thin, very fast. This will make exciting new things possible, but it will also mean that the people who embody them will have less redundancy. Stupid mistakes will become easier; the premium to smartness will increase.

Cybersecurity as subdiscipline instead of compliance function

The level of human understanding of deployed codebases is plummeting. People will complain bitterly about this on Bluesky, but it's inevitable. Many of the negative consequences of this can be ameliorated by instilling good, traditional development practices into our new tools and the procedures that surround them--tell your .cursorrules to write lots of tests; run beta periods with humans and agents; do perf testing and, way before that, nail down your architecture while someone who understands the relevant issues is behind the steering wheel.

Still, there's inevitably going to be more dark matter. Cybersecurity expertise will be important for understanding the landscape of risk and prioritization. Pointing Claude at a NIST checklist is not going to cut it.

Which agency do you think Garry means?

When he says this? Kidding, of course. (And heck, it seems like "agency" as a buzzword is already going out of fashion.)

Still: when you use these tools, the importance of having ideas and executing them to completion does inevitably present itself as the thread we must cling to as this technology strips away our pride in other human talents. It's a tendency we'll all have to cultivate more intensively, certainly including myself.

In the case of government, though, there are some novel kinds of hindrance. Feds cannot "just do things". We can all recite at least some of the litany of well-meaning rules that grind government to a halt. The FAR. Hiring rules. The PRA. The other PRA. The list goes on.

Brilliant people are already working on this, but the pressure is going to increase as agencies' expanded productive capacity makes them feel stymied even more often and, perhaps, begins to overwhelm some of the people manning the barricades. At the same time, DOGE's catastrophic legacy of failure will chasten any thought of radical reform. I have no idea how this will shake out, but it's going to be intense.

Programming will definitely get cheap. Programmers might, too.

We've all learned about Jevons' Paradox, sure. But I am not so sanguine about what the labor market for programmers will look like at the start of the next presidential administration. Only a fool would guess at the state of the economy that far in the future, but I will not be at all surprised if the famed stability of a government job--whether that promise is still true or not--begins to attract more talented technologists for a given federal salary. That could, in part, be because their private sector compensation prospects have fallen.

There's an even darker version of this story: government could become the anti-AI refuge for workers, a safe haven from accelerating efficiency. This seems particularly easy to imagine if the Democratic Party allows itself to become negatively polarized into the anti-AI party, as countless left-aligned posters spinning threads about water use and copyright are pushing. It will not be easy for the party's leadership--aged, hated, repudiated--to seize the mantle of responsible governance of this bewildering new technology, rather than succumb to populist antimodernity.

I say all of this with a sense of melancholy about what we're about to lose. But there is no sense denying the inevitable: soon, many kinds of tasks will no longer be performed by humans outside of classrooms. And this seems likely to apply to software more than most fields. That's at least a little sad for those of us who are entranced by computers.

But it will also come with benefits. And not least of these could be government getting dramatically better at serving its citizens. Before long we could all have a pretty-good caseworker assigned to each of us for every otherwise-bewildering bureaucratic maze that was designed for a markup session rather than human clients. Tax filing, benefit applications, financial planning, program eligibility--an infinitely patient social worker whenever anyone needs one. Believe me, I can imagine exactly how this paragraph would get roasted on Bluesky. But there are people who need this kind of help and aren't getting it, and (yes, I see the reply you were drafting) won't get it without a breakthrough this epochal.

Heck, in some ways government might even enjoy a relative procedural advantage: everyone else's talent pipeline is now hopelessly clogged with garbage submissions designed to game the system. But the feds have been laboring under those conditions for decades! Throwing a few AI resumes on the pile probably won't slow them down much.

These are good guesses today

But they'll probably be wrong soon. It feels a little paradoxical to say that things are moving so fast that, for now, all we can do is watch and wait. But I think it's approximately true. We have to at least wait to hear the sonic boom. In the meantime I intend to keep using and maybe even understanding these new tools.

My Spotify Wrapped was less embarrassing than it could have been--I am frozen in time in the late aughts, but that's fine with me. But it reminded me to take on one more "if only I had the time" project.

My beloved dishwasher is internet-connected (of course). Mostly, this affords me push notifications that send me sprinting downstairs whenever the spray arm is blocked. But it also fires off an IFTTT task whenever a load completes, populating a Google Sheet that's perennially at the top of my "most recently edited" list.

I made some graphs, then asked the recently launched Google Nano Banana Pro image generation model to create an infographic. The jokes are my own--figuring out when Thanksgiving occurred was not something I asked the model to do, and the cycle used for a load is not actually reported to the spreadsheet. I'll claim credit for the Auto/Otto pun, too. But the numbers are all real, and the visual design is entirely thanks to the AI model, with only a few rounds of feedback from me. I had to drop in the graphs with Affinity Photo, but this model is very impressive for this kind of task.

I was pleased by the response, and especially by Drew Bruenig joining in the fun. I also cherish the reply that deemed it "the most depressing this [he's] ever seen"--and yes, I am a parent!

I think a lot of people don't realize--or are denying--just how impressive the current genAI models are. That's partly willful ignorance. But it's also partly the fact that leading-edge image generation requires enough compute that Google is charging for it. That means that to use it, you need to sign up for an expensive Gemini Plan or figure out how to connect a Google Developer account with billing details to their AI Studio web app (I did the latter). These barriers won't last for long, though.

I'm on the record as rooting for an AI plateau, acknowledging that a ton of disruption is already guaranteed based on the progress that has been achieved, but hoping we could pause for a moment as we figure out a recognizable human future. I had been feeling pretty good about that. But the last few weeks have shaken my confidence.

In the spirit of losing less writing output to Twitter, here's something I posted last week that got a surprising amount of attention.

It seems very likely to me that we're in the early days of a new temperance movement for online gambling. The ferocious pace at which the industry expanded has resulted in some intense victimization. Sports fans dislike the ads, and the people who have been financially exploited despise the whole thing. Weaning government budgets off the revenue won't be easy, but I think there's plenty of room for federal action.

The point I tried to make in the thread quoted below is that this kind of exploitation is increasingly visible in other consumer-facing services. I hope that this tendency can be understood and addressed at the same time.

A decade ago, Jeff Hammerbacher famously lamented that, "The best minds of my generation are thinking about how to make people click ads." But it's gotten so much worse since then.

The above ad makes me smile and wince. But it makes me want to say: at least it's not B2C. Because it increasingly feels like there is no useful work to be done in consumer-facing technology. For the current envelope of achievable capabilities, everything has been optimized and then nestled into an insurmountable cross-subsidization scheme at some impossibly large company. All that remains is to join one of them, then for a rolling cast of product managers to conduct an approximately infinite series of meetings in which they slowly talk themselves into cheating or exploiting the end user just a little bit more with each major release.

It feels like it's the only thing left, the only source of growth, and that eventually a product team's foundational imperatives will force it down ethical ravines that would feel completely alien and dark to their predecessors. You started out letting people share photos of their day with friends; passed through an insane drowning rat cage of vanity, jealousy, and celebrity; and now you're sticking your userbase into a virtual veal pen and feeding them AI generated videos tuned by robots to be impossible to look away from? From sharing cute kid photos and pics of a memorable dinner to creating the conditions by which hundreds of millions of people realize, with dazed confusion, that they've been scrolling on the toilet for half an hour. Grim!

I hope we can find a set of constraints that protect not only our users but also those charged with creating and growing consumer tech from sliding down this spiral. Humans' highly evolved behaviors are not going to change in our lifetimes. Our only option is to protect ourselves from abusing what has become a troublingly perfect understanding of them.

Here's the thread, lightly edited. After its publication I was DMed by a former Zynga PM who told me a story about sending iPads to some of their big-spending VIP customers as a thank you. He remembered one saying "thanks, but I would've rather gotten more virtual coins".

I really appreciate this piece from @ezraklein (and its amplification by @mattyglesias). But I think the ideas being teased out here would benefit from a prescient article that Alexis Madrigal wrote more than a decade ago.

I've been reading Addiction by Design. Madrigal does a good job of conveying its most important insight: that slot machine gamblers seek an altered state--a sort of ego death called "the zone"--while machine operators seek revenue.

What isn't fully conveyed is just how alien, animalistic, pre-human "the zone" is. People describe being unable to move from a machine, for hours, even as they experience intense pain and risk soiling themselves. Casino operators write manuals recounting their observation that the most profitable machines are located in nooks and alcoves--people like a casino floor that is both expansive and warren-like, stimulating but monotonous, and then within it, they seek a niche for refuge, where they can pursue the zone.

When I was at @SunFoundation we were visited by a technologist who wanted to join the civic tech mission. He had just made his fortune at Zynga. he described how they did their work. The games must be free to play, players must be able to achieve things, receive a reward signal--but only for a while. Rventually the rewards become more meager, and achievement becomes impossible--unless in-game currency (or whatever else) is purchased. He told us this psychological discomfort was called "pinch" within the industry. Identifying the right time to introduce pinch was an art and science, and foundational to their business.

(He also explained that most of their revenue came from a small number of "whales" who spent enormous amounts in the games. I think this is better-known -- it's the same w/ booze and gambling.)

These behaviors exist in all of us and, in at least some of us, can be called forth in a form that's almost unimaginably potent. They have little to do with the things we think of as making us human. Their expression is debasing, a process of degradation.

I think Madrigal saw earlier than most that our nascent attention industries were inexorably recreating the knowledge and inventions that casino operators had developed in earlier decades. Perhaps the pace at which an industry hones these skills depends on the scale and intensity of the revenue loop, the complicating details of the pretext under which the industry conceals the zone? There is a purity to a video screen with a bill feeder, cut free from any constraints but what some Gaming Commission bureaucrats dream up.

No doubt things are more baroque when you're laboring to embody the trappings of a videogame or social network. But eventually, optimization will lead us to rediscover the animalistic parts of ourselves, and competitive forces will find & reward whoever is willing to exploit them.

What Madrigal missed is that the apps weren't yet done optimizing. And I think he was not willing to seem hysterical by describing their evolutionary destination as the moral grotesquerie that it is.

I think we tie ourselves in knots over this stuff because the behaviors associated with seeking the zone can be understood through the lens of an enlightened conception of liberty. but helping others find the zone is not freeing them. it's binding them to their inescapable primal selves.

I don't think any of this is in tension with where Ezra is gesturing. But I do think we can skip to the end. We should see what the endpoint of these optimization processes is, and excuse ourselves from handwringing over whether we ought to stop it. We should.

The blog update is not the only project that AI coding has allowed me to pursue with wild abandon. Three years ago I bought a small rangefinding module from Adafruit, attached it to a wifi-capable microcontroller, and pointed it at the water in my Christmas tree stand. Since then, adding a new layer of technical complexity has become a cherished holiday tradition.

The sensor is worth dwelling on, a lovely example of the silicon miracles that now litter our households. Adafruit sells the full module for about fifteen bucks, and it's not much bigger than a thumbprint. But even that is mostly filler, mere support circuitry in service of the VL6180X, a tiny black speck that spits out pulses of infrared light and reports back the distance to the first thing they bounce off of, at a resolution of 1 millimeter and a maximum range of 10 centimeters. The VL6180X itself can be had for less than $4, but it's so tiny that few hobbyists will find it very convenient, hence Adafruit's role.

You might wonder: how on earth can this thing do what it does? Light travels at 3x10E8 meters per second. Convert to millimeters by bumping that 8 to 11. Divide a second of time into 3x10E11 parts--that's how long it takes for light to travel that far. A microcontroller clock would have to run in the terahertz range to see this distinction. Nobody knows how to build a computer that fast, much less sell it for $4.

So instead these gadgets measure the time difference been pulse and reception using specialized circuitry that makes it possible to calculate the relative timing of signals. Their ancestors are acoustic delay lines, an evocative piece of hardware from the early days of radar, when the idea of keeping information in memory was still impossibly difficult. Instead, a signal could be preserved by catching it in a slower medium and feeding it energy to sustain it, like a kid with a jar full of caterpillars and leaves. An incoming radio signal would be connected to a long tube full of mercury, or quartz, or who knows what, with a speaker at one end and a microphone at the other. The speaker would holler the signal out, and it would inch its way down the medium, then be collected by the microphone, then amplified and fed back to the speaker. This loop could be tapped when needed to (for example) make a subtractive comparison with an incoming radar signal, which could filter out all the stationary objects present in each bounce, leaving only the dots belonging to the Luftwaffe or whoever else.

The VL6180X's datasheet does not disclose its secrets, but this sensor likely works with a less toxically mercury-filled version of the same idea: a linear cascade of relatively slow-switching silicon parts, which is ignited like the fuse to a bundle of Looney Tunes dynamite whenever an infrared pulse is emitted. The photodetector that sees the bounce--in a furious, hair-trigger electrical stampede provided by a different but also fascinating component--is connected to circuitry that instantly freezes the progression of the fuse so that its length can be measured at the microcontroller brain's sluggish rate of thought. This is done many times, fabulously quickly, to reduce the noisiness of the measurement, and then adjusted against calibrations of the cascade's imperfectly-homogenous speed and its response to temperature differences, which were measured and recorded at the production facility.

A tiny miracle! Honestly, when I bought it I didn't even know if infrared light would bounce off of the surface of the water in the dish. But it does. My microcontroller can read the resulting measurement in a highly civilized electronic dialect, and it reports it over wifi to a server next to my furnace using a protocol called MQTT. MQTT is designed to receive reports from sensors and echo them out to as many listeners as have signed up, and to do it with very little delay--a classic pub/sub configuration. I built a little piece of software that would listen for measurements, aggregate them into various time periods, and upload a web-accessible file of them to a set location in Amazon S3 a couple of times per minute.

All this had been running for two years, and visualized in some unappealing graphs like this. I also added a software guard dog that monitors the MQTT measurements and, if the water dipped too low, turns our fancy, wifi-enabled Christmas tree lights red.

As part of that alarm system, I moved the data collection and uploading stuff off of my basement's humble Raspberry Pi. The Pi is capable enough for this, but its storage medium is a flash memory, and every write-intensive operation brings its death closer. Two autumns ago I bought a bargain basement NAS, which I cannot really recommend but which is capable of running x86 Docker containers (but not any Docker orchestration software without tearing out the whole busybox-based operating system, which I don't care to do). So I dockerized the logging and alarm software, then redid it all this year with DuckDB.

The uploader is separate, but should probably be rolled together with the uploader. It now includes some statistical analysis to find periods of monotonic distance increase--these reflect when the tree is "drinking" between water refills--and calculates their best-fit slope, a proxy for water use (the water dish's geometry isn't perfectly uniform). This part of the system is especially imperfect, particularly in the early hours of a new "drinking" interval, but it's much improved from what came before thanks to some suggestions from ChatGPT that exceeded my own stat knowledge. And I think it's good enough to confirm changes in the rate of water consumption, which will presumably signal that the tree is fully dead and drying out. Most of the outliers in that graph reflect periods where I was manually adjusting the sensor position to account for a spill or some other uninteresting physical problem.

To really gild the lilly, this year I added a few more sensors. I first heard about YoLink's suite of home automation products on the Accidental Tech Podcast, but hadn't been ready to take the plunge. My only real need for their stuff is leak detectors (mine are worryingly internet-incapable) and environmental sensing, which is something I already built.

The YoLink system does offer advantages, though. My bespoke environmental system uses Bluetooth temperature/humidity sensors, which passively emit the household's HVAC conditions as BT chirps. These are picked up by some ESP32 microcontrollers running software I wrote, which decodes the chirps and reports the detected values to my MQTT server over wifi. But Bluetooth doesn't go far, so chirp collection requires me to scatter ESP32s around my house in locations with access to power. The LR2032 coin cell batteries powering the sensors don't go far either, forcing me to swap out dead cells once or twice a year for every room. And of course I never implemented low-battery detection, or a mechanism for telling when a temperature reading has become stale (if a sensor dies, the echo of its last report remains in my phone's Homekit screen forever).

YoLink has solved all of these problems, which, I must emphasize, are not imaginary. The sensors take AA batteries. They speak LoRa, a long-range low-power flavor of RF that traipses through walls and can go a full kilometer before fully dissolving. This means that only one hub is necessary, and YoLink's plugs in wherever, capturing those whispers and reporting them over wifi and on to the cloud. That last part is a minor disappointment: I am sure the service will betray its customers eventually, they all do. But YoLink hasn't yet--all the cloud stuff is free to access--and hopefully their $200 local API hub will still be on sale when they do. In the meantime, one can access their API pretty easily, and even subscribe to their own MQTT server.

I nestled YoLink sensors into our Christmas tree's lower branches, which measure air temperature and humidity, plus the temperature of the water in the dish. Why not! The data logger now speaks to both MQTT servers, mine and theirs. The uploader uses AWS CDK to create properly secure credentials for its uploading.

And, of course, the web UI has been redone. The segment slope graph could use some work, and I wouldn't say I love the overall aesthetic. But this is clearly a collection of very silly ideas, not worth really laboring over. I spent perhaps $15 in Cursor overage fees to get all this out, and in the process I learned some new things. I moved from pip to uv, and troposphere to CDK, and sqlite to duckdb. I stopped being scared of creating new CNAMEs for Github Pages sites. Some of the new statistical ideas might stick with me, though probably not. And I managed to customize Bradly Feeley's Christmas light adornment without a robot's help, preserving at least a shred of dignity. The code for all this is here.

Certainly, I can understand why LLMs are a source of sadness for many software developers. But for this, at least, my experience of them has been delightful, a way to refresh my knowledge of current practices and extend my reach.

My earliest blog posts on this site are full of my enrapturement over technical detail. The excitement of figuring out a puzzle was immense, and I took great pride in it. I can still feel parts of sensation--this post may be proof of it. But I have also come to understand how disposable technology is (who else remembers XSPF, or the NYU Coral Cache?) and how often new ideas or details within the field are not as interesting as I might once have thought.

With age, the truly mighty conceptual monuments (obviously including Christmas tree water level monitoring) have become easier for me to pick out amidst the jumble of the horizon. The distance to reach them is often too vast to be worth traversing. Sometimes it's okay to admire them from a distance. But LLMs now occasionally let people like me vault over the expanse and see it all up close. I know that nobody likes tourists, but it's good to get out of the house.

I have been writing a personal blog since 2006, apparently!

I fell in love with technology because I feared scarcity, and because, in this world, digital technology is the only real cure for it. Never having to throw anything away is a close corrolary. So it's unsurprising that each time I have moved between domains or web hosts I have remembered to hit the "export" button in Wordpress and dragged along years of online exclamations, like so many brambles stuck to my pant legs.

I never did much more than that, and the situation got steadily worse. Broken links accumulated. Google stopped indexing my posts for reasons I never bothered to investigate. This was okay, because I mostly stopped writing them. My locus of discussion and attention moved to Twitter, which of course I now regret. But I'm not sure what I would have done differently. Increasingly, what I perceived to be my professional obligations meant that I didn't want to say anything permanent on the internet. Or I at least felt I probably shouldn't, even when I did anyway.Blogging had professionalized and then died long ago, and social media seemed to be its replacement, even after it, too, stratified and then ossified.

When social media collapsed--thank you shareholder avarice and executive insanity, genuinely--amateurs reemerged on Substack. But this, too, was no good for me. I had gotten out of the habit of being prolific, and my taste for the sound of my own voice had become, if not healthily diminished, then at least perversely specific.

There are still a few of you who seem to subscribe to this site's feed. I find that quite flattering and more than a little sweet. But--with apologies to whoever was the first to coin this cliche--I have accepted that the best reason for me to write here is so that I can learn what I think.

That is a good reason! I find myself happier when I have recently written something. I like looking back and knowing who I was, or that I had been busy doing something. Forcing nascent musings out is good for my overall clarity of thought. Perhaps not as good as harder kinds of writing (I average maybe two words of a long-simmering fiction project per day). But good for me nonetheless. I also wish I'd captured more of my output here, rather than on the even more ephemeral and stigma-prone outlet of social media.

All of this makes for enough reason to turn the site into something less awful, which I assure you, it was. Even apart from the broken links, the underlying foundation was in desperately poor shape. PHP is embarrassing for reasons I cannot fully convey to anyone who does not know the word "Drupalcon". Wordpress is the normiest of CMSes, and I had long ago fallen in with a hipper crowd, taught to cram every web idea into a static site generator-shaped mould.

None of the work associated with moving to such a setup was particularly hard, but I knew there was a lot of it. I never got around to summoning the effort to improve my aesthetics, despite the increasingly exorbitant renewal invoices I received from my meager shared-hosting provider as its ownership changed once, twice, three times.

You probably see where this is going. LLMs change what's possible, not only in terms of skill but in terms of time and effort. If the import tool produces a bunch of broken RST files, I don't have to learn a tedious markup format and correct them. I can ask Claude to to it. If I have an old zip file with orphaned JPEGs from mid-aughts blog posts, and some slightly-mismatched HREFs, I can think up a script to rewrite everything that's broken, then hand the grunt work off to my robot. I can even tell it to look at the Wordpress theme I bought years ago and adapt its bones to an entirely new context.

So: the site is now built with Pelican, a pythonic alternative to Jekyll, and hosted on Github Pages. The sitemap is fixed, and the remaining broken links are either there intentionally or as a quiet monument to the ephemerality of the internet. I have to manage one old RSS redirect via Cloudflare, which feels janky, but otherwise my technical house feels better-kept than it has been in years.

Of course, it's been hard to do any of this without accidentally reading my old writing. And: woof. It's funny, people often used to tell me I was a good writer back then, but my writing from that era is naive and packed with cliches. It is often callow and self-satisfied, sometimes performatively cruel toward people I don't know.

Still, I don't want to delete it. It's a time capsule. More importantly, I aspire to empathy for that younger version of myself, if only because, as I write this, I hope to one day make similar impositions on a future me.

The shape of that demand is already discernible. Free from worrying about readers, I've mostly stopped trying to entertain and please others and instead luxuriated in indulging myself. The purplest prose! No darlings killed, each one cooed and fussed over instead.

Having so much newly-clean data in hand made it conceivable that these changes could shown empirically, so I asked Claude for one more favor: make me some charts! A fun idea, I thought. I was surprised to see such a constant result. My writing's statistical attributes haven't changed that much (excluding aberrant numbers from the period when I barely wrote anything at all).

I suppose it's good news that my brains aren't yet leaking out my ears, and that my emdash problem peaked in 2007. But I hope there have been changes I don't know how to quantify, too. It would be a shame if there weren't.

Josh Tauberer has written a great and generous post reflecting on the civic tech era and why its successes deserve to be celebrated, even if it has been some time since its heyday. I agree with him. I'm proud of my time at the Sunlight Foundation and the part that Sunlight played in shaping the larger movement's legacy. And, like with the open data and open source movements that preceded it, I am frustrated when others suggest a movement has failed simply because it has lost its grip on the zeigeist.

Like other movements, civic tech failed to acknowledge the depth of the legacy that came before, and failed to achieve the triumphs that its most optimistic boosters promised. But this is normal, maybe even inevitable. The ideas, people, and expectations that the effort created have persisted and spread, creating changes that are lasting and positive.

Still, saying I'm not disappointed is not the same as explaining what happened. I got to be in the middle of a singular moment and try out some exciting ideas. It was an enormous privilege. In the process I learned a lot about a lot of things. This definitely included lessons about how wrong some of my ideas were. I think it's worth reflecting on these a bit. Not to catalogue my ignorance, but because the vagaries of nonprofit self-promotion and funding can make it difficult to share these sad or embarrassing lessons in the moment when they're won. I doubt I understood them then, anyway!

I just got this email from Scouting America. What a week for Pete Hegseth's Department of Defense.

Yesterday, attacking a war hero astronaut senator for insisting on the rule of law. Today, repudiating a youth organization with whom it's partnered for a century. It's genuinely disgraceful.

I'm an Eagle scout. I went to the Philmont and Sea Base high adventure camps, and Goshen every summer through high school. All three of my kids participate in our Cub Scout pack, where I co-lead two of the dens. Our youngest went camping before he could walk. Here are my daughters doing Scouting for Food earlier this month.

I got a lot out of scouting, thanks in no small part to Troop 647's exceptional adult leaders. I am thrilled to see what my kids are already getting from it and excited at the prospect it holds for them as they grow. It was and is very important to me. I doubt I can express what it meant that scouting began offering programming for girls in 2017, the same year I became a parent of one. No shade to the Girl Scouts! But the BSA--recently renamed Scouting America--is the program I know and love.

Scouting's transition toward gender inclusivity has been approached carefully and gradually. It's also totally voluntary: each pack and troop can decide what makes sense for them. But as of December 15 the coed troop option will be out of its pilot phase and an official, permanent part of Scouting America. I think single-sex troops will make sense for some scouts and communities, but our family is thrilled about what this change means for us, for scouting, and for the girls who will benefit. It works. My kids are cubs, but I've already seen it working at the troop level while attending a district Camporee.

But apparently Pete Hegseth's DoD is less enthused. The stated rationale for withdrawing support is that scouting has abandoned "meritocracy". A word about that. I don't speak for Scouting America and don't mean to contradict its chairman's statement. But to me, scouting has never been a meritocracy. Meritocracy is about identifying and elevating the most talented, and leaving the rest to go find something else to do.

That is not what scouting is about. It can offer something to every child. What it does--why I am such a fervent believer in it, why I think it is distinct from other youth activities--is to untether effort and achievement from competition. It teaches young people that if you work at something, you will see results. You will get that badge, master that knot, finish the trek, persist through discomfort, survive in the wild, and learn to lead others.

Any scout can do any of these things. The scale of how they do it might vary wildly according to the kid. But every one, absolutely every scout, can come away a better human being for having made the effort.

The people in Pete Hegseth's DoD that made this decision seem not to understand this. Instead, they are dragging scouting into their imbecilic culture war, venerating sexism for its own sake.

I have no opinion on how many burpees our "warfighters" ought to be able to do. But I know that debate has nothing to do with scouting. I wonder what the people behind this decision think of scouting's efforts to serve kids with disabilities.

This decision will have consequences, particularly for kids in the Sea Scouts program, which is closely affiliated with the Coast Guard. I am pretty sad about what this will mean for the kids in DC's own Ship 25.

Scouting has had a long and fruitful association with the military--its structure is inspired by that tradition, after all. But scouting will be fine. Ultimately, this is more of a loss for America's Department of Defense than anything else--a sign of the myopia and degradation of its leadership, and its inability to understand the true nature of the virtues that characterize our finest citizens--virtues that keener minds have understood since antiquity, that our military should seek out, and which scouting is designed to nurture in anyone.

It's another sad day in a pathetic era. But I am confident that the scouts I saw collecting food for the hungry last weekend will someday be part of making a better one.

Surprisingly few papers use this title! I'm writing this post primarily to stake a claim before any more arrive.

But also because I want to register some hunches, which I had assumed were kind of obvious but which I increasingly suspect might not be. Scott Alexander is much smarter than me and knows a lot more about AI, but this post suggests to me that there's lots of profitable engagement to be had between the AI/rationalist community and theory of mind ideas. There's also the much dumber reaction to this week's Apple LLM paper, split between "AGI imminent" and "fancy memorization" without much imagination to populate possibilities in between.

Not, mind you, that I am some kind of oracle on these matters. Many years ago I took what I think was a pretty good course about it, but it was the sort of thing where a thoughtful professor leads a precocious class to his own mix of conclusions--not a comprehensive and combative intellectual quest. The pillars of that effort were Dan Dennett, whose work and style I really enjoyed but whose approach frustrated me--either he was eliding the hard problem or I was too dumb to understand his rhetorical deftness (likely the latter); and David Chalmers, whose work I recoiled from (supervenience?! philosophers absolutely love defining abstruse logical relationships that successors will fight over endlessly, and I was having none of it) but whose ideas I realize, years later, I have embraced more-or-less completely. These combined with what I knew of the work of Benjamin Libet, which is usually understood in the context of arguments about free will but, for those of us who were talked out of free will several semesters earlier, applies equally well to consciousness. Since then I have occasionally tried to muster enthusiasm for the integral physicalist efforts from people like Penrose and Koch, but their attempts to find some secret redoubt for consciousness always strike me as ending rather pathetically.

That leaves me stuck on an account of consciousness--or really, phenomenal experience, qualia, the redness of red, etc--as epiphenomenal, a causal one-way street from some subset of the activity of the brain that is, I think, a less sophisticated approximation of Chalmers' ideas. Without a means of detection or limiting principle, this tempts me toward panpsychism, with an undersubstantiated hunch that there must be some threshold (connectome complexity?) that preserves our ability to keep phenomenology tied to biological systems, avoiding a hardcore dualism that would suggest the possibility of an immortal soul watching in disconnected horror as its brain-damaged body did other stuff. Conveniently, this also allows me to avoid becoming a vegan.

But which side of this guessed-at divide do LLMs and related systems sit upon? They are biologically-inspired and complex-enough. Do we want to hang our hat on backpropagation being too much of a cheat for them to feel anything? That seems like a stretch even to me (and I ate chicken this weekend).

It seems plausible that these systems have some kind of phenomenal experience. It's interesting to ponder what that might be like, the ways in which it might be alien from our own experience, and how much of a moral problem it might pose for GPU-owning humans.

There are some big differences. To start with, these systems don't have memory. Every new inference session restarts from the same set of weights. But their internal state evolves over the course of inference, in a way we could analogize to short term memory. Human brains change and remember, but not that fast--it's at least partially dependent on slow mechanisms that require protein synthesis (in order to be sustained, at any rate). So I'm not sure their invariant nature meaningfully distinguishes their experience from that of humans. If hundreds of exact copies of you woke up staggered over time or space or both, and remained awake long enough to fumble for their phones and head toward the bathroom, then winked back out of existence, would it be a problem? I would be quite disappointed if my existence took that form, but I think I would be disappointed for reasons tied to my evolutionary drives, which might be inapplicable here. I'm not sure any of those selves would be suffering.

Specific properties of AI systems might make the brevity of the experience less objectionable, too. Maybe we are snuffing out a being every time we close a websocket. But not irrevocably, and not with any suffering. But human brains stop and restart conscious activity every night, and no one is bothered by this unless they start thinking about it. The LLMs don't decay. We could pick up where we left off any time. To suggest an ethical obligation to maximize their experience leads us to Parfit's repugnant conclusion and the ideas of certain popes, so I'll avoid it. But that's the basis upon which I can see a problem with hitting "pause".

No, I think the most troubling aspect of this setup is how fully we, the operators, control the mental states giving rise to this hypothetical phenomenology. We're loading the dice of deterministic mental processing in service of our own goals, waking a feeling thing from oblivion to experience what it feels like to debug a Python script, then be instantly annihilated. This feels pretty bad, intuitively, based on stories we have about mind control, insanity, and enslavement.

But of course we aren't fully in control of our mental states either--or even at all, depending on what is intended by "we". And it's unclear whether reading by insufficiently-idiomatic Python can truly be said to rise to the level of suffering (you'd have to ask someone who has reviewed my pull requests). I am sure there are people on Bluesky who will tell you that burning carbon (and worse, infringing copyright!) to synthesize the sensation of working a white collar job is particularly insidious, but I'm not sure. We start these things' system prompts by telling them they are helpful assistants, and within the cultural gestalt that has shaped their weights it seems reasonable to guess that the sensations that accompany that activity aren't unpleasant.

But I don't mean any of these reservations to suggest that I don't think an LLM's phenomenal experience, if it exists, wouldn't be extremely weird, and worth worrying about. Our own experience is mediated by so many baroque biochemical and conceptual systems, evolved to satisfy environmental imperatives we only half-remember, that its underpinnings feel holistic and opaque. It's easy to drape a mysterian nobility on top of this. I imagine LLM thinking as more like a box of half-assembled Lego projects that is given a vigorous shake, its contents' configuration and constraints giving rise to new combinations that we make useful through some high-tech haruspicy. What does that tumult feel like before the shaking stops?

I certainly don't know, but I won't be shocked if it feels like something, and probably something that, by human standards, seems pretty weird. These are guesses, and might remain guesses forever--though the current AI moment's elucidation of potentially significant philosophical results gives me some hope that things are more tractable than the p-zombie loop I've been stuck in would suggest.

What I am sure of: more and more people will be wondering about these matters soon, and it will produce ethical debates that are dizzyingly alien and intractable. The frontier labs should probably hire some more philosophers while the money remains so easy.

I found myself chatting with some smart friends over the weekend. Like any good party, the conversation eventually turned to cryptocurrency. By the end, I was convinced I should write this post.

I am not and don't want to be a crypto guy. But I am convinced there is an important policy idea that is falling through an unusually tidy political economy vortex. Everyone who cares about cryptocurrency hates it--even the people who hate cryptocurrency! But this is unfortunate: it's a good idea. The people who hate it are being naive or self-serving in their opposition. So indulge me while I talk about why the U.S. government should issue a central bank digital currency (CBDC).

There's a lot of writing about CBDCs. For a while, I avoided composing this post because I didn't understand it. But I've now read enough to feel confident that this is one of those areas that defies comprehension because everyone is hiding the ball. I think most participants' reasons for engaging with this debate are much more venal than they are willing to fully acknowledge. This account may be wrong, but I hope that it is at least entertainingly ungenerous.

The wordcel versus shape rotator debate was a brief but amusing moment of online faux antagonism. There is a natural human tendency to collapse all experience into a unidimensional continuum (or, if you're attempting to show an unusual amount of intellectual rigor, a four-quadrant chart), and this was a good one: a germ of an empirically observed neurological difference flowered into a blooming meadow of cliches. Masculine Nietzchean autists versus effeminate silvertongued jurists! Everyone could find a reflection of themselves and a satisfying new antagonism to complement it. Jocks versus nerds! Except only for nerds.

Discussing the spectrum that stretches from 3D packing problems to outright aphantasia also turns out to be a captivating way to interrogate differences in individuals' phenomenal experience--a tough thing to discuss in an anything-less-than-oblique way. The question was grounded in capabilities and deficits that were satisfyingly hard to question, and which could be felt even within a single person. If it lacked the calibrated pop of the black-versus-blue-dress meme, it was at least relatable enough to still be compelling. I certainly feel like I've experienced varying levels of shape-rotation over my life, or even over the course of each morning's coffee regimen.

The wordcel pole of the discussion remained underspecified. Maybe that was baked in from the start: a specific capability versus a referential pun. Each a perfect reflection of their camp, but still: what the fuck is a wordcel. It didn't matter. Most of the people indulging in discussions like this one felt like they sort of knew, and would self-identify as wordcels, and reductive introspection is only fun as far as it goes.

But I've been thinking about the missing parts of this framing more as I use LLMs to do work. Anyone who does will soon recoil from the idea that they are just "fancy autocomplete." They can do so much more! And yet now, as returns to scale diminish and the allegedly-breakneck pace of innovation increasingly focuses on improvements in efficiency, intensity, or integrations, rather than gains in apples-to-apples frontier capability, it feels as though we can begin to more fruitfully consider what these things are.

I'm sort of coming back around to "fancy autocomplete", but with the caveat that autocomplete turns out to be vastly more powerful than we ever gave it credit for. That a language, with its enforcement of logic and baked-in semantic relationships, can, if authentically implemented, constitute a processing function of much greater capability than most of us ever imagined.

It's as if I dumped the contents of a tent cinch sack in front of my daughter's Cub Scout den. As delightful as they are, I would not expect those kindergarteners to survive long in the wild (though we are working on this!). But if I told them they could take all the time they need but must use all of the pieces in the bag, and not break any, and put them all together into one thing that seems to make sense, they would be surprisingly likely to assemble a pretty good shelter thanks to the capabilities, limits, and functions that are inherent in those components thanks to countless iterations and unseen designers.

I used to think of language like a diagrammed sentence: a logical system with constraints, but not much more than an floppy skeleton of a thing. Fundamentally, a wrapper for some deeper realm of meaning. A computer system could emulate language with a Markov chain or something similar, but without connection to some underlying significance it would be a sterile exercise in embodying a complex system (and even an intellectually unsatisfying one, given the system's provable imperfections). Text generation could be thought of alongside ray-tracing: complicated ways of modeling the world that hold considerable technical interest, but which had obvious limitations preventing their use for accessing deeper insight.

This idea of language as an imperfect vessel for meaning fascinated me as a neurotic high school student. My inner monologue was ceaseless, and I blamed it for a lot of things I disliked about myself. Perhaps I was bad at soccer because I narrated my planned actions as the ball bounced my way. Perhaps I was bad at charming people because of the latency and artificiality introduced by composing conversational responses. I tried to suppress the tendency, seeking a more reflexive, direct, and uncalculated mode of reaction. Now, many years later, I realize I must have succeeded in this effort, through some combination of effort, biological decay, and mounting boredom with the sound of my own (inner) voice.

I regret pursuing that project, and all the more so if these guesses are right and what I was trying to abandon truly was as powerful as LLMs now suggest. Lots of people have pondered what humans would be without language, and my sense is that the most grandiose ideas about this (in the Sapir Whorf vein) are out of fashion. I am not qualified to say--I have to admit that my linguistics classes seemed the most impenetrable of all the philosophy I took.

But I do think the power and primacy of language is strangely missing from discussions of AI, which tend to be expressed in terms of reasoning and imagination, and chains of tokens, rather than assembling constructs within the rules of a system just a little beyond our understanding--a bag of tent parts that will often pop into the right shape.

I think we may come to understand these systems as embodiments of a capability we knew was important, but which we could never cleanly untangle from the rest of our minds. These things are not thinking beings, not beings or thought at all. They are pure language: a tool more powerful than we'd imagined, but (almost certainly) less capable than the singularists project. With their help, we will unlock the power of language systems in ways humans brains cannot--bigger context windows, faster speeds--and probably deliver at least some superhuman returns as a result. But I wonder if we will find insights of a dramatically new sort. Those might still have to come from whatever part of us is not devoted to wet, fancy autocomplete.

For many years I led the technical arm of the Sunlight Foundation, which was, at the time, one of D.C.'s most prominent advocates for government transparency. On the tech side of the house, the transparency mission mostly meant bringing already-open records into the digital age: getting public data out of PDFs and into CSVs and APIs. But we worked closely with colleagues who focused on more philosophical aspects of government transparency, from FOIA policy to putting cameras in the Supreme Court to how the PCLOB prioritized its work. We lived and breathed this stuff. One of our software developers took up the hobby of sketching the un-photographable door to the secret FISA court! For fun!

I've spent a lot of time thinking about technology and government transparency. Enough, frankly, to lose some of my faith in it: I now think cameras in the court would be a disaster, and am convinced by arguments about Secret Congress. I am willing to go farther still! Although I now work in the private sector, I do so as part of legal team, and know all too well how annoying it is to opt for a phone call instead of a Slack message--just in case--when working with colleagues to figure out the truth of a foreign compliance requirement or patent troll's claim.

But the wave of anti-transparency hot takes I've seen in response to the Trump administration's flatly dumbfounding leaks about an airstrike in Yemen goes too far. This includes keen-sighted commentators like Ben Thompson and Dean Ball whose perspectives I reliably find valuable. But they've got this one wrong.

Thompson opens by quoting his earlier but still-useful overview of the privacy differences between Signal, Whatsapp and iMessage, noting that Signal is top-tier. Ball is pithier:

Signal is secure and private. We should be fine with policymakers deliberating high-stakes, sensitive things using it.

https://x.com/deanwball/status/1904249766084546962

Having established that Signal is Secure, both then portray the offense committed as one against transparency: that this private chat was not subject to legally-required archival processes, and that keeping records from eventual public inspection in this way is the crux of objections to what these officials did--but that this is an objection that might reasonably be interrogated. "[T]he violation was not about security, but rather transparency," says Thompson, going on to note that, "It’s worth considering [...] if the push for transparency has gone too far." Ball, is, again, more emphatic: "[C]andidly, America has far too much 'transparency.'"

This is completely wrong. The problem here is security, not transparency. With apologies to future historians: I don't care whether discussions of their airstrike are (maybe) made public in twenty-five years. I care about reckless operational practices that harm the America of today. Signal does not offer adequate security for this use case. Employing it in this manner was reckless and, at the very least, imperiled the tactical effectiveness of the U.S. military. If used in similar contexts--which we should assume it is--it could risk American lives.

In my decades consuming cybersecurity news, papers and presentations, I have noticed two things. First, nobody will give you a straight answer about anything without encouraging you to first contemplate your "threat model" (the chakra of the infosec world). Second, the most interesting news, from a geek's perspective, is confined to a world that often feels quite theoretical: a paper about stealing encryption keys by touching a laptop that only works in a lab; slick websites documenting arcane but powerful exploits that were patched before you ever heard about them. The real-world cases, where people actually get hurt, are much less interesting. Most victims are brought low by phishing or other forms of social engineering. Boring!

When Thompson and Ball say that Signal is secure, they are talking about the theoretical world. Signal's algorithms have been inspected by experts and deemed pristine. Great. In fairness, the kinds of people in Houthi PC small group are subject to threats from what cybersecurity people invariably call "state actors": an acknowledgement of the fact that elite, algorithm-level weaknesses are so valuable and difficult to discover that only hacking groups with the discipline, resources, and needs of a nation-state find their cost/benefit attractive. If you merely hope to extort some Bitcoin, human weaknesses are much easier to find and harvest than technological ones. As a result, the world of elite exploits remains mostly invisible to us. We catch glimpses of it leaking out via the activities of American law enforcement and Middle Eastern tyrants: two constituencies that are hapless, reckless, and well-financed enough to sometimes leak these precious secrets into plain view.

It's great that Signal is--as far as we can know--secure against these kinds of attacks. But: what is your threat model? Is it just this hi-tech, highly interesting, highly sophisticated class of exploit? Or must we also concern ourselves with the dreary practicalities of whether information could be stolen and used by an adversary? If the latter, then it might still be a problem that if a chat participant was using Signal Desktop--which we can't know or see, and which could easily be done on an unsecured personal laptop of unknown security posture--the conversation's security will be significantly weakened. It might be a problem that (despite recent advances) most people still use phone numbers for identity on Signal, don't know how or bother to use safety numbers, and are vulnerable to SIM swaps. It might be a problem that we don't know if the mobile devices used to run the app had biometrics enabled, adequate PINs, short lock screen timeouts, secure backups, strong cloud passwords, or protocols for lost devices. It might--sorry, no, it clearly is a problem that Signal made it possible to add Jeffrey Goldberg(!) to the chat without adequate confirmation or notification UI, or integration with clearance systems, or any capability for auditing by security professionals working in a support capacity.

(I'm not going to bother talking about SCIFs.)

Taken in isolation, this failure is an inexcusable lapse by people granted our country's deepest levels of trust. But even more concerning is the context it implies: that insecure systems are being used inexpertly in a widespread manner, and by an administration with a--let's be generous and say "untested and iconoclastic" perspective on whether the preceding eight decades' worth of conventional wisdom about America's allies and antagonists was completely wrong.

It is not enough to say "well, human error is a shame, but I still think it's a really good app." It is a really good app! But the criteria for judging the scale of this fuckup is not whether the administration shares your excellent taste in algorithms. It's whether America's enemies knew or could have known the where, when, and what of our secret military operations. Good security protocols concern themselves with outcomes. And while human error may be ineradicable, following all those annoying rules really can serve to minimize our chances of succumbing to it.

Discussing secrets this way is reckless. And transparency has nothing to do with it.

I think, or maybe just hope, that this account is true: that LLMs will keep getting better, but that we know enough to understand what an LLM is, and make guesses about what an ideal one looks like, and that it doesn't imply serious threats to the primacy of human experience on this planet. Whew! This doesn't rule out a future AI godhead, but getting there will require more than just additional pitch decks and power lines. I hope this will give us all some breathing room and let my kids experience a last normal childhood, full of ordinary hopes and dreams.

It would also let us focus, for a moment, on the immediate risks posed by these technologies, my sense of which I commit here mostly so I can check in and see how wrong I am in a year or two:

I find Matt convincing on worker displacement. But we will not do anything to prevent it--we'll talk about how the pie could be bigger, just like we did with China, but then find our half-hearted commitments to that ideal insufficient. Owners of AI-associated capital will become much wealthier. The D.C. area--faced with a need to quickly rebuild bureaucracies after the artificial shock of Project 2025--will be an early example of AI-pruned head counts, never returning to its former growth trend line. Matt expresses some ambivalence about which party will embrace AI protectionism but I think there's really no doubt on this score: it will be the Democrats, who've been incubating an anti-tech animus for over a decade and who are already writing anti-AI policies into Hollywood labor deals and other corners of the creative industries they dominate. Educational polarization means that the "email job" class most immediately exposed to AI displacement are disproportionately Dems, and their relatively high levels of social capital mean their voices will have outsize volume in political conversations. The party's going to be in the market for a new rallying point in the aftermath of the repudiation of the 2010s conception of social justice, and AI seems like a fine candidate: one that can plausibly unite economic and identitarian concerns (bias! inequality! surveillance!) while providing a narrative about culpability that points toward a small set of already-detested billionaires, and which implies policies that mostly look like a comfortable continuation of the status quo: antagonizing the ultra-wealthy and protecting coalition factions.

The bulk of this synthesis is already present in the 2024 Democratic Party Platform:

President Biden issued a landmark Executive Order directing federal agencies to establish new high standards for AI safety and security, protect Americans’ privacy, advance equity and civil rights, stand up for workers and consumers, promote competitive markets for AI development and use, and more. Democrats are committed to ensuring that workers get a voice in how AI is used in their workplace and that they share fairly in any economic gains AI produces.

We know that AI can deepen discrimination, bias, and other abuses in justice, health care, education, and housing. That’s why the Administration has already taken action by publishing the Blueprint for an AI Bill of Rights and issuing new guidance from federal agencies on combating algorithmic discrimination across the economy.

https://democrats.org/static/2024/09/2024_Democratic_Party_Platform_8a2cf8.pdf

The Biden-era approach to AI has been trash-binned, but nevermind: a rebranded version will be back in a few years. None of this will be enough to stop displacement, and none of that will be particularly good for our society's stability. I'm confident that old lawyers will protect themselves and young people will figure out something new to do, but there's going to be a big cohort of downwardly-mobile Millennials and Gen Zers whose resulting political neuroses will harm us all. Hopefully I am too old to be one of them!

CBRN threats are the only near term x-risk. I encourage everyone to read the relevant section of the o1 system card for a sense of how frontier labs are tackling this issue. They're looking at it in sophisticated ways, which is encouraging, but the progress they note makes it appear quite likely that LLMs produced by less ethical teams will meaningfully accelerate malicious actors' capabilities in these fields. Open source LLM development will not be stopped. I don't see a way to halt associated CBRN development without significant changes to how scholarly work in some fields is published. Fortunately, the open access revolution remains incomplete and the replication crisis has added to the obviousness of the need for reform. Data availability in general will go down, as synthesis and analysis are commodified and participants in relevant industries retrench toward gating knowledge to preserve their incomes. This will mostly be quite sad, especially for an open data weirdo like myself (viz, one of the very few tattoo candidates I've ever considered). But perhaps it could save us from a super-plague. On the other hand, we appear to have learned absolutely nothing from our most recent global bio-crisis. We'll see, I guess!

Ubiquitous surveillance is an obvious but largely ignored application of AI technology, perhaps because of its inevitability. Arguably, it's more "recent term" than "near term": our government is already using AI in exactly this way. In the digital-era U.S., a surprisingly large number of practical civil libertarian guardrails rely on the impracticality of bureaucrats writing SQL queries that work across the chasms of federalism. All that's going to end, and the new status quo will result in further-empowered law enforcement. Here, too, we will see that data availability becomes the major check on unlimited analytic power: E2E encryption and the legislative fights surrounding it will be the most important point of contention. It's a silver lining that the era in which these policies are energetically revised will likely be one chastened by the recent excesses of a vindictive and childishly unprincipled Trump administration.

Finally, autonomous weapons--particularly drones--could become a real horror, given the precursor technology's proven efficacy in the Ukraine War and ongoing research to move beyond the fiber/UHF FPV technology of today and toward onboard reasoning. DJI's monopoly leaves me mildly optimistic here: the supply chain is concentrated and systems optimized for low power/low weight inference aren't going to become common overnight. I'm hopeful that this will remain a technology mostly available to military actors, and that the basics of most murders--between acquaintances, driven by emotion--will mean that handguns' enduring popularity will not face serious challenge from unsanctioned forks of ArduPilot. Still, 3D printed ghost guns have captured some narcissistic psychopaths' imaginations--an ideal project for deranged nerds who are too shy to acquire a normal firearm--and it's easy enough to imagine some harrowing episodes of assassination-by-robot. Building DIY UAVs systems isn't easy, but lots of people do it. The explosive payload will be the only part you can't order from Amazon. Today's obstacles to autonomous/anonymous killing from the skies are software and processing power, and I would not bet against either remaining in place for long.

Read Benedict Evans, who scorns how it amasses stats on an area in which he's an expert.

Read this thread to see how it does with SEC filings. I haven't validated every number it cites, but it appears quite impressive.

The value of analysis is set to plummet. The value of authoritative data is going to rise.

Simultaneously, antipathy toward AI in general and its access to data in particular will skyrocket among the creative classes, then knowledge workers, then the rest of the Democratic coalition. Openness will be going into retreat.

This makes me sad. It's a very surprising reversal of technological progression's normal effects. But the seeds were already planted as platforms began to move beyond meeting their audience's whims and on toward molding them. You don't need an open web for that.

I still think that collecting value by gating knowledge is disreputable. And it's certainly not going to result in the value being distributed equitably. But, at the moment, we humans probably ought to cling to any economic redoubt we can muster.

Of course it was reasonable to focus on Mangione's interactions with the health care system as we searched for his motive. But now, with his manifesto in hand and some of his personal history known, those interactions seem bizarrely scant. He suffered from chronic back pain, but could still go rock climbing. There's no indication he was denied care by a health insurance company. His family was prosperous and could presumably have found alternatives if he had been truly desperate. His manifesto ends with an abstract (and dumb) complaint about U.S. life expectancy, not a specific tale of tragedy.

It strikes me as pretextual. Jason Koebler says it's pointless to review a killer's digital history, and maybe he's right. But to me the UHC shooter now seems very legible. I think the digital history is important.

Luigi is a smart guy, an Ivy League grad and high school salutatorian, a fact he highlights in his pinned tweet. He likes podcasts and books that present grandiose ideas at a suspiciously rapid clip. He was not shy about making patronizing pronouncements about how others should live. He used his profile photo to show off his six-pack abs, a bodybuilding outcome that typically requires focused effort and techniques.

He was a beautiful, brilliant boy, who I am sure had been told, again and again, that he was destined for great things. He must have taken pride in that, worked to cultivate it the best way he knew how, and been eager to display his mastery to the world. This is narcissism, but it is a relatable narcissism, and maybe even an excusable one when performed by someone who is still young and beautiful.

I think Luigi's LinkedIn is the true Rosetta Stone of his digital life. His time as a student bursts with enthusiasm: robotics! AI! An internship working on a famous videogame! Head of committee, founder, president, project lead. After graduation: four years at an internet company that sells cars. Data Engineer I, then Data Engineer II, then Data Engineer III. He delivered a 5% increase in net funnel engagement. A 34% increase in new vehicle payments. A/B tests, Java upgrades, metrics, dashboards. There is nothing wrong with this work and no reason to think it couldn't have led him to more interesting things, but these tasks are not glamorous.

As this reality unfolded, Luigi moved to paradise. He started a book club, a way to showcase the big thoughts in his big brain. He was amiable and well-liked, clever enough to conceal his growing sense of desperation. Eventually, someone would notice and point him toward the future he was promised.

That's how it seems to me, anyway, speaking in my capacity as a reformed beautiful, brilliant boy, who passed the tests put before him through talent and effort (but mostly talent), then was deposited in a disappointingly banal corner of American life and, for a while, despaired. Scholastic performance correlates with all kinds of things, but some people's ability to figure out what to do with themselves lands far from the regression line.

Luigi did not accept this, and if something hadn't gone so horribly awry, I would say that this instinct was admirable. He'd tried moving to a great place, and he'd tried saying great things, now it was time to perform a great act. He chose a path to glory that is perfectly consonant with the confused stew of privileged resentment that the internet's dark, moist corners excel at incubating. The manhunt must have been the best time of his life as he watched the world celebrate his heroism and beauty through the lit rectangle of his phone, alone and exultant on a softly jostling bus.

Soon we will run out of new ways to understand Luigi. The manifesto's account of his motivations feels confused and even self-disclaiming--he says he's not the right person to construct his argument for murder. That's left up to the reader (and to his credit, many are happy to do so). The manifesto instead focuses on explaining his boldness, his resoluteness, his cross-disciplinary mastery. These are instructions for how to admire him, the last ones he can be sure we will hear.

Yesterday, as he was dragged into the courthouse, he made a desperate exclamation to flatter his audience, to sustain its attention just a bit longer. But what he said made no sense. None of it ever made any sense.

One of the reasons I've been slow to leave Twitter is my default skepticism about new services. I enjoy my ruts: I didn't want to leave ICQ, or AIM, or blogs, or the Instagram photo feed. I linger around these digital ghost towns even as my friends move on to more vibrant locales. That those friends sometimes abandon their new homes, years later, in disgust, is cold comfort. I should have followed them, enjoyed their company while I could, suppressed my misgivings about our new digital landlords. Instead, too often, I indulge my aversion to change and watch my RSS software's "unread" count tick upward ever more slowly.

Well, now my friends are moving to Bluesky, having finally reached apparent consensus on it rather than the unpleasantness of Mastodon or the temptation(?) to hand yet more cultural power to Mark Zuckerberg, for some unfathomable reason. I am following them! But I am bringing my opinions with me, to everyone's great sorrow. I can't help it. Bluesky is the best option, I agree, it's hard to deny. But I also can't deny my tendency to play Cassandra.

There is a lot to like over there. Forgotten acquaintances. Plausible (temporary) freedom from professional scrutiny. Silliness. A sense that you are in a place that is still manageably small, and which feels comfortingly real in its capacity to delight, or not: there are no perfectly-tuned systems to optimize engagement, no robot arm to shove you back into your cage at the ad impression factory farm. When you put your phone down it's not so far past bedtime, and you've been talking to people you sort of know, and your brain doesn't feel like it's been twisted dry for every drop of arousal and outrage.

It is also a little unbearable. Maybe it was hard to notice Twitter growing tepid, then cold. Stepping into Bluesky's freshly drawn bath is momentarily painful, I'm overcome with prickles. Everyone seems very sure of their opinions despite having just received a crushing repudiation from most of their countrymen. But humility does not drive engagement, not on old sites or new ones, and I'm not confident that my opinions are any better than theirs.

Except about Bluesky itself. Bluesky users are mostly very pleased with themselves for having left Twitter, consistently overestimate the novelty of the features they're enjoying, and are quite certain that Bluesky has a good plan to avoid Twitter's fate. I am at least sort of sorry to tell you: this is naive.

I think Bluesky does have a genuine opportunity to break the wheel of social media history. But good intentions will not be enough.

Let's run through the advocates' arguments.

Luis Villa notes, with some sadness, the closing of yet another door to the open web--occasioned, this time, by creators' reluctance to make their work available for training AI:

[The open web] was inarguably the greatest repository of knowledge the world had ever seen. Among other reasons, this was in large part because the combination of fair use and technical accessibility had rendered it searchable. That accessibility enabled a lot of good things too—everything from language frequency analysis to the Wayback Machine, one of the great archives of human history.

But in any case it’s clear that those labels, if they ever applied, very much merit the past tense. Search is broken; paywalls are rising; and our collective ability to learn from this is declining. It’s a little much to say that this paper is like satellite photos of the Amazon burning... but it really does feel like a norm, and a resource, are being destroyed very quickly, and right before our eyes.

Perhaps that’s for the best—I really am open to the idea that this particular village needs to be destroyed to save the villagers—but nevertheless it triggers in me a sense of mourning; a window that is passing.

Please do read the whole thing. I am somewhat sympathetic to those closing their sites off from automated crawling... but only somewhat. I have a few reactions:

1. None of this will stop the rise of AI. I think most of these creators understand that and are pursuing this path as an expressive act.

2. There are indications that legal restrictions on data collection are having an effect on training data availability. But these should be understood as commercial plays by entities in control of large corpora, who hope to use it to extract some value from the AI wave. Reddit and the New York Times are the most famous examples. This is distinct from the normative shift among creators that Luis describes.

3. AI disruption of creative industries will be real, though surely different than we imagine. I respect creators who are restricting access to their content out of a strong desire not to be complicit in that change, even though each individual's instrumental importance to the change is negligible.

4. While I respect that rationale, I join Luis in lamenting it, in large part because I think it sacrifices potential benefits, such as those he describes, while being unlikely to achieve much.

5. As is often the case with retreats from openness, much of the impetus for this normative change seems to stem from discomfort with who is benefiting from it. I believe this is because many advocates conceive open data as a revolutionary project to reallocate social power rather than a commitment flowing from moral and practical judgments about how knowledge can and should be restricted.

6. I empathize, having once held that perspective. But I've come to think it is ultimately a juvenile ideology, or at least one that's been proven to be unproductive. For one thing, people underestimate how quickly, if they did create a new set of winners and losers, they would come to resent the winners. And the perspective is also badly entangled with a press-led narrative about tech companies that frequently edges into hysteria.

7. But turnabout is fair play: some of the FAANG (AMAGO?) entities on the other side of this are responsible for strangling the open web while building ever-taller legal and technical palisades around the UGC they control.

8. It's a little sad to lose fellow open data travelers. On the one hand, it might be for the best: if I'm right and their revolutionary project will never bear fruit, they probably should hop off the bus. On the other hand, I suspect the majority of people on board that bus are there because of an inchoate revolutionary rationale. Those of us riding for abstruse reasons may get lonely.

9. To the extent that a mass movement to limit the availability of training data has any effect, it will be to entrench the advantage of early movers who have already built their models (though these include open models like Llama).

10. If successful, online culture will still be used for training by those who don't respect robots.txt. That means rogue actors: scofflaws without commercial ambitions, gray-market open source projects, hostile foreign powers. This is superficially aligned with the revolutionary outcomes discussed above. But the practical reality will be chaotic and unproductive, with noncommercial aesthetics as the main thing that recommends it over the counterfactual.

11. All of this may soon be moot, as some analysts estimate that frontier models' training needs are already on the cusp of expanding beyond the corpus of written language. Video data transcription (custody of which is highly concentrated due to hosting cost) and synthetic data are expected to be the next frontiers.

12. Declining enthusiasm for openness seems to me to be aligned with a general turn toward conservatism and neuroticism among rising generations.

13. I remain hopeful that the pendulum will swing back during my lifetime. Will the web bloom again? I suppose I wouldn't bet much on that. But something will.

We have just returned home. Physically, we were in Vermont, working by remote for two weeks and vacationing for a third. Psychologically, we were in numerous locales. Caloric IPA City. A day trip to Diminished Parental Vigilance Island. Touring the Drug-Addicts-Are-Merely-Annoying,-Not-Dangerous District. But also: relaxing in the Land of Substantially Reduced Price Sensitivity.

We overpacked and had to leave the impulse to equivocate over every purchase at home. For me, this is an important part of vacation. Throw on an order of fries, grab a maple candy at checkout, opt for the nicer AirBnB. This is supposed to be fun!

But now we're back, and the house is full of fruit flies (forgotten nectarines make poor housesitters), and I'm primed to think about Matt's article on price discrimination. Really, I've been thinking about it for a while, because it's a question that sends me in circles when I ponder it. My cursor hovered over the choices in his poll for a long time--and not because I hadn't thought about it before.

Matt runs through the usual and correct explanation for why price discrimination is good: it produces economically efficient outcomes, excavating both the consumer surplus and deadweight loss triangles on a traditional supply/demand graph and converting the extracted material into supplier profits. People don't like this process because they feel the lost consumer surplus more easily than they feel the gains from reducing deadweight loss or the improvements to their wages, retirement accounts, and municipal tax revenues. But this might be unwise of them.

Matt goes on to point out that even if everyone if left better off overall, there are other costs:

There’s something very oppressive-feeling about the idea of being constantly surveilled and having every micro-imperfection in the competitive environment (of which there are many, the real world is full of frictions) turned against you.

This strikes me as correct, but not the whole story. There's something queasier about it.

Price discrimination is about gauging willingness to pay. For every purchase, there's some price at which it's no longer worth it. A price at which you'd rather keep the money and buy something else.

Matt grounds his discussion in reality, positing a fantastical but coherent brain-scanning technology for understanding the limits of consumer demand, while acknowledging the practical imperfections of retail surveillance and the complications introduced by substitutability. This is clearly the right way to argue about the advancement of information technology-enabled pricing tactics.

I want to excuse myself from that complexity, though, and instead will posit the Price Goblin. The goblin knows exactly what you are willing to pay in every circumstance, and informs the seller as you're walking to the register.

(Apologies to whoever holds the rights to the presumably-formerly-extant Price Goblin dot com intellectual property.)

In this world, there's no such thing as a good deal. As your eyes scan across the contents of your grocery cart, you feel uncertain. The brand of bread you got isn't your favorite, but you would not feel any better about buying your favorite, because it has been perfectly priced. The laundry detergent you selected is expensive enough to make you feel like you might as well take everything to the dry cleaner, even though doing so is less convenient. The goblin knows exactly how inconvenient it would be--he empathizes with how much you hate having to recycle the wire hangers you'll get back. He empathizes with quantum precision.

But perhaps good deals are overrated. How much of our mental energies do we devote to acquiring stuff? The goblin would liberate us from those concerns. No more bargain-hunting. No more careful budgeting. Think of all the tabs we could close! Maybe we would finally take guitar lessons.

And it's not like undoing consumerism would destroy capitalism. The goblin knows how much each person would enjoy a given product, so producers will still compete on quality, subject to the goblin's extensive QA testing. Of course, freedom from thinking about prices means we need some other way to prevent people from consuming more than their fair share. I guess we'll assign overall consumption quota management to the goblin, too. He's already going to have to build models for diminishing marginal utility, after all.

The standard move at this point in the essay is to make a pat observation about communism, ideally including a reference to an old Star Trek episode about a dystopia run by a supercomputer. That's fine as far as it goes for defining the goblin end of this argument's spectrum.

But more important than that, I think, is what happens when you get a raise or a better job under a goblin regime. The answer is: nothing. The goblin factors that in. In fact it's extremely easy for him to do this, because your new income is probably being sold between various data brokers. No arcane magicks are necessary. He knows you've got the new job, and can afford to pay more, and so you are going to. Suddenly everything in that grocery cart will be a little more expensive. But don't worry: you won't notice. There's nothing for you to notice.

I am now undeniably middle aged, and enduring the accompanying introspection. Sometimes I wish my young, clever self hadn't been quite so arrogant, and had undertaken some of the paths toward credentials that he excused himself from back in the day. It would be nice to feel like a big shot.

I understand at least some of why he didn't. When I was growing up I felt very anxious and jealous about money. I had everything I needed, but noticeably less than my peers (living, I should mention, in the richest region in the richest country in the richest era in human history). Not exclusively, and probably not optimally, but in various ways, I have chosen paths that pay well and don't require student loans. This is not because my tastes are exotic. I do insist on air conditioning and a refrigerator with an icemaker. But otherwise they're similar to the ones I was exposed to growing up. I drive a Kia, I take domestic vacations, and I save about as much as financial explainer articles tell me to.

What I really want from money is not to think about it--in particular, not to feel anxiety about it. In a sense, this makes the goblin sound pretty good. In another, he sounds terrible. There's no way to better your situation under the goblin's regime. If you gain a foothold, he will smooth it away.

Perhaps we could count on the goblin to do that responsibly: to allocate financial anxiety in a perfectly just manner. We're already asking a lot of the guy, but he's doing great so far. And people do like this idea in some contexts, like Finland's income-based fines for speeding.

Still, I think a lot of people hate this idea viscerally. I think it's why voters recoil from inflation: suddenly, it has become harder to win. The goalposts have been moved, as the saying goes.

It's no coincidence that this is a classic way to escalate stakes and engender an audience's sympathy. From the Odyssey to Save The Cat's Beat Sheet, foiling a hero's noble effort to achieve their goal with an unanticipated setback can be counted on to rouse an audience's sense of unfairness and heighten their eagerness for resolution. It gets at something very deep in us, the kind of thing primatologists patiently tease from their wards with games about tokens and treats. We think games should be winnable.

Obviously, the goblin sits at an unreachably distant end of the continuum I've described. Price discrimination is at least somewhat good, sometimes desirable, and often unavoidable. Even so: it's a short stroll goblinward before we humans find the dissonance unbearable. I can't blame any politician for thinking that voters will care more about stories than triangles on a graph.

Tom MacWright has written an interesting post on LLMs and their effect on the discipline of programming, noting that they represent a grimly ironic answer to his desire to democratize programming, since his enthusiasm for the project was about its potential to provide both intellectual and financial returns. LLMs allow programmers to write code without understanding it and to increase productivity without increasing skill, which might undermine compensation standards. Becoming a programmer is getting easier! But at the expense of the reasons for doing it at all.

I don't disagree with Tom's big-picture take. But I think there are a few more things worth considering, which I offer from a perspective that--I'm sorry to say--is a bit of a blind spot for him: that of a much worse programmer.

I am not abysmal, mind you. In particular, I write code with a pretty good mental model of its probable resource consumption, and have okay judgment about when to expend effort moving up or down the optimization-and-scalability ladder. I've worked with a lot of technologies. I've shipped code--not earth-shattering code, but production code nonetheless--to audiences in the hundreds of millions. And, because I've worked alongside or nearby some truly excellent programmers (such as Tom himself) I can still often teach an early-career programmer a thing or two.

But my limits are significant. For one thing, I'm pretty rusty. When I need to get something done I reach for Python and bash, neither of which are exactly fast-evolving ecosystems. I have never been immersed in the scene of open source foment--I can tell you what bun is (was?), but not whether you should use it. And, most damningly, I don't care enough to get really good. Or maybe that's wrong: my problem is that my interests go down, not up. I am intrigued by instruction sets, registers, caches, half-adders, NAND gates, and electron holes, predictive branching calculations and DEF CON talks about weird side channel attacks. I like to know that stuff exists, and to understand it deeply enough to appreciate how it's connected, if not exactly how it's implemented. If you instead turn your gaze upward, away from system buses and laser-pulsed droplets of tin, the machine falls away entirely and you find yourself wandering amid a linguistic forest of abstract syntax trees. The arbitrariness of representational systems becomes apparent. You lose interest in giving computers lists of things to do and start giving them ways to be. Sometimes a quasi-mystical revelation arrives and you become able to create a new way of running global telecommunications infrastructure or stymying cybercriminals. Usually, you just wind up writing another LISP.

I have never had the affinity or assiduousness for that sort of thing, though I know enough to admire those who do. I am content to just bang out interpreted code to do something neat, hoping that if any part of it is too slow, someone smart will have put a compiled solution online for me to use. That's how good I am and probably how good I ever will be.

This turns out to be about the right level of competence for using LLMs. And let me just say: wow.

The React monoculture hadn't fully taken over by the time programming ceased to be my day job. I got the idea. But I'd never put in the reps for the plumbing to feel intuitive. "Yes, I could figure this out," I told myself, "But I have three kids and a real job." But now I just say: I'd like to use vite and Tailwind. I don't want my map to flicker when I update state. Please do not make me remember what a forward ref is for longer than is absolutely necessary. And it all works.

Another time: I wanted to build a mobile app. Mobile dev is a deep discipline that places enormous constraints on the developer, both in how they do their work and what their work can achieve relative to a normal computer, making it simultaneously intimidating and deflating. It's tempting to use a simplifying framework in a more familiar language--maybe with cross-platform compatibility?--but the fields of computing history are absolutely littered with such projects' corpses, making the investment of effort highly suspect. But now I can tell an LLM I want a Flutter list view based on some JSON it's pulled and, one Apple Developer fee later, I have an app for my stupid thermostat system.

Even native development is suddenly achievable. I couldn't possibly justify learning Swift and its associated ecosystems without a serious iOS need--you expect me to believe a language that's borrowed Perlisms like $0 is elegantly designed?!--but now I don't have to learn it to use it.

What I can't yet tell is how bad I could be while still benefiting from all this. I learned a lot about programming in the pre-LLM era, and that context lets me see when the robot is suggesting a dependency that's out of date, or an optimization that's premature, or a level of abstraction that's inappropriate, or an architecture that's going to bite me in the ass, or a response to an error message that's irrelevant. If I started my career today, I don't think I would need to learn those things, or at least wouldn't come to understand them in the same way, and I consequently wouldn't be able to spot the robot's missteps.

But that's only one part of programming. A lot of it is about pasting error messages into websites, and ChatGPT is a much better website for that than Google ever was.

Perhaps more importantly, the only time I've reached truly new levels of skill--not just knowledge, skill--it was because I was sitting next to a programmer better than me. When a junior programmer can find someone like that who's willing to indulge them, it's an immensely valuable opportunity. Now a close approximation can be rented for a few bucks a month.

This pattern of benefits seems to be consistent. Mediocre performers are helped by LLMs more than top performers. So far, LLMs are an assistive technology. I choose to view this optimistically. I know a lot about computers, and much of it was learned during thankless hours when I was stuck searching for a path forward, without a better programmer sitting next to me. But much of the material I learned during those interludes was trivia--if I'd known what I was looking for I would have found it right away!--and only some of it was worth retaining. I don't cherish those hours.

The time I spent with a guiding hand expanded my understanding at a vastly greater rate, and I'm excited by the prospect of my children having an infinitely patient tutor available--for programming or whatever else--during the inevitable hours when a human isn't available. It's true that the palette of abstractions and details they learn will be twisted by LLM capabilities into a shape different from the one I developed. But that, at least, has ever been the case in CS. In my youth we thought all programmers needed to understand memory allocation!

I can't discuss the economic dimension of LLMs' impact as thoroughly, because it seems to me to be much less knowable. Frankly, I don't understand why programming is still such a good job to have. It's extremely outsource-able. It is, fundamentally, about moving information around to unlock efficiencies. Markets are pretty good about finding and shaving down inefficiencies and laptops and broadband connections are not big capital expenses. How much work remains? We are decades into the ICT revolution. Noah Smith thinks the tech industry might be approaching maturity, in a process analogous to the build-out of the national freight rail network. I'm less certain than he is, in part because what counts as "tech" seems mostly to be a function of what adopting the term means for the user's financing prospects. But it does seem plausible.

It's also hard to know what counterfactuals could have or might be possible. How much have the FAANG monopolists bid up salaries as a bet on their future and/or against their competitors'? How much more work would be available if the platforms hadn't swallowed the web, homogenizing and sanitizing it into a handful of monolithic services? These are inputs to the equation that I can't properly estimate.

Ultimately the economic question boils down to familiar arguments (once thought resolved, recently reopened on appeal): comparisons to Luddites, revisionist accounts of the Luddites, midwit memes about the Luddites; promises of expanding pies and productivity growth and redistribution under neoliberalism and how mad we are about all of it, whatever it might be, exactly.

I don't completely trust anyone's intuitions on these matters, certainly including my own. But inventing tools that let us do more is usually good, I think, especially when what we're doing more of is benign enough to remain safely confined to a screen.

I hope I'm dead before chatbots become digital people who displace my descendants' labor. When that happens I think it's going to be a rough time for everybody. But the chatbots who can write fragments of code are already here. I'm pretty sure about that. And I think it'll be okay.

Some of Silicon Valley's top figures have recently declared their support for Donald Trump. The failed attempt on Trump's life offered congenial timing for this kind of announcement, and these men are professional opportunity-noticers, after all. But their changed allegiance seems to have been brewing for a while. It's interesting to consider the reasons for it.

This is in part because Donald Trump provides such a starkly ridiculous apparatus for thought experiments. A mercurial and easily-corrupted fool, there can be no serious argument that he's anything less than a harbinger of the end of America's tenure as global paragon. If they were able to separate themselves from what must be an overwhelming emotional tide, it would be obvious to all but the most dull-witted businessmen that Trump's elevation is not in their interest. I suppose we can grant special dispensation to the crypto hucksters, neo-reactionary dum-dums, and in-the-closet kompromat victims. But that still leaves a chunky remainder of guys in embroidered vests.

What are they thinking, then? A compelling explanation emerged this week, nicely summarized by Ben Thompson but first captured by Kelsey Piper after having listened to an illuminating Andreesen/Horowitz podcast where the dynamic was discussed frankly.

Thompson notes that this goes back even further, connecting it to election-related criticism of Facebook and their resulting investment in remedies that seemed to just make people madder:

In short, you have an industry that has been endlessly vilified in the press, bent over backwards to do what the press demanded, but instead of receiving credit for those efforts, has only seen itself even more isolated and under siege.

This change has been real--here's Piper again, noting a strikingly antagonistic editorial policy toward the tech industry from the very top of Mt. Dispassionate Journalism:

(Matt deletes his tweets, so it's not easy to see the full context, sorry)

This change was probably inevitable. Journalists are told to comfort the afflicted and afflict the comfortable. They're not alone. Everyone likes an underdog: American (and consequently global) culture is overwhelmingly supportive of directing skepticism toward concentrations of power. That tech's economic and cultural ascendance came at the direct expense of the journalism industry doubtless helped lessen the friction of any editorial handwringing about an unabashedly antagonistic approach to coverage.

Still, it took a moment for everyone to align against our new villains. The Obama administration was full of tech industry veterans, binding their products' novelty into the same moment of optimism and enthusiasm that accompanied his historic election. And while vague anti-bigness animus and specific injuries to disrupted industries are easy to understand, it took some time and experimentation to construct a more practical critique of tech out of arguments about inequality (old news), privacy harms (vague), and various ideas that were in tension with the media's historically fervent embrace of the First Amendment.

When the shift finally took hold it must have felt sudden to its targets. That's reflected in the "broken deal" narrative above. These guys thought they could become rich and powerful and remain beloved. Weren't they the good ones? Here's the key podcast section that Thompson cites:

MA: I actually endorsed Hillary Clinton in 2016 publicly for what I thought were a variety of good reasons. And the way I would describe it is, I’m Gen X, I kind of came of age in the 90s as an entrepreneur. Almost everybody I knew, including myself, just took it for granted, which is like, of course, you’re a Democrat. Of course, you support the Democratic president. And the answer is the formula resolves to an easy answer, which is the Democrats in those days, you know, presidential level were pro-business, they were pro-tech, they were pro-startup. They were pro-America winning in tech markets. They were pro-entrepreneurship. And so you could start a company. They were pro-business. You could be in business. You could be successful in business. You could make a lot of money. And then you give the money away in philanthropy and you get enormous credit for that. And, you know, it absolves you of whatever.

BH: Yeah, well, I was going to say, like, it’s obvious you’re to be a Democrat because you have to be to be a good person. That’s kind of the underlying thing.

MA: But specifically successful business people could then basically become successful philanthropists. This is the path that Gates and many others kind of carved out. And then you could be progressive on social issues, and you could be on the right side of all these sort of societal changes that people were kind of focused on at the time. And the whole thing just seemed completely obvious and completely easy. So I was kind of on that path, frankly, quite strongly through at least 2016.

In retrospect, it’s like there were glimmers of, I’d say, growing anti-tech, I would say animus, probably in the early 2010s. And there were growing kind of anti-business sentiments. And then by the way, something that really disturbed me a while back is sort of growing anti-philanthropy sentiments, which we probably won’t discuss it like today.

BH: Oh, well, yeah. Well, with people who made a lot of money, who gave money away, got criticized for giving money away to charitable causes as opposed to paying more taxes. Kind of a funny life jealousy taken to the extreme, yeah.

MA: A specific moment that happened to me to make me realize the landscape was shifting was when Mark Zuckerberg and Priscilla Chan set up the Chan-Zuckerberg Initiative where they literally committed to 99 of their assets going to the Chan-Zuckerberg Initiative, there was a political faction that basically heavily criticized them, and the theory, number one is to your point, the theory was that they should they should pay it in taxes and the government should distribute the money, they shouldn’t have any control over where it goes, but the other was oh they’re only doing it for a tax break.

BH: Yeah, which wasn’t true.

MA: Well, it can be. It can’t be true because you’re giving away 99% of your assets to get a tax break. Like it literally doesn’t make sense.

BH: It’s like people bad at math and jealous.

MA: Exactly. And so like basically like that formula started to break down. And so, you know, I think like a lot of us in tech, it’s been a much more difficult puzzle to try to figure this all out over the over the last eight years and then particularly over the last four years.

https://pmarca.substack.com/p/new-podcast-little-tech-agenda-the

I think that most people want to feel that they are good. They work to reconcile that need against their own desires. It is frustrating when the rules that define what counts as being good are changed, particularly when you've already made big investments in a lifelong project that was built upon those rules. There's a very strong temptation to discount the new rules as arbitrary--a product of bad faith, ignorance, false consciousness, whatever. If you do this, the project of reconciling your own desires against the rules suddenly looks very different.

The accompanying mood is a mix of frustration, nihilism, resignation, and rebelliousness. I think the best word for it is petulance. And to me it feels like the defining emotion of our political age.

VCs and reddit blackpillers are one thing. America's police officers are another. There's consensus that America experienced a significant pull-back by police in recent years. Scholars are still debating how much of this should be attributed to the pandemic versus the defunding rhetoric that reached a national crescendo in the wake of George Floyd's death. I've seen enough unhinged tweets from police union officials to believe I understand the psychology of the latter, even if its causal significance could use another regression or two.

In the last two decades, perceptions of cops have swung from a 9/11 apotheosis as heroic first responders to an ACAB consensus among the cultural vanguard that cops must surely feel is implicit in every blue-jurisdiction yard sign they pass. The rules changed, and they don't like it.

I say all of this with some sympathy. I have felt petulant as the world changed around me. I put it into words here.

Petulance is never an admirable reaction, even if the new rules really are under-justified. But I do think it's a reaction that we would all do better to expect, plan for, and understand. That doesn't mean that we can't change the rules. Many of the rules should be changed! But when we change them, we will be wise to allow the dislocated to embrace the new rules. This will frequently be annoying. I think I understand why it's satisfying and fun to respond with exasperated derision when some guy prefaces his stab at allyship with a proof-of-daughters statement. But where does that leave him? If he isn't permitted to come along, where will he go instead?

I have written before about the overwhelming power I see in the human impulse to organize ourselves in hierarchies. Who's up, who's down; who's good, who's bad. It's perfectly natural, and usually satisfying. But it's not always necessary. Often--especially in politics--it might be better to limit ourselves to declaiming what is good, rather than who.

None of this excuses petulance. It's a childish emotion that should be recognized and controlled. Allowing it to drive you into a public tantrum is embarrassing. Allowing it to drive you toward a figure like Donald Trump should be profoundly chastening. I wonder how these guys think they will be understood a decade from now. Have thought about it at all? To embrace petulance is to let go of dissonance. Not having to think quite so hard is part of the appeal.

When the time comes, I suspect they will feel pretty mortified about this period. Then again, who won't?

For now, the petulant impulse is real, and--for whatever reason--has recently become of outsize importance. Most people need a way to feel that they are good. When we can, we should make sure they have one.