By spectator standards, it was a terrible match. The Pac-Man player spent the entire game at a distance, throwing out projectile attacks, and the other player failed to find a way to approach. There were no flashy combos, no tense moments. We watched Pac-Man throw fruit for a while and then it was over. The crowd hated this, the Pac-Man player was booed the whole time. He still advanced to the next match.

I think about food a lot. A nontrivial amount of my daily willpower points are spent on not snacking too much, not overeating, and not being distracted by food in between for long enough to be productive. Some days this is harder than others.

One of the tricks that I've learned works on me is to prepare a snack for myself, then set it next to my keyboard. I don't even eat it! It just sits there! This is somehow enough to quiet my brain.

I discovered this on accident after grabbing a snack a few times and then later realizing I hadn't eaten any of it after I sat back down. It sometimes feels a bit lame. It works anyways.

In 2016, Mark Zuckerberg sent an email to some of his exec team about Snapchat analytics:

Whenever someone asks a question about Snapchat, the answer is usually that because their traffic is encrypted we have no analytics about them.

Given how quickly they're growing, it seems important to figure out a new way to get reliable analytics about them. Perhaps we need to do panels or write custom software. You should figure out how to do this.

About a week later, Project Ghostbuster allowed Facebook to install a man-in-the-middle attack that would let them intercept otherwise encrypted user data. They got the user analytics they were looking for.

We automatically fabricate and attach rules to a goal we're chasing that aren't actually part of the goal, "dignity" being a common one. This is a low agency move, you can just do undignified things.

That's not to say you should ignore your honor all the time – I don't endorse Project Ghostbuster for example – but sometimes winning looks like deciding that you don't give a shit. Notice when the rules you attach are fake and don't hobble yourself when they are.

Burnout is a pretty confusing thing made harder by our naive reactions being things like “just try harder” or “grit your teeth and push through”, which usually happen to be exactly the wrong things to do. Burnout also isn’t really just one thing, it’s more like a collection of distinct problems that are clustered by similar symptoms.

Something something intro, research context, this is what I’ve learned / synthesized blah blah blah. Read on!

Models of burnout

These are models of burnout that I’ve found particularly useful, with the reminder that these are just models with all the caveats that that comes with.

Burnout as a mental injury

Researchers can be thought of as “mental athletes” who get “mental injuries” (such as burnout) the way physical athletes get physical injuries, and we should orient towards these mental injuries in the same way we orient towards physical injuries.

I think this is by far the most useful model, because it makes it easier to actually generate a degree of self-compassion that makes you stop slamming your face into the brick wall. It is much easier to be hard on yourself while burnt out that you’re not producing “enough” legible output than it is to be hard on yourself with a twisted ankle that you aren’t keeping up with your weekly cardio goals. The way in which running on a twisted ankle is self-evidently Bad is more obvious than the way in which pushing through yet another deadline is Bad.

I’m increasingly convinced over time of how deeply this physical analogy runs. Success as an athlete is as much about managing injuries and recovery as it is about optimizing your peak performance. Managing burnout is a core research skill as much as managing recovery is a core athletic skill.

(For rehab of a physical injury, you have “active recovery”, where you aren’t back to normal but it benefits the healing process to start using the injured part again (mobility, physical therapy, light workouts). There is probably some way to triangulate active recovery for burnout here, but my guess is most people’s first attempt at doing so will still end up way too “active” to be recovery.)

Burnout as a deficit of activation energy

I like this description of willpower as “living money” and burnout as being “bankrupt”. In general, tasks have a certain activation energy to them, and we have some capacity (that varies across time) to cross activation energy thresholds. When we’re high capacity (not burnt out), moderate to high activation energy thresholds don’t feel hard to cross, but when we’re low capacity (burnt out), they feel ~impossible.

You can track progress on burnout, both in severity and recovery, by approximating how hard different tasks feel now vs. in the past. Progress on recovery should feel like making things feel easy, not about figuring out how to do things that feel hard. Sometimes life calls and you do just need to buckle up and do hard stuff, but meeting that deadline through blood, sweat, and tears is not the same as recovering from burnout. To borrow the physical analogy, you’re successfully recovering from a broken leg if walking feels easy / normal again, not if you’re managing to walk but your leg remains painful and fucked up each time you take a step.

Sources of burnout

Burnout has many mechanisms, and it can be kind of hard to pinpoint what the root cause is because sometimes they spill over into each other. For example, maybe you’re just burnt out because your sleep has been garbage and you can fix that directly. But it could also be the case that your sleep is bad because you’re anxious about some other root cause, and just fixing your sleep won’t make the other problems go away (it’ll probably help a lot anyways, but might instead be resistant to improvement). I don’t really know what to say to guide you here except that you should introspect, and if that doesn’t work, read other articles that have more to say about it and… keep trying until it does?

Possible mechanisms can broadly be physiological, psychological, or work-related. For work-related burnout, Emmett Shear identifies three sources: “permanent on-call”, “broken steering”, and “mission doubt”. I’ll add a fourth one, which is how “heavy” your work feels.

Physiological + Psychological

Physiologically: if one of [sleep, diet, exercise] isn’t solid, generally try to fix that first and see if the problem goes away.

Psychologically: are you like, generally happy? Do you get enough social interaction? Are there major sources of psychological stress (acute or not)? One of my minor spells of burnout turned out to be solved by just actually leaving the research dungeon and spending more time with friends.

Broken steering / responsiveness

People like being able to poke at the world and have the world react (see also this blog post on responsiveness). It feels bad when this doesn’t happen. It feels really bad when this doesn’t happen for a long time. This is pretty closely related to the “living money” / willpower model; none of your bids to spend energy are sufficiently rewarded with some kind of response. 

I think this is one of the hardest sources to deal with in research, because the steady state of challenging research work is to be stuck and muddling through things. That just kind of sucks! To some degree I think you just have to learn to be ok with this (but there is also a lot that you can do to improve the responsiveness of your work, and if it’s really not responsive that can be a sign that you’re chasing the wrong things or just pressing buttons hoping something happens).

As a sidebar, a friend was recently talking to me about the importance of being able to be bored. Several years ago, I gave TikTok a try. I lasted about a month before I became nearly incapable of focusing on work because it blew up my mental baseline for what a reasonable reward loop with the world was. I quickly returned to my previous baseline after deleting TikTok; I think this probably implies a More Dakka thing exists here (like a digital detox), but I haven’t tried it yet.

Permanent on-call

I’m actually not exactly sure how to map this onto research; research often has stereotypically difficult work-life balance, but some of the ways this happens can be bad, while others are fine or at least instrumental. Crunch time before a research deadline is a form of this, as is being the single person that knows how to do X and having lots of people that need you to do X for them all the time. 

From another angle, it might be when a research problem is really consuming you and it just follows you around everywhere. This is actually kind of ideal in some ways; a lot of conceptual research progress happens as background processing, but in order for background processing to happen, it has to occupy a lot of your background thoughts.

Mission doubt

I would guess that this one is relatively less problematic in research work? It’s probably less common for people to be forced into certain research roles through life circumstances than it is for other types of jobs, so it’s more likely that you have some kind of investment in the mission you’ve chosen. There’s also comparatively more freedom to wiggle your work in a direction that you care about. 

One specific way in which I could see this manifesting in research is believing in some grand vision, but not really believing in how some sub-mission connects to it (sort of like broken steering but at a different scale). You might just keep working on the sub-mission due to inertia and not really having a real-feeling outside option. If this is the case, I suggest that you stare into the abyss.

My coworkers sometimes talk about having radical, irrational optimism in your work – sometimes you have to channel a little crazy to convince yourself to do the hard, uncertain work. It’s not good to always maintain that state, but if you can’t muster it up even a little bit, that seems Bad.

Anyways, if this is your problem with your work, stop working on it! As a researcher you almost certainly have outside options, including spending your energy finding outside options.

Lightness and heaviness

Lightness roughly points in the direction of how fun or playful or generally low-stakes the work feels. This is like that whole attachment thing of holding things lightly. 

• Hackathons / game jams have typically felt very light to me, and my recollection is feeling pretty good afterwards despite them being extremely concentrated periods of effort. Here, the stakes are fairly low and the work (and its impacts) are well-scoped. There’s an atmosphere of (sometimes type 2) fun.
• I’ve also felt this when doing supporting work for some project that feels valuable but isn’t addressing the crux. The project will survive without me, and someone competent that I trust is on the ball. 
• A former mentor of mine would save a certain kind of toy problem for her students to give to them when they seemed stressed out by the core work of their thesis. The toy problems weren’t meant to be publishable, they were just little pieces of mathematical candy to remind her students that math could still be light and fun.

Heaviness roughly points in the direction of high-stakes. 

• You need this to work in order to get or keep your job. 
• It feels like you’re singularly responsible for the crux of the project and other people are counting on you. 
• Your self-worth depends on producing legible outputs.
• If you have to champion your work to other people, it can feel like you have to believe in it enough for several people, and every outcome carries the weight of needing to justify further investment.

These things produce significant extra mental fatigue / friction, even though it might not feel like that when you’re at high capacity. There is always some irreducible degree of difficulty, but to use the physical analogy, it’s like running with a weighted vest vs. not.

Early warning signs

In sports, people say something like “listen to your body” in order to avoid injuries. I didn’t find this particularly helpful until I got injured a few times and actually developed a sense of the limits of my body. I think if you pay close attention when that happens, you actually can learn a fairly fine-grained sense of what it feels like to push something close to its limit and how bad an injury really is.

I think getting better at noticing burnout is like this. That sucks a little because it sort of means that you just need to experience some burnout in order to have any kind of training data, but it’s definitely possible to have high sample efficiency (if there is a way to make people good at this without them ever burning out, please let me know). I suspect each person’s warning signs will look a bit different depending both on their personality and the type of burnout they’re facing. My most common warnings are gradual increases in revenge sleep procrastination, playing more video games, letting chores pile up more, working out less, and eating worse. 

It’s tricky to know the correct response because these signs don’t always indicate a serious problem, and when they do, sometimes you can just grind a little harder (but see the next section for a warning about this) and other times you can’t. This is a skill that you can empirically make progress on though: in grad school I burned out horribly and didn’t do any substantive research for over a year, and these days I’ve been able to intervene quickly and (mostly) prevent long term work disruptions. Nearly all of that progress factors through “listening to my mind” (see also: Noticing).

Coping mechanisms and solutions

My biggest warning here by far is to be clear-headed about which things are coping mechanisms and which are robust, long-term solutions. As a general rule, things that feel like “action scaffolding” that push you in the direction of making short-term progress tend to be coping mechanisms. If you just do the action scaffolding stuff, you might get lucky and outrun your burnout (if, for example, you were grinding out job applications and you finally get an offer), but you might also just get more burnt out until those things don’t work anymore either. This is a very dangerous fire to play with; extra marginal burnout can have extremely outsized impacts. Necessary recovery time can grow surprisingly quickly, and I know of at least one extreme case where someone had to quit an entire area of research permanently.

A (non-exhaustive) list of examples in your action space, (very) coarsely ordered from cope-y to solution-y (don’t read too much into the ordering though, that will vary from person to person too): 

• Take a break (a short vacation, reduced hours, etc).
• Body doubling / coworking; this doesn’t work for everyone, but I find that this robustly lowers the activation energy costs and reduces distractions.
• Try to manufacture some fun in your work. Maybe set aside some time to chase down a sidequest or organize a casual hackathon. Do things that feel easy and don’t do things that feel hard, try to follow the gradients for a while.
• If you’re stressed about other people’s perceptions or of letting other people down, talk to them about it. Sometimes bearing the secret accounts for a large portion of the weight, and it suddenly becomes manageable once you say it out loud to someone.
• In the case of heaviness-from-self-worth, referring to past evidence that you’re worthy. It felt like a huge weight had been lifted from my shoulders during my PhD once I’d finally published a paper I was proud of, and I really milked that as a persistent reminder that I could be a competent researcher. Ideally that wouldn’t be necessary, but it did turn out to be a fairly robust coping mechanism and buys me time to have a healthier attachment to my work. If you’re a very young researcher, maybe you don’t have papers, but surely you’ve ever done something you were proud of in life, let yourself feel a little good about it.
• Fix your feedback cycles in your work; rearrange things or figure out how to actually feel like the world squishes appropriately when you poke it.
• Focus on process: if you’re persistently learning and improving, the power of compounding investment makes it inevitable that impressive results will eventually just fall out.
• Just like, generally being kinder to yourself? Research is a domain where there is always an ~infinite amount of useful work that can be done, and it is also easy to be ~infinitely hard on yourself for not accomplishing more or for not being good enough. This is particularly hard when you’re really burnt out and need to scale down your productivity, because it feels like you’re going in the wrong direction (this is where I find the physical injury analogy most useful personally).

These are just generic examples that tend to work across broad subsets of root causes for burnout, and there’s a decent chance you have to do something more bespoke to solve your problems. In any case, lots of people have had severe burnout and made it through to the other side. It is empirically a tractable problem even if it doesn’t typically feel like it from the inside.

Thanks for discussion and feedback: Andy, Cecilia, Maggie, Maggie

Some ideas were generated in collaboration with Claude but generally spot checked. This post was also sanity-checked by a friend who works in epidemiology. I feel ok about the ideas presented but would not be surprised if someone with more expertise has a significantly different conclusion. Any mistakes are mine.

I went into this exercise with a prior of “I’ve been hearing about bird flu for years, and it’s always been nothing, it’s probably nothing again this time.” The main upshot is that I walked away from the exercise thinking “I don’t know if this is going to be something or not.” As far as updates go, that seems directionally bad. My current orientation towards this is something like “watch and wait, and spend appropriately more effort on this if / when certain milestones happen (and also make some trades).”

What’s different this year?

One main thing that seems to generate a heightened level of ongoing risk is sustained infections in dairy cow populations. This is already bad because it’s mammal-to-mammal transmission in a population that hasn’t historically had problems with bird flu, and it would be worse if it becomes endemic in farmed cows. As long as it is sustaining infections in farmed cows, H5N1 has a convenient breeding ground for mutations that is:

• In a high density population
• Year round (bird flu is seasonal among birds)
• In regular contact with humans

The last point gets a bonus with the human flu season coming up. If a dairy worker gets sick with both bird flu and human flu simultaneously, the two strains might share their genetic information and mutate into a pandemic-worthy strain.

Another thing being talked about is the teen in Canada who got sick from an unknown source. I suspect that at least some of the attention here is because scary things generate ad revenue, but there is also evidence that a wild strain of H5N1 has a mutation that improves binding to human receptors.

These seem to represent two distinct threat vectors to me; the Canadian teen seems to be very sick, whereas dairy workers have generally had mild symptoms (although in the case of the 1918 Spanish flu (also avian), younger people had higher fatality rates). A jump from birds directly to humans would be mostly independent of whatever is happening in dairy farms, and vice versa.

I don’t have a concrete prediction here, but I notice that Polymarket being at 14% for a pandemic by August 2025 doesn’t feel wild to me. I would tentatively assign more probability to something coming from the cow situation than directly from birds.

What to watch for

As I said above, my plan is to watch and wait, but what to watch for? There are a few key things IMO:

• A (sustained) jump to pigs. Pigs are traditionally considered a mixing vessel for flu viruses, since they are apparently susceptible to them and are able to host avian, human, and swine flus. This is how the 2009 H1N1 pandemic mutations happened. There has been one confirmed case in a pig, but that was over a month ago now with no more news that I've seen.
• A mutation to airborne transmission among cows. Seems self-explanatory, indicates that the virus is undergoing significant mutations in the cow population. Also airborne bad.
• Whether it continues to infect cows / becomes endemic (see previous section on why this creates elevated risk).
• A lot more cases like the teen in Canada, with evidence that the suspected mutation that makes it easier to bind to human receptors is common in wild H5N1.
• Of course the big one is human to human transmission, with each of the previous points being plausible stepping stones.

Something speculative is watching how pharma companies are reacting. As I understand it, to some crude approximation pharma companies take a hits-based approach to investing in drugs. In the venerated tradition of pulling numbers out of my ass, if O($100M) is an appropriate amount to spend on pre-empting a pandemic, and they expect to make O($1B) on a hit, then this indicates an O(10%) belief that some kind of pandemic happens in the near future? 

The US government has paid Moderna $176M to work on H5N1 vaccines (which impacts the above speculation, since this de-risks the investment for Moderna), Pfizer is working on it as well (though I didn’t find anything indicating a federal grant), and the government has also paid $72M to manufacturers to make existing vaccines ready to use.

Vaccine response

The US government has a strategic reserve of bird flu vaccines. Given that pharma companies are also preemptively working on a vaccine, what could we expect in terms of immediate response and an eventual bespoke vaccine for the pandemic strain?

The strategic reserve of bird flu vaccines is small relative to the population and seems basically intended for emergency / essential workers. The above link about paying $72M to manufacturers mentions that this would double the strategic reserve from 5 million to 10 million doses, which is already far from the total US population before accounting for multiple doses per person. Furthermore, these vaccines are based on known strains and may only provide limited protection against a new pandemic strain.

So we probably aren’t going to get an immediate bail out of a pandemic. How long would it take to develop and ramp up a vaccine for the full population? For reference, widespread availability of the covid vaccine took around a year, while H1N1 took around 6 months. Why was H1N1 so much faster? Basically we’re really good at making flu vaccines because we make them every year and there are mature platforms for this, whereas the coronavirus vaccine used fairly novel technology (mRNA vaccines). 

So we could plausibly move as fast as with H1N1, but here are some serially bound steps that need to happen to make a pandemic-strain-specific vaccine, so much faster than H1N1 sounds hard. Something like 4-6 months seems plausible to me, not accounting for any political friction.

Sociopolitical response

We didn’t spend much time thinking about the political response and I don’t super feel qualified to shoot from the hip here, but some potentially important factors to note:

• Pandemic fatigue seems real, and it might be hard to coordinate effectively a second time.
• AFAIK RFK is pretty anti-vax (and is the nominee for HHS) and vaccines are super politicized now. Not really sure how to game this out – if a hypothetical pandemic was bad enough, would it moderate everyone’s views?
• Individual reactions might be modeled by a bimodal distribution? Seems plausible to me that some people overreact while others underreact.

How bad would a pandemic actually be?

Wikipedia has a page on past influenza pandemics. I was somewhat surprised to learn that the seasonal flu already kills hundreds of thousands of people every year (!), and is in fact typically more lethal each year than 2009 H1N1 was. I don’t really understand the HXNY labeling, but it seems like H1N1-like strains have caused three pandemics, each time less bad than the last, which makes sense if you no longer have a naive population.

Our base case might look more like H2N2 or H3N2 then. The case fatality rate for these is very low compared to the observed fatality rate in some H5N1 strains (~50%?), but the H5N1 strain that is infecting dairy cows and workers hasn’t killed anyone yet and seems much more mild than a coin-flip death sentence (and an effective pandemic strain is unlikely to have a 50% fatality rate – being too lethal causes a virus to burn out before spreading widely, in addition to motivating a stronger response). H2N2 and H3N2 happened before we figured out how to surveil these diseases before a jump to humans, so it’s not clear whether these looked like H5N1 does today or if they were much milder even before the jump. 

For another point of reference, covid had an original reproduction number of around 3 and a case fatality rate of around 1% (on top of other “lucky” features, like being contagious during a long asymptomatic incubation period).

We also already have antivirals for the flu like Tamiflu, in contrast to covid where it took a long time to develop things like Paxlovid. There’s no guarantee that it would be super effective, but so far it seems like it has some effect and is currently being used to treat human H5N1 cases.

My best guess is that we should expect an H5N1 pandemic to look much more like a (possibly more lethal) H2N2 or H3N2 pandemic than a covid pandemic, though we shouldn’t rule out something worse. This would be bad, but not incredibly bad. 

What else?

I’d be interested in hearing more takes and especially places where I might have gotten something wrong or missed something important. My guess is that there are also trades that are more profitable than buying Yes shares on Polymarket if they hit, though I'm not sure about publicly speculating about this.

Thanks for ideas, discussions, feedback: Maggie, James, Linda

This is a lightly edited write up that I initially made for myself and for the team at Timaeus about my experience and takeaways. It's not super polished, but seems better to not clean up and publish than to not clean up and not publish.

What was the workshop about?

Ray is interested in metacognitive improvements. A rough definition of these are: skills or strategies you can learn that make you better able to understand your cognitive process and to influence it in ways that make you more effective as a person. Some examples might include:

• Plan / decision making
• Noticing
• Prediction / calibration training

This workshop was specifically about being better at making plans. The terminal goal was to help people who work in deeply confusing and long-feedback loop areas (particularly x-risk reduction). The target audience of this workshop was people who are in a position to be making meaningful plans in their job (i.e., they determine most of their own work and/or the work of others). This workshop was probably most helpful for people who are somewhat familiar with how to make good plans

Why plans?

• It's tractable: you can plausibly learn at least one concrete thing in a weekend that will make you better at making plans.
• It's understandable: you can just follow the procedure of a concrete thing without needing to have inscrutable mental motions described to you.
• It's impactful: the absolute best plan you can follow (which is might not be the best plan you can make) can often be >>10x better than your default plan, even if your current plan is good.
  • I think this sounds a little crazy until you think about it? But someone could be doing a plan that's not even net positive. Someone could be working on entirely the wrong thing, and there could be a totally different research agenda or career that has 10x or 100x impact. Someone could just be chasing down the wrong medium-time horizon targets for their grand strategy that slows down the whole thing by massive OOM factors.
  • I personally have tons of examples where I retroactively saw the path to 10x better returns if I hadn't missed something that I could have figured out if I'd thought way harder or more clearly to begin with.
  • These results compound: a plan might result in >>10x returns because it unlocks resources for even better plans.
  • It's unfortunately super hard to know ahead of time which plan is >>10x better, but the long-tailed nature of plans means that if you become only a few % better at plan making, you might actually get really outsized returns from it over a long time horizon.
• This basically turned out to be a bullet point outline of my theory of value for this type of work. FWIW I’m extremely deliberate-practice-pilled, but I think I would judge the value of this workshop to be O($1-10k), and my personal speculation is that it’s totally possible for this to have value on O($100k+) if one tried really hard to adopt the tools / mentality from the workshop (and had the leverage to use them, though I guess the sky's the limit if you have enough leverage).
• I think due to time constraints, this workshop chose to take the shadowmoth approach of "throw people in the deep end and make them learn to swim or not". I don’t think this was a crazy choice, but I think there are more gentle ways to introduce ideas before shadowmothing someone? I’ll speculate on this later. (I did not end up speculating on it)

Examples from the workshop

(Note: I would recommend not trying to solve or play around with these if you think there is any chance you’d like to try out some of the workshop exercises in a formal setting)

Here are some examples of exercises that we did (may not be in order):

• [Redacted]
• One-shot Baba Is You
  • This was my favorite exercise, and the second time that I’d done it. I learned a few useful things from this and from listening to other people approach the problem, which I’ll elaborate on later.
  • You are given a Baba Is You puzzle to solve, but you are not allowed to interact with the puzzle in any way. This is what makes it particularly hard, because Baba Is You is a notoriously unintuitive and difficult puzzle game, and usually the process of solving puzzles involves fiddling with bits and collecting little pieces of evidence and finding out how rules interact through empirics. In this exercise, you don’t get any of that, you just get to Think Very Hard and try to make a plan that solves the puzzle on the first try ("one-shotting" it).
  • A motivating theme here is "pretend like empirics / experiments are really hard – every plan you make takes weeks or months of engineering time, but time spent planning up front is cheaper"
    • Unfortunately, this is kind of a hard mindset to embody, and it seems like a common takeaway is "you can’t think about anything successfully ever" and "empirics are incredibly OP" (I think that cheap empirics are incredibly OP, but that’s different)
  • Another motivating theme is "what does it feel like to feel surprised and what does it feel like to have gained an insight?"
• Thinking Physics
  • This is also a good exercise I think (and one that I've done on my own before). The idea is that you get a puzzle from Thinking Physics and you just sit and try to solve it.
  • A motivating theme here is "what does it feel like to have a fuzzy idea of why something is true vs have a very crisp idea of why something is true, in the sense that the world must be deeply fucked somehow if the crisp idea is true, but the world might be totally fine if the fuzzy thing is not true?"
  • Another motivating theme is that of "noticing insights", similar to with Baba Is You.
• Some time was spent on freewriting exercises about plans that you actually have IRL, to try and immediately apply lessons from the workshop to IRL problems. This was a significant portion of the time of the workshop, but I won’t get into those much.

What lessons were useful to take away?

General skills that are OP (not all from the workshop, including my own takes in here)

(These are also skills that would make the workshop much more useful for you; having them doesn't mean the workshop won't help, it means you can use it to build on them)

• Being well calibrated
• Noticing (I feel like metacognition is ~impossible if you have zero noticing (most people don’t have literally zero noticing even if they’ve never tried to train it))
• Being unafraid of looking stupid / not having an ego attached to your performance
  • This is a problem I have! I notice that I get emotionally stressed / anxious about not being competitive in solving the exercises compared to other participants. Noticing / acknowledging it seems to help a little.
• Patience / taking breaks
• Taking naps / caring for physiological health
• Embodying skills as a lifestyle.
  • Being able to get into the mindset where it "counts" even in trivial situations. If you can’t live the values when it doesn’t matter, you won’t do it when it does matter

Specific skills from the workshop that are OP

• Live logging (similar to having a research log): basically writing down all of the thoughts you’re having while thinking.
  • Related to white boxing (which is trying to un-black-box your thought process as much as possible). Live logging is trying to white box yourself, to yourself.
  • This is a clever ~shortcut to Noticing, for the purposes of this workshop (so the workshop doesn't have Noticing as a prereq)
• Metastrategic brainstorming: taking a step back and trying to think of radically different approaches to solving the problem
  • Ray thinks this is like, 40-60% of the skill of making better plans (generating them in the first place)
  • Basically, you notice that you’re stuck, and you see if there’s a different type of thing you can try or some intervention that you can make that will shortcut your process. Some examples of different categories of strategies you might brainstorm:
    • Think about the generating process of the problem
      • In places where the problem has contacted reality elsewhere, what does that imply about it?
    • Brute force + heuristics
      • Is the space of options small enough that you can just do this?
    • Babble
      • Maximal explore, minimal exploit, or something like that
    • Instead of building a wall, build a brick
      • What’s the smallest thing you could do that would make progress or tell you something new?
    • Combinatorially examine your assumptions + action / option space
      • Sometimes you will miss something that you were overlooking before (e.g. because your brain automatically pruned it as an option)
    • Identify OODA loops or make some if you can’t find any
    • Red team / construct an impossibility proof (of a strategy)
      • Common approach in math, but if you try to prove that your idea is impossible, you might notice something interesting about it
      • A related personal takeaway that I noticed here is that this can be a shortcut way to mitigate rabbit-holing on a solution (you pretend like the solution is impossible even if you don’t think it is – if you lived in that world, what would your next guess be?). I found that at one point I thought that a solution was the only clear solution, then the game told me I was wrong, then I immediately found the right solution once I had let go of the previous one. So maybe if you could just really make yourself believe that your favorite solution is wrong, on the level of receiving hard evidence without having actually received the evidence?
    • Take a nap / break / walk / eat something
  • One participant was skeptical of this, got stuck on something, tried it, and then almost immediately became unstuck.
• Shortening feedback loops / “How could I have thought that faster?”
  • This is tied to live logging, since having a log of your thoughts helps a lot, but going through your actual process for solving a problem and looking for the earliest points where you could have made an insight (instead of when you actually made the insight) and what you could have thought to have made that insight sooner
  • This is like, reinforcement training / conditioning for your brain
• Having 3 plans, 2 frames, and a crux
  • I’m not sure I’m super sold on this, but the idea is something like “can you try to generate multiple plans and multiple perspectives, and ideally a crux or two about what would make you choose one perspective or plan over another” to avoid rabbit-holing (avoiding rabbit-holing is a big underlying theme, I’m noticing)
• Some other misc things that are small or known elsewhere or weren’t a big focus of the workshop
  • Quantified intuitions about “good” – can you figure out how to measure two plans in the same “units”? E.g. at one point Ray chose between “should I help people in the workshop during the next (open working) session or should I spend some time working on iterations and reflections for improving the workshop?”
    • An example of a shared unit here is “in expectation, the amount of (weighted?) improved researcher-hours on x-risk” or something like that
  • Figuring out your cruxes and ideally internally double-cruxing yourself (between two plans)
  • Framing things as ~OODA loops
    • I think this is a useful tool if you haven’t run into it before
    • We did an exercise where we tried to frame things that we do IRL as OODA loops and identify places where we aren’t using them but could be

What did I personally take away?

This is a bit more stream of consciousness / thinking out loud.

This phrase kept getting offhandedly repeated throughout the workshop when Ray was giving examples, and it’s stuck in my head. Something like, “I would just be in the middle of doing something and I would wake up and become sentient and look around me and be like, what am I doing?”

Something about the “wake up and become sentient” thing feels like a really major core of the workshop to me. I’ve spent a bunch of time (diffuse over many years) working on or thinking about deliberate practice and decision making and stuff, but I noticed that a lot of my patterns and intuitions here have kind of become a bit too subconscious, and I’ve forgotten that they’re a thing that I can just look at and continue to polish (I’ve somehow gotten too busy to remember that deliberate practice is a thing that I should still be doing everywhere).

For example, in the Baba Is You exercise, I ended up doing 3 different levels (across two sessions), and it wasn’t until I’d thought about it a bunch afterwards that I realized I had the same type of blind spot in all three cases (my brain is a bit too eager to prune things that are above some threshold of "I’m sure the world works this way"). I think this is a pretty generalizable pattern to notice, and it required "waking up" + live logging + reflecting about the process to notice it in the game setting.

Another big thing in the workshop for me was how important it is to

1. Actually do these as exercises and not just read about them
2. Actually do these as exercises and not just remember that I’d learned these lessons once upon a time (thinking of like, muscles that are not used or stretched in a long time become stiff or atrophy)
3. Actually think about calibration and practice noticing more often, and how much these things have helped me in the past and that I can just keep improving these things until I see diminishing returns

This thing wasn’t really part of the workshop (the workshop was not focused on execution of plans), but it is something the workshop reminded me of: I can in fact just spend time thinking about how to deliberately practice execution as a research lead. This is something that just seems obvious to do.

TLDR the thing that maybe had the biggest impact was the workshop as a catalyst for a meta-level "waking up and becoming sentient" about the fact that I can "wake up and become sentient" about object level things. (not that I have just literally been autopiloting, but there are degrees to getting out of your head about things / switching to manual control). Besides that, two object level patterns:

• I still get emotionally agitated sometimes when I feel like I’m stupid because I can’t solve something
• I prune a little too much and maybe explore a bit too little. I probably trust my subconscious process a little too much and should more often have some doubts about the end result (but also don’t want to overcorrect on this)

And some skills that I’m interested in practicing more and / or trying to use everywhere to internalize them:

• Noticing
• Metastrategic Brainstorming
• Live logging (way more than I already do)

The latter two are the kind of thing where my theory of internalizing them is based on how I got myself to use ChatGPT originally, which was: shoehorn it into everything for a few weeks, then gradually prune use cases to what actually feels useful.

I also now have the idea of a “Deliberate Practice Monastery” tattooed on my brain and hear its beautiful siren call. I think there’s a real argument to be made that this work is extremely important and in its idealized, full-potential form, can make our best x-risk researchers significantly better.

I also like a rule of thumb that Ray mentioned during the workshop, which was "spend ~10% of your time on meta".

Other misc thoughts

• I found myself surprised at how not deliberate-practice-pilled a lot of people are. This type of practice and mindset feel like a natural companion to rationality, but it seems like very few people have spent significant time or attention on tuning these things.
  • It feels like the fruit is hanging so low here, like I genuinely believe there are just Ideas that are not impossible tacit knowledge that can be turned into Words that you can just Say to people and if they Listen they can just immediately be more competent. Perhaps this workshop didn't yet have the perfect Words, but I think such incantations are definitely within reach.
  • It feels kind of insane to me that Ray is fighting to prove that this is useful enough to spend more effort on. I feel like this is just self evidently important or something, I don't know if I'm missing something or everyone else is?
• It is hard to come up with good exercises for this type of workshop, and I’m impressed at what Ray has done so far
• I think case studies would be a really useful addition. My suggestion here was to include a few demonstrations (by Ray) of the proposed tools, but my wishlist would include examples in science or history where a plan making process was documented / legible and resulted in obviously better plans.
• I still mostly think planning is essential and plans are meaningless or whatever the quote is.
• Cheap empirics are OP as hell, you take things for granted until they’re missing.
• All else being equal, velocity is OP. If you can do things 10% faster, you learn 10% faster too, everything compounds and is more than 10% better.

Concrete value in my day-to-day research

I'm redacting most of this section b/c it's high context and also maybe private, I'm not sure and don't want to think too hard about it. The thrust of this section was something like:

• Looking back, if I had really, with my whole ass, internalized and practiced some of these lessons at the start of this year (maybe more than what the workshop alone would give? but definitely something achievable with the workshop as a catalyst), I could easily imagine saving a few weeks here or there, adding up to potentially months of speed up in my productivity (in the original document, there were four concrete examples).
• In practice, I would guess that over the past few weeks, trying hard to apply things like metastrategic brainstorming and other forms of deliberate / purposeful practice have already saved time measured in days (in expectation, over the next few months).

Go to the next one!

Ray is hosting another workshop this weekend (Oct 25-27). Go do it. Actually doing the exercises is miles better than just reading about them (and going home afterwards and continuing to practice is miles better than that).

I am specifically aiming to cover advice on getting a job in empirically-leaning technical research (interp, evals, red-teaming, oversight, etc) for new or aspiring researchers without being overly specific about the field of research – I’ll try to be more agnostic than something like Neel Nanda’s mechinterp quickstart guide but more specific than the wealth of career advice that already exists but that applies to ~any career. This also has some overlap with this excellent list of tips from Ethan Perez but is aimed a bit earlier in the funnel.

This advice is of course only from my perspective and background, which is that I did a PhD in combinatorics, worked as a software engineer at startups for a couple of years, did the AI Futures Fellowship, and now work at Timaeus as the research lead for our language model track. In particular, my experience is limited to smaller organizations, so “researcher” means some blend of research engineer and research scientist rather than strictly one or the other.

Views are my own and don’t represent Timaeus and so on.

Requisite skills

What kind of general research skills do I need?

There’s a lot of tacit knowledge here, so most of what I can offer is more about the research process. Items on this list aren’t necessarily things you’re expected to just have all of or otherwise pick up immediately, but they’re much easier to describe than e.g. research taste. These items are in no particular order:

• Theory of change at all levels. Yes, yes, theories of change, they’re great. But theories of change are most often explicitly spoken of at the highest levels: how is research agenda X going to fix all our problems? Really, it’s theories of change all the way down. The experiment you’re running today should have some theory of change for how you understand the project you’re working on. Maybe it’s really answering some question about a sub-problem that’s blocking you. Your broader project should have some theory of change for your research agenda, even though it probably isn’t solving it outright. If you can’t trace up the stack why the thing you’re doing day to day matters for your ultimate research ambitions, it’s a warning flag that you’re just spinning your wheels.
• Be ok with being stuck. From a coarse resolution, being stuck is a very common steady state to be in. This can be incredibly frustrating, especially if you feel external pressure from feeling that you’re not meeting whatever expectations you think others have or if your time or money is running out (see also below, on managing burnout). Things that might help for a new researcher are to have a mentor (if you don’t have access to a human, frontier LLMs are (un)surprisingly good!) that can reassure you that your rate of progress is fine and to be more fine-grained about what progress means. If your experiment failed but you learned something new, that’s progress!
• Quickly prune bad ideas. Always look for cheap, fast ways to de-risk investing time (and compute) into ideas. If the thing you’re doing is really involved, look for additional intermediates as you go that can disqualify it as a direction.
• Communication. If you’re collaborating with others, they should have some idea of what you’re doing and why you’re doing it, and your results should be clearly and quickly communicated. Good communication habits are kind of talked about to death, so I won’t get into them too much here.
• Write a lot. Writing is thinking. I can’t count the number of times that I felt confused about something and the answer came while writing it down as a question to my collaborators, or the number of new research threads that have come to mind while writing a note to myself or others.
• Be organized. Figure out some kind of system that works for you to organize your results. When you’re immersed in a research problem, it can feel deceptively easy to just keep all the context of your work and the various scattered places information is stored in your head. I currently keep a personal research log^[1] in a Google doc (also visible to my collaborators) that I write entries into throughout the lifetime of a project. The level of detail I aim for is to be able to revisit an entry or a plot months later and to be able to recall the finer points from there – on average this actually happens in practice about once a week and has saved me a great deal of headache.
• Continually aim just beyond your range. Terry Tao has a ton of great career advice, much of which is transferable to other fields of research beyond math. Research is a skill, and like many other skills, you don’t grow by just doing the same things in your comfort zone over and over.
• Make your mental models legible. It’s really hard to help someone who doesn’t make it easy to help them! There’s a ton of things that feel embarrassing to share or ask, and this is often a signal that you should talk about it! But it’s also important to communicate things that you don’t feel embarrassed about. You might be operating off of subtly bad heuristics, and someone with more experience can only correct you if you either say things in a way that reveals the heuristic or if you do a thing that reveals it instead (which is often more costly).
• Manage burnout. The framing that I find the most helpful with burnout is to think of it as a mental overuse injury, and the steps to recover look a lot like dealing with a physical overuse injury. Do all the usual healthy things (sleep enough, eat well, exercise) and ease into active recovery, which emphatically does not look like taking a few days off and then firing on all cylinders again. Much like physical overuse injuries, it’s possible to notice signs ahead of time and to take preventative steps earlier. This is much easier after going through the process of burning out once or twice. For example, I notice that I start doing things like snacking more, procrastinating sleep, and playing more video games. These things happen well before there’s any noticeable impact on my productivity. Finally, burning out is not a judgment of your research ability – extremely competent researchers still have to manage burnout, just as professional athletes still have to manage physical injuries.

What level of general programming skills do I need?

There is a meaningful difference between the programming skills that you typically need to be effective at your job and the skills that will let you get a job. I’m sympathetic to the view that the job search is inefficient / unfair and that it doesn’t really test you on the skills that you actually use day to day. It’s still unlikely that things like LeetCode are going to go away. A core argument in their favor is that there’s highly asymmetric information between the interviewer and interviewee and that the interviewee has to credibly signal their competence in a relatively low bandwidth way. False negatives are generally much less costly than false positives in the hiring process, and LeetCode style interview questions are skewed heavily towards false negatives.

Stepping down from the soapbox, the table stakes for passing technical screens are knowing basic data structures and algorithms and being able to answer interview-style coding questions. I personally used MIT’s free online lectures, but there’s an embarrassment of riches out there. I’ve heard Aditya Bhargava’s Grokking Algorithms independently recommended several times. Once you have the basic concepts, do LeetCode problems until you can reliably solve LeetCode mediums in under 30 minutes or so. It can be worth investing more time than this, but IME there are diminishing returns past this point.

You might also consider trying to create some small open source project that you can point to, which can be either AI safety related or not. A simple example would be a weekend hackathon project that you put on your CV and your personal GitHub page that prospective employers can skim through (which you should have, and which you should put some minimal level of effort into making look nice). If you don’t have a personal GitHub page with lots of past work on it (I don’t, all of my previous engineering work has been private IP, but do as I say, not as I do), at least try to have a personal website to help you stand out (mine is here, and I was later told that one of my blog posts was fairly influential in my hiring decision).

Once you’re on the job, there’s an enormous number of skills you need to eventually have. I won’t try to list all of them here, and I think many lessons here are better internalized by making the mistake that teaches them. One theme that I’ll emphasize though is to be fast if nothing else. If you’re stuck, figure out how to get moving again – read the documentation, read the source code, read the error messages. Don’t let your eyes just gloss over when you run into a roadblock that doesn’t have a quick solution on Stack Overflow. If you’re already moving, think about ways to move faster (for the same amount of effort). All else being equal, if you’re doing things 10% faster, you’re 10% more productive. It also means you’re making more mistakes, but that’s an opportunity to learn 10% faster too :)

What level of AI/ML experience do I need?

Most empirical work happens with LLMs these days, so this mostly means familiarity with them. AI Safety Fundamentals is a good starting point for getting a high level sense of what kinds of technical research are done. If you want to get your hands dirty, then the aforementioned mechinterp quickstart guide is probably as good a starting point as any, and for non-interp roles you probably don’t need to go through the whole thing. ARENA is also commonly recommended.

Beyond this, your area of interest probably has its own introductory materials (such as sequences or articles on LessWrong) that you can read, and there might be lists of bite-sized open problems that you can start working on.

Should I upskill?

I feel like people generally overestimate how much they should upskill. Sometimes it’s necessary – if you don’t know how to program and you want to do technical research, you’d better spend some time fixing that. But I think more often than not, spending 3-6 months just “upskilling” isn’t too efficient.

If you want to do research, consider just taking the shortest path of actually working on a research project. There are tons of accessible problems out there that you can just start working on in like, the next 30 minutes. Of course you’ll run into things you don’t know, but then you’ll know what you need to learn instead of spending months over-studying, plus you have a project to point to when you’re asking someone for a job.

Should I do a PhD?

Getting a PhD seems to me like a special case of upskilling. I used to feel more strongly that it was generally a bad idea for impact unless you also want to do a PhD for other reasons, but currently I think it’s unclear and depends on many personal factors. Because the decision is so context-dependent, it’s a bit out of scope for this post to dive into, but there are some more focused posts with good discussion elsewhere. I think my own experience was very positive for me (even if it wasn’t clear that was the case at the time), but it also had an unusual amount of slack for a PhD.

Actually getting a job

What are some concrete steps I can take?

Here’s a list of incremental steps you can take to go from no experience to having a job. Depending on your background and how comfortable you feel, you might skip some of these or re-order them. As a general note, I don't recommend that you try to do everything listed in depth. I'm trying not to leave huge gaps here, but you can and should try to skip forward aggressively, and you'll probably find that you're ready for later steps much sooner than you think you are (see also upskilling above).

• Learn to code (see above)
• If your area of interest is low-level (involves digging into model internals), learn the basics of linear algebra and run through some quick tutorials on implementing some model components from scratch (you can just search “implement X from scratch” and a billion relevant Medium articles will pop up)
• Find some ML paper that has a walkthrough of implementing it. It doesn’t have to be in your area of interest or even AI safety related at all. There are varying degrees of hand-holding (which isn’t a bad thing at this stage). Here is a particularly in-depth guide that eventually implements the Stable Diffusion algorithm in part 2, but it might be a bit overkill. You can probably find plenty of other resources online through a quick web search, e.g. this HN thread.
• Learn a little bit about your area of interest (or some area of interest; you don’t have to know right this second what field you’re going to contribute to forever!)
• Find a paper in your area of interest that looks tractable to implement and try to implement it on your own. If you have trouble with this step, try finding one that has a paper walkthrough or an implementation somewhere on GitHub that you can peek at when you get stuck.
• Find an open problem in your area of interest that looks like it could be done in a weekend. It’ll probably take more than a weekend, but that’s ok. Work on that open problem for a while.
• If you get some interesting results, that’s great! If you don’t, it’s also ok. You can shop around for a bit and try out other problems. Once you’ve committed to thinking about a problem though, give it a real shot before moving on to something else. In the long run, it can be helpful to have a handful of bigger problems that you rotate through, but for now, just stick to one at a time.
• Ideally you now have some experience with small open problems. This is where legibility becomes important – try and write up your results and host your code in a public GitHub repo.
• Now you have something to take to other people to show that you’re capable of doing the work that you’re asking someone to pay you to do, so go and take this to other people. Start applying for fellowships, grants, and jobs. While you’re applying, continue working on things and building up your independent research experience.
• It might take a while before you get a bite. A job would be nice at this point, but it might not be the first opportunity you get. Whatever it is, do a good job at it and use it to build legible accomplishments that you can add to your CV.
• Continue applying and aiming for bigger opportunities as your accomplishments grow. Repeat until you have a job that you like.

The general idea here is to do a small thing to show that you’re a good candidate for a medium thing, then do a medium thing to show you can do a bigger thing, and so on. It’s often a good idea to apply for a variety of things, including things that seem out of your reach, but it’s also good to keep expectations in check when you don’t have any legible reasons that you’d be qualified for a role. Note that some technical research roles might involve some pretty unique work, and so there wouldn’t be an expectation that you have legible accomplishments in the same research area. In those cases, “qualified” means that you have transferable skills and general competency.

How can I find (and make!) job opportunities?

I used 80k Hours’ job board and LessWrong (I found Timaeus here). If you find your way into Slacks from conferences or local EA groups, there will often be job postings shared in those as well. My impression is that the majority of public job opportunities in AI safety can be found this way. I started working at Timaeus before I attended my first EAG(x), so I can’t comment on using those for job hunting.

Those are all ways of finding job opportunities that already exist. You can also be extremely proactive and make your own opportunities! The cheapest thing you can do here is just cold email people (but write good cold emails). If you really want to work with a specific person / org, you can pick the small open problems you work on to match their research interests, then reach out to discuss your results and ask for feedback. Doing this at all would put you well above the average job candidate, and if you’re particularly impressive, they might go out of their way to make a role for you (or at least have you in mind the next time a role opens up). At worst, you still have the project to take with you and talk about in the future.

Sensemaking about impact

When I first decided to start working in AI safety, I had very little idea of what was going on – who was working on what and why, which things seemed useful to do, what kinds of opportunities there were, and how to evaluate anything about anything. I didn’t already know anyone that I could ask. I think I filled out a career coaching or consultation form at one point and was rejected. I felt stressed, confused, and lonely. It sucked! For months! I think this is a common experience. It gets better, but it took a while for me to feel anywhere close to oriented. These are some answers to questions that would have helped me at the time.

What is the most impactful work in AI safety?

I spent a lot of time trying to figure this out, and now I kind of think this is the wrong way to think about this question. My first attempt when I asked myself this was something like “it must be something in AI governance, because if we really screw that up then it’s already over.” I still think it’s true that if we screw up governance then we’re screwed in general, but I don’t think that it being a bottleneck is sufficient reason to work on it. I have doubts that an indefinite pause is possible – in my world model, we can plausibly buy some years and some funding if policy work “succeeds” (whatever that means), but there still has to be something on the other side to buy time and funding for. Even if you think an indefinite pause is possible, it seems wise to have insurance in case that plan falls through.

In my model, the next things to come after AI governance buys some time are things like evals and control. These further extend the time that we can train advanced AI systems without major catastrophes, but those alone won’t be enough either. So the time we gain with those can be used to make further advances in things like interpretability. Interpretability in turn might work long enough for other solutions to mature. This continues until hopefully, somewhere along the way, we’ve “solved alignment.”

Maybe it doesn’t look exactly like these pieces in exactly that order, but I don’t think there’s any one area of research that can be a complete solution and can also be done fast enough to not need to lean on progress from other agendas in the interim. If that’s the case, how can any single research agenda be the “most impactful?”

Most research areas have people with sensible world models that justify why that research area is good to work in. Most research areas also have people with sensible world models that justify why that research area is bad to work in! You don’t have to be able to divine The Truth within your first few months of thinking about AI safety.

What’s far more important to worry about, especially for a first job, is just personal fit. Personal fit is the comparative advantage that makes you better than the median marginal person doing the thing. It’s probably a bad idea to do work that you’re not a good fit for, even if you think the work is super impactful – this is a waste of your comparative advantage, and we need to invest good people in all kinds of different bets. Pick something that looks remotely sensible that you think you might enjoy and give it a shot. Do some work, get some experience, and keep thinking about it.

On the “keep thinking” part, there’s also a sort of competitive exclusion principle at play here. If you follow your curiosity and keep making adjustments as your appraisal of research improves, you’ll naturally gradient descent into more impactful work. In particular, it’ll become clearer over time if the original reasons you wanted to work on your thing turned out to be robust or not. If they aren’t, you can always move on to something else, which is easier after you’ve already done a first thing.

On how to update off of people you talk to

Ok this isn’t a question, but it’s really hard, as a non-expert, to tell whether to trust one expert’s hot takes or another’s. AI safety is a field full of hot takes and people that can make their hot takes sound really convincing. There’s also a massive asymmetry in how much they’ve thought about it and how much you’ve thought about it – for any objection you can come up with on the spot, they probably have a cached response from dozens of previous conversations that makes your objection sound naive. As a result, you should start off with (but not necessarily keep forever) a healthy amount of blanket skepticism about everything, no matter how convincing it sounds.

Some particularly common examples of ways this might manifest:

• “X research is too reckless / too dual-use” or “Y research is too slow / too cautious.” We all have our own beliefs about how to balance the trade-offs of research between capabilities and safety, and we self-select into research areas based on a spectrum of those beliefs. Then from where we stand, we point in one direction and say that everyone over there is too careful and point in the other direction and say that everyone over there is too careless.
• “Z research doesn’t seem (clearly) net-positive.” People have different thresholds for what makes something obvious and also have disagreements on how much not-obviously-net-positive work is optimal (I claim that the optimal amount of accidentally net-negative work is not zero, which is probably much less controversial than if I try to claim exactly how much is optimal).

I emphatically do not mean to say that all positions on these spectrums are equally correct, and it's super important that we have truth-seeking discussions about these questions. But as someone new, you haven't yet learned how to evaluate different positions and it’s easy to prematurely set the Overton window based on the first few takes you hear.

Some encouragement

This isn’t a question either, but dropping whatever you were doing before is hard, and so is finding a foothold in something new. Opportunities are competitive and rejections are common in this space, and interpreting those rejections as “you’re not good enough” stings especially hard when it’s a cause you care deeply about. Keep in mind that applications skew heavily towards false negatives and that to whatever degree that “not meeting the bar” can be true, it is a dynamic statement about your current situation, not a static statement about who you fundamentally are. Remember to take care of yourself, and good luck.

Thanks for feedback: Jesse Hoogland, Zach Furman

Okay, and what about calibrating? By calibrating, I mean something like "bringing into alignment with reality," i.e. making it so that your gut feelings seem to be good predictors of real-world outcomes. But words like intuition and instinct seem to imply a somewhat immutable relationship to your gut – what is there to do about it?

My answer to this is good, old-fashioned, high-quality, educated speculation grounded in a sample size of one. As with many things, it starts with the art of noticing. That is, to "listen to your gut," but not in the sense of do-what-your-gut-tells-you, more in the sense of try-to-notice-that-your-gut-is-saying-anything-at-all-even-if-you-have-no-idea-what-it's-going-on-about. In fact, you should be careful of the first sense, because at this stage your gut might still be wrong.

Noticing

There's a video game series I'm fond of called Fire Emblem. It's a tactical RPG where you try to strategically hit things and not get hit back too badly. If you do this well, eventually you get to hit a dragon until it dies and you win. There are various game mechanics involved, but the most basic one is your hit chance. For this, Fire Emblem uses a system called True Hit. Naturally, it lies to you.

The details of how exactly it lies to you aren't that important. The important effect is that displayed hit chances that are above 50% are actually higher in practice, and those below 50% are actually lower in practice. Fire Emblem does this because most players, on a gut level, feel that a 95% chance event ought to happen much more often than 95% of the time, and that a 99% chance event is an almost certainty – it feels like an injustice when it doesn't happen! In that sense, Fire Emblem calibrates the game mechanics to better fit a player's gut feelings to make the game feel more intuitive. But what this really does is just hide what your gut is trying to say, even if it's wrong.

Reality typically won't recalibrate itself to match your gut feelings, but that means we can notice what happens when our gut is saying something. The exact "motions" (mostly involuntary and mental rather than physical) happening aren't super important, only the noticing of them. There might be a certain way that we plan around an event, taking it for granted when we shouldn't. We might mentally dwell too much on an irrelevant fact or gloss over something important. We might flinch away from an uncomfortable thought or fidget or feel various physical sensations. There might be a specific flavor of mental unease that pushes us to defer entirely to spreadsheets. Whatever it is, there's something there to attend to.

Calibrating yourself to your gut

Last year, I attended an ad hoc workshop on improving your calibration for prediction markets. There was surprisingly little quantitative analysis – it was mostly about learning to pay conscious attention to the specific gears moving in our brains that led to them spitting out a probability. The point was to be able to correctly associate these gut feelings / gears / mental motions to reality. That is, suppose you think something is the right call 90% of the time and that comes with some kind of internal subjective experience composed of various feelings and motions. Actually though, suppose it turns out that's only true 70% of the time. If we're paying attention, we can learn to re-associate our mental connection with that cluster of emotions from being "what we feel when something is right 90% of the time" to "what we feel when something is right 70% of the time."

That exercise was limited to prediction markets and assignment of probabilities, but I suspect this is much more broadly applicable. If we're listening to a new argument for something, we can pay attention to where we feel resistance, if our nervous system gets activated, or where our mind tries to pattern match, and later cross-check that against a more objective judgment. When presented with an unfamiliar situation, we can notice the things that happen internally as we move through the situation and make decisions and later review if our internal feelings seemed to correspond well with what turned out to be the right decision or not.

As a personal example, I've historically struggled with writing the bottom line first and working backwards to justify it. I recently started noticing – feeling in my gut – what it feels like when I'm rationalizing something, before I consciously realize I'm doing so (and I then try to follow that up by explicitly acknowledging it and positively reinforcing myself for successfully noticing). This has lots of smaller, day-to-day consequences, but it was also a larger scale version of this noticing (that I was rationalizing my trajectory at the time) that became a catalyst for my recent career shift to AI safety.

Actually trusting your gut

So far we've been talking about how to listen to and interpret what your gut is saying. That seems pretty valuable to me already, but sometimes you might have to do the opposite of what your gut is saying if you've noticed it's consistently getting something wrong (such as being uneasy with hard conversations despite them typically turning out well). I claim that at this point, most of the difficult work is done, and you just have to give it time.

If you maintain this explicit mental attention / feedback loop between your internal motions and outcomes, then your gut feelings will gradually evolve. They'll tend to ease their discomfort in situations where you notice that such discomfort is unnecessary and to strengthen their conviction where you notice it is warranted. If you stay in tune with these changes, then better calibration – and being able to trust your gut – is something that just naturally follows.

I've had this conversation a bunch of times now, and I'm happy to talk about it each time, but it seems like it would be more convenient for all parties if there were just a compiled list of my advice somewhere, so here's an attempt at such a compilation. Apologies in advance for any gaps, please feel free to email me with questions, and answers may get added to this reference, though I think I'm approximately aiming for an 80/20 rule here.

Why leave?

Whatever reasons there are will be highly personal. I can list some of my own reasons below, as well as some that I've heard, but if you're reading this, you might have some of your own as well. Note that this is unashamedly biased towards leaving academia, but I don't think someone is wrong if they conclude that staying in academia is right for them, so long as the decision is well-informed.

But my first lukewarm take is that the discoverability of other options is generally terrible, and the friction of figuring out what options there are and how to actually achieve those options is enough to stop many people from seriously considering an exit.

I'll also note a personal mental obstacle that I had as a young grad student. For whatever reason, I had a sort of cached mental impression that not "making it" to an academic job was a failure. It's hard to pinpoint exactly where that impression came from. I'm sure some of it was internal, but there's also a degree to which this view is baked in around us – for example, it's common to refer to leaving academia as selling out, which has some negative connotation. I don't think holding the view that it's a failure is healthy, nor is it true.

In reality, I think most of the reasons why someone might remain in academia are superseded by the appropriate arrangement outside of academia (of which there are many), up to and including being able to do more research outside of academia. Read on for more details.

Reasons to leave

• You make (a lot) more money outside of academia
  • By the time you would get tenure in academia, you could counterfactually be a senior or staff level researcher/engineer (see later on for a note on "levels") in tech and earn 300-500k+ total compensation (combined salary + equity grant, which is as good as cash)
  • The opportunity cost of staying in academia is well into the six figures per year, just from the raw salary difference, not even accounting for compounding growth of investments. If you weren't already in academia would you pay $200k+ per year in order to be in academia? (if you look further down, my estimation of the opportunity cost at the tenured professor level is much higher)
• The job demands, stress, and time commitment are typically lower
  • The demands are often low enough that you can hold a full time job and still have more time for research than in academia
  • The Google employees in the tweet above are probably on the low end, but not that far from the true median.
• You have more flexibility in where you live (especially with remote work)
• Remote work in tech specifically also comes with extremely flexible hours
• Less administrative burden, zero course preps, no outreach/service expectations
• If you learn certain practical skills, you are incredibly employable and don't need the job security guarantees of tenure (and make enough money anyways that you can absorb unlikely periods of unemployment, or just straight up retire after ~10-20 relatively chill years and just do whatever the hell you want after)
• There may be a personal situation tying you to your job. Depending on that situation (e.g. with visa status), you may find a solution anyways outside of industry (many tech companies sponsor visas)

Where do I go?

Ok, so let's say you're considering leaving. What are your career options? When I first Googled "math major career options" in undergrad, the top result was being an actuary. Nothing against actuaries, they're probably cool people, but I don't want to personally find out. Your career options are much more varied than this.

• Tech / Silicon Valley
  • This can mean researcher (e.g. machine learning, data science, etc) or software engineer
  • This is broadly split into Big Tech (FAANG companies) and startups
  • Big Tech has the highest compensation between the two and the lowest job demand (which is paradoxical, but FAANG makes a lot of money, and it's hard to be ultra efficient at their scale)
    • As an intuition pump: Elon caught a lot of flak for firing like, 80% of X (f.k.a. Twitter), but it's still running (albeit with some performance degradation here and there). I was among the doubters, but so far it seems like he was right on this point. Many big tech companies could probably survive >50% cuts if they had the guts to do it.
  • Startups have the most exciting work, but lower compensation (higher than academia) and higher job demand (comparable to academia). If you're an early employee at a startup that eventually IPOs, you're probably going to walk away very rich, but don't count on it. The attraction here is working on problems that feel like they're either building the future or really fixing existing problems in a meaningful way. Be careful not to choose a startup whose mission you don't care about. The entire point of working at a small startup is mission alignment. Working on a mission you do care about feels amazing. Some real examples of startups that you can really work on:
    • Flying houses (mission: tiny houses that can be airlifted into disaster relief areas)
    • Lab grown meat (mission: end animal suffering due to factory farming)
    • Alternative energy (mission: solve world energy supply)
    • Wearable tech (biometrics: live healthier lives / AI assistants: extend our cognition with new tools)
    • Self-driving cars (lots of potential missions, e.g. solve car accident mortality)
    • Literal space travel (mission: go to SPACE and maybe COLONIZE MARS)
  • Those are just the eye-catching ones. If you can think of a real problem in the world, even if it's incredibly unsexy, there's probably a startup somewhere working on it (lots of startups also suck though and are working on fake problems, but evaluating startups is an entire 5k word guide on its own)
• Quant finance
  • Pay is similar to big tech
  • Great if you are specifically extremely competitive, resilient to stress, and can be happy with high variance positive EV decision making even if the realized consequence was negative (think activities like poker)
  • There is a particular type of person for whom this is a good fit, but most people are not, and that is fine
• NSA
  • The NSA employs a lot of mathematicians, but I'm not familiar with the precise kind of work they do. Presumably a lot of cryptography related work.
  • I hear the work-life balance is great at the NSA
• CCR campuses
  • Apparently the NSA also has more "relaxed" research centers called Center(s) for Communications Research
  • Examples: Princeton, La Jolla (there are more if you Google, for some reason they don't seem to be aggregated anywhere)
• NASA
  • You can do cool space stuff!
• Government labs
  • There are a variety of national labs that fund fundamental research
  • If such a lab happens to fund the type of research you do, you can enjoy what is essentially an academic job but without the teaching
• Healthcare / biostats
  • A close friend of mine does work in epidemiology using statistics and gets to do covid modeling
• Honestly, almost anything
  • Your most powerful tool is being agentic. You can, in some sense, really do whatever you put your mind to – most people are limited by artificial walls. A PhD is proof that you can do hard stuff on your own. The degree of career freedom you have outside of academia is incredible. Not all artificial walls are worth tearing down, but before you think "I could never do that," look with fresh eyes and ask if that's really true.
  • There are plenty of careers that there aren't degrees for. America employs ~160M people, and my undergrad institution had like ~120 majors. Lots of people have jobs that they didn't formally study for, and you can do that too.
• Things that I did or almost did at various points (in the span of only 2.5 years!) that no one had to give me permission or a formal list of prereqs for: education tools, music tech, science communication, game design, sports betting (for American football – and I still don't even know what a tight end is), algorithmic trading, software engineering, blockchain/web3, tech consulting/advising for early startups, data science, supply chain optimization, AI safety research, and probably a lot more that I've forgotten already
• Other thoughts
  • Perspectives that people have on PhDs are mixed, but there's more than enough people who look at it and say "oh shit, they have a PhD, they're probably really smart" rather than "their thesis isn't relevant, let's not hire them" that I think it's net positive (you only need one good hiring manager to think positively towards a PhD)
  • There is a ton of interesting work that happens outside of academia.

How do I actually, concretely get a job?

[This is from the perspective of a student, but postdocs and professors can adapt the same general strategies.]

For most of the common career paths, the primary new skill you need to pick up is some degree of programming (coding combinatorics experiments is a good starting point, but not representative of the skills you need) and/or some degree of statistics/data science/machine learning (these three all blend together to different degrees at different jobs). Everything else you can learn on the job (and don't worry – everyone does). If you're picking the "choose-your-own-adventure" option in the previous section, then I'll leave it up to you to determine how much of what skill you need to learn and how you learn it.

• You probably don't need to learn much programming, if any, for jobs like at the NSA or at government labs (or if you do, they might just teach you on the job)
• You need to learn a little programming (probably leaning towards statistical languages, e.g. Python, R) but mostly data science concepts for things like biostats and data science
• You need to learn a decent amount of programming if you want to do software engineering
• You need both programming (less than for software engineering) and data science/ML for quant finance
• You need both if you want to specialize to certain parts of software engineering, such as machine learning engineering
• Lukewarm take, but I think basically any job in the realm of knowledge work will benefit from some proper programming experience, even if the job description doesn't explicitly call for it (and even if none of your coworkers know any)

From here on, the main focus will be on getting tech/quant/data science jobs. The reason for this is that these are 1) what I'm most familiar with and 2) what I expect will be the most involved processes / require the most domain specific prep knowledge.

Learning data science and machine learning

Your core resource is Kaggle. Start with the Titanic and Housing Prices intro competitions (in that order). Find some example notebooks, or if you're braver, try to analyze the dataset and generate predictions on your own without hints. These notebooks exist to familiarize you with the common tools, languages, and techniques that are used in data science. Pay attention to the questions asked, but also the heuristics and rigor. Notice the types of questions which are not asked. Data science is a different field from mathematics with different standards for what is or isn't a good answer to a question.

Work your way up to trying some live competitions, including competitions with real prize money. Pick one that seems interesting, maybe find a few friends who are also trying to learn data science and form a team. Give yourself somewhere from a few weeks to a couple months and dig really deep into that competition. There is no substitute to learning by doing.

As a reference point for the level of difficulty of questions you might get in interviews:

• One of my onsite questions at a quant hedge fund was essentially a series of questions about the housing prices dataset
• Another ML/data science role asked me to code the k-nearest neighbors algorithm from scratch (I later learned that they didn't mean literally from scratch, it was meant to be a test of how well I knew the numpy library in Python)
• Various questions on linear regression. These types of questions did not inherently require knowledge beyond the undergraduate level, but they did require a depth of understanding of those concepts that wouldn't normally be expected of an undergraduate.
  • E.g. suppose you have a dataset on the 2D plane, where data is uniformly distributed in a rectangle. What happens to the linear regression line as you rotate that rectangle 90 degrees clockwise or counter-clockwise?
  • Similar questions about normal distributions are common

Learning to program

Programming experience in the software industry is typically split into "production" and "not production" level experience. "Production" means work that was done in a commercial setting or otherwise at a commercial level. This typically implies things such as: real money is on the line, code is run live without supervision most of the time, and coding is done as a team in a shared repository with peer reviews, among other things. Unfortunately, within the software industry, things like personal projects or coding experiments for math research are not typically considered production-level coding experience.

At first glance, there is an apparent catch-22 that it's hard to find a job if you don't have production-level coding experience, but you can't get production-level coding experience if you never get a job. Internships can be one solution, but maybe you're applying in your last year and you don't have any summers left, or maybe you're a postdoc/professor already. In that case, here is a "minimum viable solution" to getting out of the catch-22:

• Watch the MIT OCW Introduction to Algorithms videos. Doing problem sets is optional.
  • Become comfortable with the data structures and algorithms presented as well as in the analysis of time and space complexity of solutions.
• Doing problem sets in the previous step is optional because doing problems in this step is not optional. Make a LeetCode account, consider if a premium subscription is useful to you (I bought one month of premium at one point and found it useful enough for one month but not more), and then do a ton of problems.
  • Start with easy problems, then move up to medium once you feel good about easy problems.
• There is a (now somewhat infamous maybe) list of 75 LeetCode problems that aim to be representative of the types of problems you'll run into in interviews. If you're pressed for time (e.g. graduating soon), this is the minimal set of LC problems you should aim to solve
• LeetCode is seriously the core of the current meta of (entry level) tech interviews. It is a fake skill in the sense that the connection with ability to do real work is dubious (though I don't think it's totally indefensible), but it is a real skill in the sense that you can really study to the exam and get good at LC style problems and this is highly instrumental for getting a job in the first place.
• A good benchmark is being able to consistently do LeetCode medium problems on your own, with no hints, in 15-30 minutes. The vast majority of interview problems are not harder than LeetCode medium. Past this point, you start to get diminishing returns on the time spent on LeetCode, but it is (IMO) quite hard to accidentally get to a point where marginal practice is pointless.
• Quick plug for a fun alternative to LeetCode: Advent of Code. This is an annual advent calendar of coding problems that increases in difficulty as it approaches Christmas and has a silly story to go along with it.
• Finally, work on some actual personal projects. Find something interesting and just do it.
  • You can sometimes get a 2 for 1 with data science/ML experience by doing some kind of coding-intensive competition on Kaggle.
  • The first project I ever worked on that made it onto my resume was an automated arbitrage bot for Magic The Gathering: Online. The code was trash, it was a buggy mess (in our defense, this was partly because MTGO itself was a buggy mess), and we got banned from the platform shortly after. We made some actual money off of running the bot, but the real value was returns on career capital – it has one of the highest return rates of any single week I've ever spent coding. It is the most common talking point in my interviews and is still proudly displayed on my resume.
  • Check out this Hacker News thread for examples of long-tail outcomes of interesting projects
• Another avenue to show off some of your coding (and to actually get some real code review/feedback) is to make open source contributions on Github. I won't get into how to do this, but some employers view this very favorably and like to see actual code examples as a way to de-risk hiring you.
• A longer path (which you can take if you decide well in advance that this is the path you'd like to take) is to take some computer science classes, and maybe even get a masters in CS. This seems effective if you're willing to put in the time commitment and can plan ahead for it.

Misc quant questions

There is a small industry built around certain types of quantitative problems and brain teasers that get asked almost exclusively in quant interviews. Getting into these problems is probably out of scope for this document, but you can find plenty of examples online.

General interview advice

Interviewing is a skill, too. Your first few interviews will probably feel like garbage. If I haven't interviewed for a while, my interview skills get rusty, and it takes a few warm-up interviews to get comfortable with them again. Bias towards interviewing earlier in your interview prep process and just get the crappy first few interviews out of the way. I promise it gets better, and there are enough companies out there that flubbing a few interviews for middle-range targets isn't an issue.

For most of the quantitative jobs above, the vast, vast majority of your interviews will be technical interviews. Other interviews (such as with HR or recruiters) will be more like formalities in the process, mostly just checking that you're reasonably easy to talk to. I didn't have a single real behavioral interview until one point where I was speculatively interviewing for a totally different kind of job.

For technical interviews, a full guide is also out of scope for this reference, but here are some general pointers:

• Think out loud (seriously, it is sometimes a red flag for interviewers if you are too quiet)
• Ask clarifying questions (green flag if you ask them, red flag if you don't + were wrong about interpreting the question)
• Don't be afraid to start with naive solutions to programming problems where the time complexity sucks and is obviously not optimal. As long as you say that out loud and then iterate from there, it's totally fine and counts as partial progress towards answering the question
• If a question seems really unusually hard, it might be your unlucky day, but it might also be the case that it's hard for everyone. Many places rate interview performance on problems "on a curve"
• For some questions, you can get more "points" by sharing high quality thinking out loud without arriving at the right answer than by just jumping to the right answer. What this looks like is: asking the right questions, thinking about the right things, coming up with tractable, reasonable sounding ideas (that may not ultimately work)
• If you know someone who is in tech, call in some favors and do practice interview sessions and get feedback
• The interview process can be pretty long; I think my longest interview processes had 5 rounds (where the last one is an "onsite," though these are often over Zoom now instead of physically onsite), and I've heard of more. Have a constant pipeline of interviews, and don't wait for one interview process to end before looking for another.
• Don't get too attached to any one job. I made this mistake a few times, and still ultimately ended up with a job I was very happy to get each time (and which was likely better than the counterfactual offer anyways)

A note on "levels"

One of the great (IMO) features of tech careers is the notion of a "level." Although they aren't fully standardized between different companies (and it takes some experience to know what to expect from a "senior" engineer coming from different types of orgs), I still find them a pretty useful framework for understanding jobs in tech. A typical sequence of levels at tech companies (or quant sometimes, for larger firms) looks something like the following, where if you're on say, a data scientist track, you might replace "software engineer" with "data scientist" everywhere.

• Junior
  • Sometimes can be a title like "Software Engineer 1" or "SDE 1" or "SWE 1"
  • Entry level for bachelors
  • Very low expectations, your job at this level is mostly to learn. Companies typically view juniors as an investment more than a clear source of value.
• Mid
  • Often comes without a title prefix, e.g. just "Software Engineer" and not "Mid-level Software Engineer"
  • Otherwise might look like "Software Engineer 2" or "SDE 2" or "SWE 2"
• Senior
  • Typically considered a "terminal" role in tech – there are many levels beyond this, but this is where you're considered good enough for companies to be happy to have you sit here indefinitely and provide this level of value without growing further. Many people do just that.
  • Typical expected time is ~5 yrs to reach this level (for the median engineer that makes it to senior level, not for the median engineer with a PhD that makes it to senior – you can make it to senior much faster if you want but you do not have to)
  • When interviewing for this level and above in software, you will start to need to know system design. Expect at least one technical interview to be on system design problems.
• Staff
  • This is where there's an option to move into the management track, which has its own levels that parallel the "individual contributor" (IC) track
    • Most people actually move into management instead of moving up the IC ladder, see notes on that below
  • It is harder to define what your role is at this level and above, staff+ ICs tend to get increasingly specialized roles based on their individual capabilities and the needs of the company
  • Compensation was already stupid, but this is where it gets real dumb. 500k+ total compensation is common at FAANG
• Principal
  • Here levels start to break down further at different companies. Sometimes there is a "senior staff" before this level and sometimes there is a "senior principal" after this level. Sometimes staff is skipped, sometimes principal doesn't exist.
  • Compensation gets downright offensive: FAANG principal+ can earn 7 figures

Management track has similar levels (and compensation) starting with "Engineering Manager" or something similar being the equivalent of something like "Staff Software Engineer." It then has its own titles like "Senior Engineering Manager," and may go towards titles like "VP of Engineering." All of these levels (for both IC and manager track) have their own expectations, which I won't cover here. There are good/comprehensive resources online, like this one.

Benchmarking levels against academia

Ok, here's where the real hot takes begin. I'm not going to bother justifying these, so take them with a grain of salt, but I think this is what I believe from my limited perspective.

• If you don't have a PhD in computer science or machine learning, expect to enter tech or quant at the junior or mid-level. You might be able to get a data science role at a senior level if you did e.g. probability and do well on your interviews.
  • The startup I first worked for didn't have levels in a formal sense until a year after I joined, but I think I was retroactively initially placed at the junior level and then promoted to mid-level after a year.
• If you can get a PhD in math, you can reach senior level in tech, full stop. Tech jobs are high paying because 1) tech companies make a shitload of money, and marginal engineers really do actually bring a lot of value to the company at scale and 2) because the jobs are competitive and hard and have high expectations. But crucially, they are competitive and hard and have high expectations relative to the typical job in America. They are not more competitive or harder or have higher expectations than earning a PhD (modulo personal fit for tech, some people may just find software easier than others).
• Postdocs up through tenured professors are probably somewhere around the "staff" level equivalent for academia.
• I'm less sure about this, but I'd guess that full professor in academia is comparable to principal level in tech or to the equivalent management track role (hey, remember that 7 figure compensation number above?)
• These don't mean they're where your cap is, but I would confidently place money at 1:1 odds (and probably much worse odds, though I don't want to commit to figure those out here) that someone with the given level of academic accomplishment can first reach the given comparable tech level in a typical amount of time
• In general, the number of people who reach each progressive level in the tech ladder tends to decrease exponentially, and the difficulty and expectations of the levels also increase substantially past senior. It is totally fine to hit ~senior level and just chill there forever and do other stuff with your time, like fun side projects or your math research or just having more time to enjoy life in general.

A personal request

If you've gotten this far, hopefully this resource has been useful to you. I am happy if this improves your life in any way. My only request to you is that you please try to avoid working directly on AI capabilities as a result of this advice.

I've written about AI safety previously and have pivoted my career to technical safety research because I think this is such an important problem. This is not asking you to say, turn down the only offer you get and not be able to support yourself or your family, but to please think about this if you find yourself having a choice in the matter. Most job opportunities don't fall under this umbrella, and it'll probably be clear if a given job is aimed at significantly advancing AI, so you mostly have to be specifically looking for such jobs.

Dear reader, it was foggy and overcast. There was no sunrise. I'm not upset, but thanks for asking.

While I was walking, I thought back to the conversations I had in Oxford and to previous work retreats before that. I've only experienced remote work, and a lesson that I keep relearning is how impactful meeting in-person is. It's clearly helpful for productivity – for about a week, everyone's primary focus is on planning, on sharing knowledge, on aligning the big picture. But I don't think this is actually the most important part. There's a key social cohesion created, and not just from intentionally designed icebreaker activities.

When you first interact with someone, you get the broad strokes that everyone gets. He's a teacher, she's an engineer, they're a parent, a spouse, a sibling. Work intros often start with everyone rattling off education history, past jobs, and maybe a well-rehearsed fun fact. Basic knowledge is of course a necessary starting point, but how much of a picture do you really get of someone from that?

These are the things that often come to mind when we think of our identity, but billions of other people in the world are, say, cat or dog people, and I'd take short odds that there's more variance within one of those groups than between them. You can learn a dozen facts about a person and still be no closer to understanding them or building a close relationship.

Most people never see the the small, mundane, in-between moments in our lives, but I think this is where intimacy lies. I don't identify with a foggy morning; it doesn't feel like a part of who I am in any way. And yet, I'd only share this kind of story with a few people (and you of course, dear reader). I don't broadcast this to the world (kindly ignore that I'm publicly writing about it) because it isn't that important, and most people just won't care. For the people that do, they care because it's a story about me, because someone in their tribe experienced it, and because it means something to be one of the first people that comes to mind when anything happens.

I'm not saying that spending a week in person gets you anywhere near that level of closeness. But having such dense overlap in organic moments sure seems like a bigger step forward than as many video calls to hash out business logistics. There's no choice but to experience the ordinary together and much more space to fill with mundane questions (how was your morning?) and mundane stories (the funniest thing happened...). Maybe there's something like the Ben Franklin effect happening too – if sharing is caring, then perhaps caring follows sharing as much as the reverse.

I also notice that a drop-off in mundane sharing predicts an eventual drop-off in closeness**. I think this makes sense; most of what happens in our lives isn't news-worthy, so not talking about the boring stuff means much less contact overall. But beyond this, there's probably something powerful in gradually losing the urge to share and in the worry that you might be bothering someone with something uninteresting rather than feeling secure in the idea that anything is interesting to them if you're the main character. Sometimes the last things to develop are the first to go.

*Technically this happened a few days ago by time of publishing, but I wrote that line on the day it happened. Don't sue me.

**I'm also noticing while writing that I'm generally not sharing much of the boring stuff with anyone right now (so for my close friends reading this, don't go off thinking I secretly hate you in particular because I didn't tell you about a non-sunrise). This seems like a sign that I'm getting too isolated and need to give more attention to my social wellbeing.

It's been an eventful year! I quit my job, tried to start a company, had a summer fling with freelancing (it was awesome), and did a ton of soul searching. It's also been a bit of an emotional rollercoaster: lots of stress, lots of confusion, lots of fun (from New England sunset boat rides to laser tag in a castle). The year's not over yet, so there's still time for curveballs, but it looks like some dust is finally settling.

The dominating consequence of my soul search was the conviction that AI safety is the most important thing I can work on right now. I spent the better part of the last 6 months exploring how rapid AI development will transform our world, and the world in turn kept smacking me in the face with how wide the cone of possibilities was. Unfortunately, not all outcomes are positive – I now believe that this is the most perilous century for humanity, with AI being the primary driver.

The hype around AI – that it could solve cancer, poverty, conflict, disease, even aging and death – massively accelerates its development. To be clear, AI safety shouldn't be conflated with the belief that approaching Reedspacer's Lower Bound is inherently bad. I hope we cure all disease! I hope we get functionally infinite prosperity! I am so on board with solving death! It would be tragic if we never achieved safe artificial superintelligence.

But hype breeds recklessness. Almost all of our forecasted extinction risk this century comes from AI – many times more than from nuclear war, climate change, and bio-risk combined. And development goes on accelerating anyways. I want all the great things that AI promises, I just think we can get them for a better price.

So what's next for me? I'm excited to work towards a safer future with Timaeus, a brand spankin' new technical AI safety research org. I'm hopping on board to help scope out a new field of interpretability research as a research assistant/engineer for at least the next ~6 months. I'm also frantically cramming Spanish for the AI Futures Fellowship in Mexico City early next year, where I'll hopefully develop a broader base of knowledge, meet brilliant people, and eat far too much food.

The future is unclear

I think more people should spend time thinking hard about AI. Whatever consequences you think there will be, it seems hard to get away from the conclusion that they'll be transformative in some direction. But sensemaking about AI is really hard, especially if you don't have a technical background. Even as someone with the relevant background, I spent months highly confused about the right path to take and still feel fairly uncertain. We can and should at least still strive to be directionally correct though, and voting for safe policies is just as critical as technical research (if not more).

Finally, I think this is the ~correct direction (or a correct direction) for me to be moving in right now, but it's all pretty nuanced, and I'm always open to more perspectives. Please reach out if you're curious or have thoughts you'd like to share. Don't be a stranger, odds are I'd love to chat – everyone who's let me talk their ear off so far can attest to that.

This thought keeps coming back to me lately: everything is about to get really weird. For the past two years, I’ve had a front-row seat to the primordial soup of AI research and startups. I’ve built a couple of products based on language models, consulted on generative music tools, and dabbled across the board of generative AI, including image, audio transcription, voice, and more. A massive wave of commercial products is coming, for which ChatGPT is just the tip of the iceberg.

AI has been fairly consistent in progressing exponentially. The thing about exponential growth is that for a long time, it looks like nothing is happening, then it looks like something is finally happening, and then while you’re still processing the something that’s happening, the growth explodes and… uh oh, suddenly everything is weird now. We’re still in the “huh, it looks like something is finally happening” stage with AI.

A certain recent pandemic demonstrated that we’re not so good at dealing with the next phase of exponential growth. But covid didn’t care what was or wasn’t intuitive to us; the world just changed overnight anyway. We scrambled to react and flatten the curve, but not before it was already a foregone conclusion that there would be a huge curve. When it comes to AI, it’s fortunately not all downside like it is with a global pandemic. AI has incredible potential to bring prosperity – some have even compared it to the discovery of fire or electricity. But there’s a darker side, and that side grows exponentially too.

While writing this, I hopped on X (formerly known as Twitter, RIP) and saw a post on practical acoustic side-channel attacks on keyboards. The model in that paper can correctly identify 93% of your keystrokes from the audio that a Zoom call picks up. I see news like this more often than is comfortable, and then I predictably think to myself that everything is about to get really weird. You have to really stand out these days to be noteworthy – a few months ago, a research team managed to read the human mind(!) with >90% accuracy by training a model on fMRI scans. [Note: since writing this, we've managed to reconstruct music by reading brainwaves too.]

The optimistic among us would hope in vain that AI tech isn’t abused. FraudGPT and WormGPT are two cousins of ChatGPT, repurposed for enabling cyberattacks and scams. ChaosGPT was a (fortunately ineffective) bot whose task was literally to destroy humanity. What happens as we progress along the exponential growth curve and the people building these harmful AIs get access to shiny new toys? Dario Amodei, CEO of Anthropic, testified to the US Congress that a straightforward extrapolation of today’s systems suggests that large-scale biological attacks could be possible in only two or three years. He went on to say that if we don’t have mechanisms in place to restrain AI systems by 2025-26, “we’re gonna have a really bad time.”

Two to three years isn’t a lot of time, and we don’t even know what the right restraining mechanisms are yet! It would be great to have a bit more time to figure things out before the exponential progress curve blows past us. Fortunately, unlike with the pandemic, we can actually make that happen for AI. The Future of Life Institute wrote an open letter with over 33,000 signatures calling for a six-month moratorium on AI development, signed by many of the biggest names in AI research and tech. It’s not clear that the specific plan of a six-month moratorium would be most effective, but this seems directionally correct. Policymakers might consider prophylactic legislation that would allow such a moratorium to come into effect if certain conditions are met. Compute governance is a similar slowing mechanism, where large training runs of AI models are limited in the amount of compute power they’re allowed to use. The threshold can be adjusted over time to have some control over the pace of development without it being all or nothing. These aren’t long-term solutions, but they would buy time to figure out more permanent measures.

Besides short timelines to serious risks, another core challenge with making AI safe is the asymmetry between offense and defense. There are simply too many angles of attack to be able to cover all vulnerabilities – but policy can reshape the AI landscape into something more defensible. When advanced AI is too openly accessible, any individual can choose to take one of those many angles of attack, so an obvious solution might be to prevent advanced AI from being too openly accessible.

We can accomplish that via regulatory action, but this is a pretty controversial topic in the tech world; open-sourcing (making publicly available) your research and your code is usually synonymous with nice things such as transparency, collaboration, and even security since you have more eyes on fixing vulnerabilities. Furthermore, a common counterargument to closing off access to AI technology is that it centralizes too much power in the hands of the few major AI labs and that we can only have a fair balance of power if the technology is available to everyone.

But the world is not always a safer place when people have more access to technology. The US has a lot of guns and – surprise – a lot of mass shootings. If everyone had a nuke in their pocket, it would only be a matter of time before most urban cities disappeared in a mushroom cloud. All it takes is one person doing the wrong thing. And if everyone has a large-scale-biological-attack-capable AI on their home computer, Dario Amodei put it well: we’re gonna have a really bad time.

It doesn’t seem likely that we can entirely close Pandora’s box on AI, but we can make it manageable. Everything is about to get really weird, but we still have the power to decide if that’ll be a good weird or a bad one.

To get that number, I made some salary approximations, calculated taxes and cost of living, and was naively surprised at S&P 500 returns in a zero interest rate environment, which managed to account for a substantial amount of the difference. The total gap will increase over time too, through the interminable power of compounding returns. Your ultimate career path might not have quite such inflated salaries, and zero interest rates probably aren’t coming back any time soon, but the opportunity cost for a PhD may still be much higher than you expect.

Some people hold the position that they should do their best to deter anyone and everyone they can from getting a PhD. Their belief is that you’re fit for one if and only if you ignore that advice and boldly forge ahead anyways.

I think this is unnecessary. There are plenty of real reasons not to do a PhD (I have at least 1.17M of them), and one should have a better reason to do one than an inability to absorb advice. These considerations, and my own story, are the best I can offer to you if you're a prospective student; the final decision is up to you.

The happy path of grad school

Whatever path you choose in life, you’ll take some form of opportunity cost along the way; the important thing is just to notice the tradeoff. For example, last I checked, it was quite expensive to raise children in the US. Most people go on to have children anyways, and many go on to be happier with their kids than the money would make them. An econ PhD friend of mine talks about a concept of “Life EV”: sometimes we do things that are financially suboptimal because they sufficiently improve our life along other dimensions (he often yells this as he's halfway through leaping into a clearly bad decision because it seems fun).

But just as you shouldn’t have kids for the sole reason of hoping you have someone to visit you in your nursing home, you shouldn’t only think of a PhD in terms of what comes after. The best reason to do a PhD, and one which is both necessary and sufficient, is that you simply want to experience the journey.

That journey can be a beautiful one, if you’re lucky. You’ll meet other like-minded, bright-eyed grad students. You’ll meet generous mentors who pay things forward like it is their moral duty. You’ll travel to conferences (on someone else’s dime!) across the world, and you’ll meet more people that care about your tiny corner of reality than you could have imagined existed (there were dozens of us).

You can work on the kind of problem that follows you to the shower. The kind that itches your brain at night. The kind that jolts you awake as you lay in bed, because a piece finally clicked into place and you can’t hope to sleep until you write it down. The kind that gives you an adrenaline high for days when you figure it out, because you’ve been working on it for months or years.

There are other reasons that help, too. It’s hard to believe you can tackle a challenge that no one else has, especially if the pressure of completing your thesis hangs over you. A PhD is training and proof of your ability to grapple with the unknown.

You might want to do something you can only do with a PhD, or which a PhD can open doors for. In the world of math, this could mean working at a university, but it could also mean working at a hedge fund, at a national lab, or at an AI lab of a big tech company. Most things don’t need a PhD in the strictest sense, but people do pay attention when you have one.

These things are icing on the cake, but at the end of the day, the surest way to make it a positive, worthwhile experience is to want the experience. A PhD should not feel like paying your dues, and it is not noble to suffer through one. An important corollary is that if you are decidedly not enjoying the experience, it’s ok to leave!

A note on "slack"

It can be a little paradoxical to describe grad school as a time with a lot of slack. After all, grad students have a reputation for being over-worked, under-paid, over-caffeinated, and under-rested. But it isn't always like that, and there is a unique thread of intellectual freedom through all of it.

For me, that freedom manifested as space to grow. Once I finished most of my coursework (first two years), the only expectation seemed to be that I spend a lot of time thinking. In theory, that means thinking about your research, but in practice, my mind still managed to wander for many hours a day.

That space expanded considerably in my last two years; the pandemic happened, I'd already decided to leave academia, and I had enough research to graduate. I had few expectations to fulfill, and any remaining research work was mainly tying up loose ends. This was an unusual abundance of slack, but it led to substantially more hours being alone in my head.

I used to think this time was a bit wasted. The amount of concrete things that came out of those last couple of years felt pretty dismal in number. I was probably burnt out for a substantial portion of it, and the social isolation was hard.

Looking back, those years were probably some of the richest years of personal growth that I've ever had. Among other things, I developed a lot more self-awareness and confronted large chunks of emotional baggage that I never previously noticed.

This constitutes a large part of why I would make the same choice to do a PhD, if I could go back and choose again. It also highlights another kind of opportunity cost, which is of time. Whether I spent 5 years in grad school or at Google, those 5 years would pass either way. I only got to spend my early 20s once, and I'm glad that I spent them the way I did. Just as money in the bank would compound year over year, I can benefit from that space to grow for the rest of my life.

The darker side of grad school

The PhD journey can be a beautiful one, if you’re lucky. If you’re unlucky, academia is a meat grinder. There are various opportunity costs to consider, but there are real costs too, mostly in the form of stress to the mind and spirit. From my limited view, the following things were the main culprits.

Impostor syndrome

Academia has a high density of brilliant people. Many academics, including ones you will think are far more brilliant than yourself, feel they don’t belong. They do, and you will too if you choose to join them. There is an entire post that can be written about this topic, but the short version is that it is important to know that you may feel this way, to know that it is a normal and well-worn path, and to know that there are resources for this.

Social isolation

It can be intoxicating to be consumed by your work, but the darker side is that you can end up disconnected from and left behind by your social network. PhD programs are selective, so you can’t be picky about which city you end up in. I think this is especially hard for overseas students, where even phone calls are difficult to coordinate thanks to time zones.

You’ll see other people moving on to the next stage of their life, starting their careers, starting families, buying houses, cars, vacations, and other shiny things. Your family and friends may valiantly try to understand your work, but they probably won’t. Your cohort might end up a bit south of social and outgoing (n.b. you are allowed to be the one that organizes social events!).

My advice for this is to find a hobby with a strong community around it (preferably a physical one; you get exercise and get to meet people in person). I took up rock climbing in my first year, and it changed my life.

A bad advisor relationship

Professors are people too, and in any group of people there will be unsavory individuals. Some rare professors have a reputation for dealing in politics, gossip, and lies. It is not always easy for a fresh grad student to figure out who these are, but one should do everything in their power to avoid such professors.

Some other professors have very particular (and antagonistic) approaches to mentorship. One who shall not be named would intentionally fail students on their quals. Their alleged motivation for doing so was to use the pressure to pass quals as a motivational whip or forcing function on their students. By doing so, they could compel their students to study additional foundational material for the second attempt. Think twice before working with someone like this.

Setting aside extreme examples, you might just not get along well with your advisor through no fault of yours or theirs. Working chemistry is fickle, and you’re allowed to decide it isn’t a good match. Shop around for advisors aggressively until you find a fit (or at least do the best you can with the size of your department), since from the point of picking an advisor onwards, they’ll be the biggest factor in your journey. It can be worth picking your second favorite topic if it means working with your favorite professor.

Academia or bust

One of my undergrad math professors once told me how they got their job. While preparing to defend their thesis, their advisor got a phone call from a colleague asking if he had any good students finishing soon. That was it. No interview, no postdoc needed (typically a 3 year contract), they just discovered the next day that they had a job waiting for them. That was some 60 odd years ago.

Today, you’ll fight to earn 1-2 postdocs, then fight even harder to earn a tenure track position, then work on earning tenure (usually a 6 year process). Although it’s uncommon to be denied tenure once you have a tenure-track position, that’s still a total of ~16 years from the start of grad school before you can be completely sure of stable employment. A pretty low (but not insignificant) portion of PhD students that want tenure eventually make it.

I believe in chasing dreams; if tenure is your dream, then aim for tenure. Just don’t hinge all of your expectations and happiness on that outcome. Luck is enough of a factor that brilliancy won’t inoculate you. Yitang Zhang proved a partial result of one of the most famous open problems in math in 2013. He was incredibly persistent, but prior to this major result, still found himself living out of his car and working at Subway (I think this is exceptionally unlucky though).

At the end of the day, there are plenty of other places you can do research or teach.

Unexpected regrets

Few journeys end with no regrets. 5 years is a long time for regrets to find you, and they may not be what you expect. I am happy to have done a PhD, and I wouldn’t change that decision. But my biggest regret was that it kept me away from home for what ended up being the last years of our family cat’s life. She unexpectedly passed away shortly after defending my thesis, and before I could make it home to say goodbye.

Appendix: miscellaneous advice + other articles

External links

• Read Terry Tao’s career advice, it generalizes well. Some standouts for me:
  • Does one have to be a genius to do maths?
  • Continually aim just beyond your current range
• Sara Billey’s advice on getting a PhD in a timely fashion.
  • Choose your advisor systematically. Talk to current and former grad students. Are they happy? Do they have the kind of job that you want? Do they have jobs at all?
  • 20 hours per week of focus time is enough. Hours in front of the TV don’t count. Hours interrupted by texting and checking your phone don’t count.
• John Baez’s advice for young scientists.
• A great list of questions before starting a philosophy degree.
• A list of resources by Matt Might.

On teaching

• Put your heart into it when you do it, because you have a responsibility to your students, but set reasonable boundaries, because you have a responsibility to yourself, too.
• It’s easy to put infinite time into teaching. It feels productive compared to slamming your head against a research problem with no visible progress. Resist this urge. In the language of the Eisenhower Matrix, teaching is frequently important and urgent, while research is frequently important but not urgent. The latter category contributes most to long-term success.
• Grade efficiently. If you can choose the grading rubric, simpler point rubrics are easier and faster to grade. The difference between an answer that is 2 out of 3 points and 3 out of 3 points is much easier to spot than 17 or 18 out of 20.
• Grade fairly and predictably. Students will compare grades and answers and bring papers back to you with complaints (even if you grade perfectly, students will do that anyway, it'll just cost you less time).

Other notes

• Develop connections with people outside of your department. If you're gunning for academic positions, aim for at least one recommendation letter from someone outside. Your advisor can and should help you via warm intros.
• People who brag about how much they work don’t work as much as they say they do. Don’t let it get in your head, and don’t try to catch up. Grad school is a marathon and not a sprint.
• People who brag about how little work they do often work more than they say they do. Some people really are built different, but for the most part, it isn't the norm to work very little and make visible progress.
• There’s some folk wisdom that you should go to a different institution at each stage of your career. The idea behind this is that each department (or research group/lab) has its own school of thought. This might hold true beyond grad school, but it’s overblown at the undergrad -> grad transition. Don’t be afraid to stay at your undergrad institution for your PhD.
• Prestige isn’t everything, but it still matters more than it should. If you want to stay in academia, then the prestige of your advisor can open doors for you. If you want to leave academia, people still pay attention to the university brand next to your degree.

Thank you to those who volunteered to read drafts: Alex, Felix, Ross

Feedback appreciated!

There might be some collective wisdom on this. I honestly expected a lot more concern and muted skepticism whenever I revealed that I’d quit my job. What I received instead was congratulations, excitement, and curiosity. My guess is that good things seem to happen when someone voluntarily makes a big life decision. Maybe they have an amazing opportunity waiting for them, maybe leaving their comfort zone creates amazing opportunities. Maybe I’m just surrounded by relatively fortunate young people without enough financial commitments to make unemployment truly scary.

Anyways. The point of this isn’t to convince you to quit your job. This was a fairly lucky outcome, but I didn’t plan for that, and the point isn’t to brag about it either. I’ve had a lot of lucky outcomes in my life, and the point of this is to share how I made that happen and how you can be luckier, too.

What do you need to be lucky?

Luck is random, but it’s not purely random. If we think of being lucky as having net positive impact from random chance in your life, then we can be skillful by selecting more opportunities where the odds are stacked in our favor. This is hard because the odds are usually too fuzzy to compute.

This implies a few prerequisites for practicing luck as a skill. First and foremost, you must have an appropriate amount of belief that you can have an impact on the seemingly random outcomes in your life. Our belief in this is heavily conditioned on how lucky we’ve been, and we tend to over attribute skill when good things happen and over attribute random chance when bad things happen. As a result, there are people for whom this feels obvious and people for whom this feels impossible, but both parties are off the mark. Underconfidence means you never take enough shots on goal, but overconfidence means you’ll bite off more than you can chew, and you’ll really be gambling more than practicing luck as a skill.

You must also have an appetite and tolerance for risk. The goal here is to select more opportunities with odds stacked in your favor, but that still means taking a lot of opportunities where odds are involved. That means you will still lose a lot of bets. Humans are naturally risk-averse, and it will feel bad to lose bets. Dealing with this over time requires a high tolerance for failure, and you must have tolerance both in your mental game and in having sufficient resources to absorb the negative outcomes. Resources here can mean things like time, energy, money, social status, etc, and having more resources makes the mental game easier. Quitting my job was made much easier by knowing how long my savings could last and that I was fortunate to have a valuable skill set.

It’s a rigged game

Fortune is not evenly distributed. Some people are born richer, some with more social status, some are more neurotypical, some more attractive, and on and on. People with more of these resources have access to more opportunities, and as a result, access to more opportunities with favorable odds. It is easier to afford the necessary risk of these opportunities if you have more resources. Although this post is not about the distribution of these resources, make no mistake: this is the truth and this is not fair.

That being said, you can make a meaningful impact on your future luck. Whatever your starting point is, you can beat a majority of people who don’t try, and most people don’t try. It also gets easier over time; luck begets luck. Quitting my job was easier because of my savings and skill set, but the size of my nest egg mostly came from a couple of unique bets that turned out well. I chose my career from the overlap of things I like and things that pay well, which increased my access to good opportunities. Those in turn came from earlier, smaller things. Wherever you start, your luck can gradually snowball and change your life.

A closely related concept is playing to your outs, and nearly every life situation has outs, however improbable. These might only serve to make a terrible situation into a slightly less bad one, but they exist for those with the desire to find them. I sincerely hope you are not in such a terrible situation, but if you are, you probably aren’t an exception. As harsh as that sounds, it’s actually a good thing. The upshot is that you can make a meaningful impact on your future luck. The game is rigged, and the game is hard. We’re stuck playing it anyway, so let’s play with our eyes open.

Luck is making chances to dig for outliers

The first hard thing about being lucky is realizing that you can just do things. This is like the extreme version of noticing you can just choose to have cake for dinner. You have ownership over your life, and you are allowed to do more things than you realize to make it better.

The second hard thing is noticing all the places in your life where you might want to do something. When I say all, I really mean all. Noticing that you can quit your job isn’t hard. Neither is noticing that you can go on a backpacking trip through Europe, that you can have kids, that you can exercise more, or that you can move to that city you’ve always dreamed of living in. People talk about these things all the time. They’re worth thinking about, but far from the only things worth thinking about. Sometimes exploring the full space of Bets You Can Make looks like staring into the abyss. Sometimes it looks like ordering a dish you’ve never tried before that sounds kind of weird.

The next step is evaluation; not all of these bets are good bets to take. I’m going to focus on a particular type of bet, which I think is fortunately both the easiest to talk about and the most useful. That class is one where there’s potential for huge outliers in outcomes. This is what that looks like as a graph:

This type of distribution is called long-tailed, heavy-tailed, or right skewed. Line 1 shows roughly where the median outcome is. If you only take a few of these long-tailed bets, then you’re likely to only end up with approximately median outcomes, and your luck will feel average. Line 2 shows roughly where the mean outcome is. The key here is that the downside outcomes are minimal and bounded, and the upside outcomes are massive and practically unbounded. If you take enough bets with this distribution of outcomes, then you’ll eventually pick up enough outliers (line 3) to bring your net outcome well above the median. This is also a theory for how diversified portfolios manage to beat treasury returns. Most individual stocks end up with lower return than treasuries, but they’re incredibly long-tailed, and it’s the few outliers that cause a diversified portfolio to outperform treasuries.

Ben Kuhn has a great post on systematically searching for outliers within specific domains, such as job selection, blog post writing, or startup investing. These tasks have common features:

• You can keep resampling from the distribution by taking new jobs, writing new posts, or investing in more startups
• It’s hard to tell how good the outlier outcomes can actually be, especially with low sample size
• Scoring a few outliers covers the cost of searching and then some (and usually a lot more than some)

One of the key takeaways is that finding success in those domains is largely a function of how well you can repeatedly sample from their outcome distribution.

My addition to this process is that you don’t have to compartmentalize it to each specific domain. Suppose you have 1 domain with a specific outcome distribution and 100 other domains that all happen to have a similar outcome distribution. Assuming you’re equally happy to get outliers in any of these domains, you can get as many expected outliers by sampling from the first, single domain 100 times, or by sampling from each of the 100 unique domains once each. In either case, you get to the same number of attempts at scoring outliers.

Long-tailed results also typically take a long time to be realized. You don’t get immediate feedback on whether the job you picked or the partner(s) you’re with is an outlier. There’s a serial nature to these selection tasks that limits the number of shots you have in your lifespan. If it takes a couple of years to figure out if the job you’re at is really exceptional, then focusing solely on resampling new jobs might leave you with only a couple dozen shots on goal in your entire life (and the longer it takes to find an outlier, the less you can reap the benefits). You can roughly double your shots on goal to find (any) outliers in life if you parallelize the process of resampling your job with resampling your romantic partner.

Why stop at two parallel processes? There are more unique domains than you realize that have long-tailed outcomes and that you can parallelize your sampling from all these domains.

Unfortunately, this is where the whole thing gets much harder. The biggest source of parallelization is going to be from decisions that are very specific to your life path and that you might only get to make a few times in your life. You might miss them, and if you see them, you might misevaluate them. There are a lot of these types of decisions, and not all of them have this long-tailed outcome distribution. Here is a quick, non-exhaustive list of things you might not get to do very often and which have varying degrees of difficulty, impact, and endorsement.

• Picking an advisor in grad school
• Getting married
• Having an expensive wedding
• Getting divorced
• Having kids or not
• The number of kids to have
• Adopting a kid
• Adopting a pet
• Choosing a career (especially a high-commitment one, like being a doctor)
• Trade on a potential Black Swan event
• Committing major financial crimes
• Buying a house
• Emigrating to a new country
• Donating a kidney
• Undergoing gender-affirming care
• Following a particular religion
• Taking a particularly unique job abroad
• Whistleblowing

These are still things that are common or high profile enough that anyone can think of them, but access to opportunities is highly path dependent.

Your full list of opportunities will look very different from mine, and it will be much longer than the list above. Your list will be unique and constantly changing, because it depends on your skills, resources, connections, and the state of the world around you. To be luckier, you must be able to frequently and accurately update your list and pick which ones to take the leap on (please don't pick major financial crimes).

A recent example that comes to mind are these beautiful, functional(!) QR codes generated by ControlNet. I’m familiar with Stable Diffusion and other generative AI models, but my brain contains basically zero knowledge about QR codes (it turns out QR codes are quite malleable). I wouldn’t have considered this to be on my personal list of opportunities, but for this person, it was and they took it. This wasn’t just a once-in-your-lifetime opportunity, this was once-in-anyone’s-lifetime. Now they get to be “that person that made super cool QR codes” forever, and no one else does. Depending on what you’re optimizing for (visibility, career opportunities, internet fame), this could be a fantastic outlier outcome for a speculative project.

Going for those one-off chances doesn’t lend itself to a clean framework of resampling on the same decision, but it does expose you to many more potential outliers over a lifetime. You can get lucky in a particular domain by understanding how to search for outliers in that domain and by repeatedly sampling from that domain. This is good, but this still limits your shots on goal. You also might not find a life-changing outlier until late in life.

You can get even luckier, and sooner in life, by being able to accurately identify all the domains of long-tailed decisions that are accessible to you and learning to search for these outliers everywhere. Even if some of them are once-in-a-lifetime opportunities, once-in-a-lifetime opportunities abound.

“But X is already pretty good”

At the job that I left, I previously held the belief that it was an exceptionally good place to work. My basis for believing this was that many of my coworkers frequently commented on it being the best place they’ve ever worked at. Surely if so many people think so, there must be some truth to it being exceptional?

I still believe it was a great place to work, but (as Ben Kuhn also discusses in his post) there’s a difference between being at the 90th percentile (great) and being at the 99.9th percentile (exceptional). If you’re at a 90th percentile company and most of your colleagues have worked at around 5 places in total, there’s a 65.6% chance that your 90th percentile company is better than all of the previous 4 places they’ve been. So being at a 90th percentile company might look like two thirds of your coworkers truthfully being able to say it’s the best job they’ve ever had, even though 1 in 10 places are better.

Truly strong evidence of an exceptional outcome looks stronger than two thirds of people with something comparable saying it’s the best outcome they’ve ever had. Evidence of an exceptional outcome looks more like zero people around you with something comparable. Without this kind of evidence, chances are that a given situation in your life is closer to a median outcome than you think it is. If so, the risk for rolling the dice again is also lower than you think it is. In general, we bias away from change, but we should try to embrace it to become lucky. In 2020, a study asked 20,000 people to make some difficult decision on their mind with a coin flip:

For important decisions (e.g. quitting a job or ending a relationship), individuals who are told by the coin toss to make a change are more likely to make a change, more satisfied with their decisions, and happier six months later than those whose coin toss instructed maintaining the status quo.

(n.b. If you're a former coworker, don't take this as advice or judgement. I haven't even worked at 5 places yet!)

What you can do to get luckier

Network more. Every new person that you develop a connection with, either socially or professionally, can be a long-tailed outcome. That might sound a little sociopathic, but the quality of our relationships predict a lot of our life happiness, so we should be more intentional in creating chances for outliers. Most people will be acquaintances, but some can be lifelong friends, romantic partners, and/or collaborators. Furthermore, a rich and healthy network might also spot long-tailed opportunities that are uniquely suited for you and bring them to you. Noticing good bets is easier when you have more bets to choose from, and having a strong network is much faster than searching alone (pay this forward!).

Be curious, too. It costs less energy to spend time learning about things that excite you. Doing great work often looks like following your curiosity to places where people haven't tread before. This can mean going deeper than anyone else has, but it can also mean going broad enough that you find a unique combination of knowledge, as with the earlier QR code art.

In the long term, it’s crucial to develop a strong probabilistic mindset and intuition, which makes it easier to reason about whether a decision is potentially long-tailed or not. This also helps avoid certain types of opportunities, such as the lottery, which is arguably long-tailed due to massive outliers, but not actually worth it. Finally, this helps identify downside risk, so that you can avoid making a decision that leads to a complete blow up. As you’re exposed to more chances, you’ll develop a better sense of what makes an opportunity good or bad in this framework, including making more accurate assessments for more unique long-tailed decisions that you only get to make once or twice.

There isn’t a lot of general advice that I can offer beyond this. The ideal situation for developing a skill is one where you can do deliberate practice, but this framework can’t be applied to the domain of luck. There are no well-established training methods, and as far as I can tell, no one officially coaches for being lucky. A lot of my personal skill in this domain falls under gut feeling, or tacit knowledge, and I haven’t thought explicitly about this enough to make that gut feeling explicit. If you’ve thought about this and have ideas, please share them. In lieu of a perfectly written guide, here are some things you might start with:

• Interview occasionally at jobs that sound interesting, even when you aren’t looking for a new job
• If you’re introverted, get adopted by an extroverted friend
• Say yes to invitations as a default
• Do more things where the worst that can happen is nothing
• Do fewer things that look like sitting at home and watching Netflix (I am guilty of sitting at home and playing video games)
• Go to random events or activities on Meetup or Eventbrite
• Start saving up some resources to be ready for opportunities when they come; remember that your resources are resources, and you should spend them (wisely)
• Don’t do pyramid schemes

Mileage is the best teacher available, so find some risk you can tolerate, and start taking some shots.

This is my first post. It was inspired by this unfinished draft by David R. MacIver and by this post on starting a blog by Ben Kuhn. I recently re-read it for the third or fourth time and finally got off my ass to try my luck with it.

Thank you to those who volunteered to read drafts: AJ, Cyrus, Felix, Ross.

Any feedback is appreciated!