0:00
/
0:00
Transcript

Breaking into AI Research - Unblocked!

Stefano is about to finish a master's in data science. What comes next?

Today I chatted with Stefano about what to do after he graduates. He dreams of one day running his own research lab where he can explore the intersection of AI and cultural evolution — but he's convinced himself he needs years of preparation before he's worthy of even having conversations with the researchers already doing this work.

Transcript

Note: This transcript is AI-generated and may contain mistakes!

Andrew: This is another episode of the Unblocking Podcast. This week I have Stefano here. What are we talking about today?

Stefano: I'm trying to figure out what I'm going to do after graduating.

Andrew: Classic. Many people have this issue, so this will be a good one.

Stefano: I'm finishing my master's in a month and presenting my thesis at the end of August. I did my master's at EPFL in Switzerland in data science - basically machine learning and computer science. My bachelor's was also in computer science, so that's my main field of expertise.

Through the master's, I did research in reinforcement learning and computational social science. Now I'm writing my thesis at a startup that does search engines for legal firms. It's not that I have no idea what I want to do - there are some constraints. I know I'd like to be in machine learning, maybe reinforcement learning related fields.

I also have interests outside the technical field - social science, psychology, cultural evolution. I'd really like to find something that combines these two. I'm interested in startups and feel I have an entrepreneurial attitude, but I've been exploring ideas with a friend and figured out that I don't want to do a startup just for the sake of it. I really want to do something that excites me and that I want to work on for the next five to ten years.

My best bet would be to join an industry research lab. I don't want to do a PhD because it feels lonely and you end up optimizing for citations, which I don't really care about.

I realized recently that one thing that would combine all my interests is to open a research lab, something like DeepMind. But I think that's too ambitious right now. I wouldn't feel confident doing something like that. So I'd probably try to find a job at a place that's similar. The idea of working in a group to solve research questions that are also applied and not just theoretical for papers excites me.

Andrew: Is it fair to say that your ideal job right now would be to work at somewhere like OpenAI, Anthropic, DeepMind - one of these AI research labs?

Stefano: I think so. The only thing missing from something like that is that I'd like to have my own thing - the project that I bring and where I decide the direction. I think that might be missing there. But even if it's not OpenAI or the most famous one, something in that direction would probably be interesting.

Andrew: I'm going to start asking a lot of questions to get data. How much money are you making right now?

Stefano: I'm doing an internship making 3,500 francs per month.

Andrew: Do you think that's going to convert into a full-time offer if you want it?

Stefano: Probably not.

Andrew: How much runway do you have if you didn't have a job?

Stefano: I could probably have four or five months of runway.

Andrew: Would you want to stay in Switzerland or move to the US or another country?

Stefano: I'm fine moving anywhere - Europe or the US. If I could choose, I would stay here right now, but it's not a strong preference.

Andrew: Do you have people you really want to work with?

Stefano: I have a friend I've been exploring ideas with for the last semester and I really like working with him. But right now there's no clear alignment. I told him about the research lab idea but he's not too into that topic. We were more aligned before, so I feel like we're not aligned now. But I would really like to work with him specifically.

Andrew: Are there any groups doing the sort of work you want to be doing?

Stefano: Very recently, in the last two weeks, I got interested in open-ended AI. I saw some groups at DeepMind and OpenAI doing this. My interests shift quickly, so I'm not sure that's the thing I'm going to do, but there are people doing similar work.

Andrew: How good do you think you are at what you do compared to how good you could possibly be? Like as a percentile.

Stefano: 30% compared to what I could probably reach, which is 100.

Andrew: How good do you think the average person at DeepMind is on that same ranking?

Stefano: 50, I think.

Andrew: Are you happy?

Stefano: Overall I'm happy. I see improvements.

Andrew: Does work feel easy or difficult and stressful?

Stefano: Not stressful. I often feel like I would like to do something else. I have many other interests and find it hard to get excited about stuff that gets imposed on me. I always feel like I would like to escape and work on other stuff - reading other things, implementing other things - but it's not what I'm working on.

Andrew: Do you have a good concept of your attention? Where is it going as a percentage breakdown in a week?

Stefano: 50% on my current internship, 20% on stuff I read online including Twitter - scrolling through that, watching videos, trying to explore topics that seem interesting. Another 20% on relationships like my girlfriend and other relationships. Last 10% on sport, music, and free time enjoyment.

Andrew: Can you break down that 50% work time?

Stefano: Probably 5% is meetings. The other remaining half is code - writing code, running experiments, trying new stuff. Right now the other half is writing my thesis, which takes a lot of time. Before that, it was mainly writing code, reading papers, coming up with new ideas to improve on.

Andrew: What are you improving? What's the objective?

Stefano: I'm trying to improve the search engine they have there. It's a research task - I need to read papers and figure out how to apply stuff from there to our product.

Andrew: If you could teleport into a world where everything was going really well, what would you be focusing on instead?

Stefano: I'd like probably 60% of my attention focused on research. Right now it's mainly an exploratory phase. There are different ideas that really excite me and I'd like to look deeper into them. Through the years I went through different topics I was interested in, and I asked ChatGPT what fields represent this. It came up with cultural evolution - the idea of how intelligence emerges from culture and how culture shapes itself. I'd like to spend time looking into that and see how it can be applied to AI, maybe how AGI could be found from that.

I'd like to spend time just reading, maybe making small experiments, and especially interacting with people who are experts in this field.

Andrew: Right now the block on that is primarily you spend so much time on work, or is the blocker more that you don't feel ready to talk to experts?

Stefano: Work is one factor, but the main one is that I don't feel ready. I don't feel like I have enough knowledge in the field to go up to them and talk. I'd probably need more time to read through stuff and feel like I could make meaningful questions before approaching them.

Andrew: How many hours of reading and thinking about questions would you need before somebody would enjoy a conversation with you?

Stefano: If I pick a specific person I want to talk to and look at the topics they're interested in, I think in 10-15 hours I could be prepared for that specific person to have an interesting conversation. That's a rough estimate.

Andrew: I think that's a pretty high bar. My instinct is people enjoy talking about their work a lot. You could probably have conversations people would enjoy with a lot less effort. If somebody came up to you and talked about your work, how many hours of prep would they need? 15?

Stefano: Probably not, no.

Andrew: If you have to do 15 hours of reading before you can have a conversation with other researchers, then you can't run a research lab because you're going to be working with a lot of researchers. That doesn't mean you shouldn't do those 15 hours - I'm sure you'll learn a lot. But one pathway toward getting closer to research lab stuff is just having more conversations with researchers.

Do you journal? Do you have a writing practice?

Stefano: I write quite often.

Andrew: All you really need to do is journal about what questions you think somebody maybe hasn't heard before, or they have but you personally would enjoy the conversation so much that you think they would appreciate that enjoyment.

Stefano: Do you think recording conversations and opening a podcast just for the sake of that would help?

Andrew: I think of it less like starting a podcast show to become popular. The reason I record them is because it improves having an artifact - it allows you to go back and review it, remember your life, remember the conversations you have and what people have been thinking about. Those artifacts can be useful in different ways.

You'd be surprised - even if just one person (your future self) appreciates the recording, it's useful to have.

Andrew: Realistically you have five to six months. I don't think that's enough time to start a company. Selling stuff is really hard, especially for people as smart as you. You're going to think you need something ridiculously good before you can sell it, and that's going to make the five or six months more stressful.

You should have a clear plan if you want to spend that money on an independent path. That plan should not be completely indie. Maybe spend three months trying to raise, or applying to accelerator programs, or becoming a fellow somewhere. You should join some structured program that will help you because that's about the amount of runway you have.

I don't think you have enough time for completely free exploration if you want that to also sustain you. On average, for people doing complete exploration to be able to pay their bills and not feel stressed, it's probably two to three years.

Stefano: That makes sense.

Andrew: What feels fun? What are some options that seem really cool?

Stefano: The most fun thing right now would be to join a research lab in the industry.

Andrew: And having these conversations with people about social and cultural evolution. Do you have a blog?

Stefano: I really like writing. I tried to start it in the past, wrote some articles, but never felt like they were good enough. I was also thinking I should start being more active on Twitter or LinkedIn.

Andrew: Where do you think these people studying cultural evolution are? Can you name some of them?

Stefano: It's not just cultural evolution, it's also intersecting with AI. Universities, research labs. I think I didn't look that much into it, but I could name some. I should definitely start reaching out more to people I think are interested, trying to get conversations because they're not even that famous and popular.

Andrew: What would you call this grouping of people?

Stefano: AI researchers, but I feel like most AI researchers now are very narrow in their thinking. They don't try to see the long-term. So more open-ended AI researchers that also try to think long-term about what's still missing and have a bit of philosophical and high-level picture of what's the situation right now.

Andrew: I wonder if you can come up with a punchy name for this and just start to identify as one of these people. These are just my people - me and all the other open-ended AI researchers. When people ask what you do, you say "I'm an open-ended AI researcher. That's what I spend my free time doing. Those are the conversations I have. That's what I write about on my blog."

You don't care about the search engine thing. There's no reason to identify with it. It's what you're doing, but what you really identify as is this other type of thing.

This will help you navigate social space. There's a network you're connected to already - through this podcast, through your Twitter, through these people you're imagining. You're already part of it. You might be at the edge of the graph and want to traverse it and get closer, but you're in it regardless of how many steps removed you are.

If you start saying that's what you are - I'm in this network, this is what I'm doing - people will start routing you around. They'll say "I should introduce you to my friend Ivan" or whoever.

Stefano: This feels very convincing. In the past, I struggled a lot with defining what I'm interested in. Only recently I tried to go deeper and categorize it. It's one of the big problems because if you don't have a clear definition when you start talking with people, you can't give a two-hour talk on all the little things. You need to summarize and condense it.

Andrew: Think of these things as queries. A blog post is a very long search string. When you put it on the internet, it starts to run a very long-running search process that will send things to you over time. A tweet is a relatively short search string that goes out into the network and searches for replies.

What do you want to search for is the question of the internet. You'll need to do active search by publishing your search questions into the network. The things you want to search for are mostly stored inside people's minds rather than already existing in text. So you can't use Google search to find it, but you can use Twitter and Substack because you can get inside people's minds by running social queries rather than text-based search queries.

Stefano: Did you just come up with this now?

Andrew: No, there's this guy Henrik Carlsen who has an essay about this. It was very influential to me because I started to realize every conversation I have is an opportunity to run some kind of search on the universe, or on the network of all people's minds.

Andrew: Roughly that's the direction I'd take. It sounds like you're feeling pretty good - you're pretty happy, pretty healthy, relatively happy with your life and relationships and how you're spending your time, except you're just not in the right subfield yet. You haven't gotten there.

Regardless of what you do next specifically, are you worried about getting your next job? Are you worried about your ability to pay bills or work somewhere you like?

Stefano: I'm a bit worried. It's something that stresses me and gives me anxiety because it feels very uncertain and very important. It's okay if I don't get it right at the right time, but yeah.

Andrew: Those are the two sides I would focus on. On one side, start doing the social work because the reputational stuff takes the longest. Think of this as social entrepreneurship. Start identifying with these open-ended researchers and having conversations.

Don't pressure yourself to do any particular thing. Just realize you need to be doing more of all the things. You need to write more on it. If you don't feel like writing, just DM people. If you don't feel like DMing people, just send a tweet. If you don't feel like sending a tweet, just journal about it. But put yourself closer into this open-ended research bucket as something you identify as and actively contribute to, even in a small way, by participating socially.

Given that you know you're smart and you care about it, all you have to do is show up and you're naturally going to do good work. That's going to take a year or two, but you can make it go faster. If you're ambitious, you can make big progress in a couple months. But building a reputation in that group inevitably takes time. People want to see that you consistently contribute and don't burn out.

Andrew: On the financial/career angle, spend some time figuring out what's a step that brings you at least one step closer to the open-ended research thing and opens up options for you, making the next decision less stressful with less anxiety.

Do you have any top-of-mind options for career steps? Accelerators, structured programs, research labs you want to join, or jobs that pay a lot of money so you have more years of runway?

Stefano: I could join a research lab at my university. That wouldn't be too difficult and they'd hire me for six months or a year. I'd be doing research, probably on something aligned to what I'm interested in. On the side, I could explore and meet more people.

There are smaller research labs here in Zurich - there's one that does reasoning. I could try to join them as well. I tried to reach out but still haven't, and I should try to meet them because I don't know anyone there.

Andrew: At the moment, that's a relatively short list. I'd try to make the list longer and more ambitious. You talk about DeepMind a lot - just put DeepMind on the list. They should be on there and you certainly should apply. There's no world where you shouldn't.

Even if you feel like there are people at DeepMind doing better work than you and using their potential more, that's incremental progress no matter what. There are smart people there probably thinking about the same thoughts you're thinking about. You might feel it's an imperfect fit, but as long as it's incrementally progressing you, you'll probably meet collaborators there who feel the same way.

Make sure the list is as long and ambitious as you can make it. Then take the most ambitious item that feels like the hardest - surely there's no way I could do that - but feels the most exciting to you.

Start breaking down what would be required to do that specifically by reaching out to people. You value that thing - you find it deeply exciting and it's a very ambitious goal. People love to help other people achieve ambitious goals.

Start reaching out to people like how you reached out to me: "I have this really ambitious goal. I feel like you're somebody who might know how to achieve that goal. I could really use some advice because I'm not exactly sure what to do. I can tell you about things I've been trying, but I would really appreciate your advice."

If you send a message like that to even three people, probably one will respond because everyone likes helping people. Then you'll get direct advice on how to achieve your most ambitious goals. Rather than thinking in your bedroom about whether this is possible, you don't have the information required to know that. These other people have a much clearer picture of the environment.

Stefano: No, it feels very exciting. I'd like to do that.

Andrew: I'd definitely do that. Those connections - you're going to be able to talk to those people about your ideas. It's all connected.

I tend to encourage people to do this with their most exciting and ambitious goal during phases where they're considering stuff, because many people count out their most exciting goals and try to pick further down the list, which is the exact opposite approach of how you make good career progress.

You take the most ambitious and exciting goal and use that as motivation, as fuel. That's the thing you tell everyone about because they get excited about that too. It's more combustible - it has more energy in it. You use that and those conversations as fuel to get as far as you can.

Realistically, you might not make it all the way to the goal, but because of all the conversations you're having that are fueled by the motivation of the goal, you'll get way further than if you just went for something less ambitious.

Stefano: Makes sense.

Andrew: I think those are some good next steps. Anything else that would be worth talking about? If you journal on those things, you'll at least have some good ideas. If you want my advice on next steps, you're welcome to reach out to me privately.

Stefano: Cool. Thank you so much. This was super helpful.

Andrew: Perfect. Take care and best of luck.

Discussion about this video

User's avatar