Compilation of Advice for ML PhD Students
I need guidance as transitioning directly from an undergrad at Minerva University1 to a PhD student at Brown University is already difficult enough, let alone being in a competitive machine learning field. Hence, I put together a compilation of recent resources for budding researchers like me.
Before I started my Ph.D. program, I frantically searched for advice for incoming ML PhD students. What are the right things to do? How to be successful? Any mistake to avoid? I have come across many great advice posts/threads/videos, and I notice that many recent ones are scattered around the internet. These recent posts usually paint a more up-to-date landscape about ML research field, which has observed an unusual exponential growth in publications and a dissatisfactory review process.
Therefore I have compiled these recent resources, which are written or revised after year 2017, here in an alphabetical order.
Resources
Alex Irpan - The 5 Year Update on Skipping Grad School (and Whether I’d Recommend It)
This post puts the question “What to (really) get out from a PhD program” in perspective by comparing doing research in industry and in an academic program. By knowing the differences, I can focus on opportunities unique to my PhD.
Arun Kumar - The Secret Lives of Millennial CS Assistant Professors (Part 1)
Even though this Medium article targets Assistant Professors as its audience, I benefit a lot from reading about how Arun chooses problems to work on and to collaborate on. I particularly like how he values intellectual independence and takes advantage of academic freedom. Besides, his takes on research dissemination and freedom of speech inspire me to be more outspoken.
Bill Freeman - How to do research
I’m adding this as a third-year PhD student because I think the advice of “in graduate school, it’s the hard workers who pull ahead.” and “you can build up intuition about what matters with simple toy models” really stands the test of time.
Christof Monz - So, you want to do a PhD…
This is one of my most favorite blog posts for the fact that Christof based his advice on the end goal of “finishing the PhD thesis”, so his tips are very actionable. On top of the things to do, he also detailed the situations to anticipate at different stages of PhD education.
Diana R. Cai - Tips for New PhD Students in Machine Learning
This blog post touches on many skills that PhD students need to equip. One thing I like about Diana’s post is its comprehensiveness – it even includes a section about working from home!
Dmytro Mishkin - How to navigate through the ML research information flood
I love this information-packed deck of slides because it talks about different strategies of reading ML papers, metrics to evaluate quality of papers, and most importantly, what to do after reading the papers. My favorite part is Dymtro’s breakdown on task understanding.
Eric Gilbert - Syllabus for Eric’s PhD students
I believe Eric’s document is applicable to every PhD student as it dives deep into the expectation of an advisor, the realities of the research job market, the daily schedule of a researcher, and many more. I particularly like the section discussing how many projects to work on concurrently as I fit his description of an anxious student.
Generalized Error - My Machine Learning Research Jobhunt
What to expect during an interview for research position in industry? Unlike its software engineering counterpart, the interview preparation resources for research role are scarce. This post, from an anonymous writer, attempts to shed light on this topic.
Heng Ji - The Art of Doing Good Research
Heng has a unique take on internships and collaborations. As a person who loves collaborative research, I feel great knowing that collaborations will come naturally. When you read this article piece, make sure to spend some time thinking about “how we can have it all”.
John Schulman - An Opinionated Guide to ML Research2
I believe that PhD is a journey of honing research taste, and John concisely outlined different possible ways to achieve it. He also provided a heuristic for gauging if we are changing our research projects too liberally - we should have many small dead-ends from exploring new ideas and a few projects from start to finish.
Maithra Raghu - Reflections on my (Machine Learning) PhD Journey
I specifically love how Maithra talks about actionable tips to deal with the feeling of being stuck and the importance of community. Her framing of “Why PhD?” is succinct: become an independent researcher with a rich (articulable) research vision.
Michael Ernst - Advice for researchers and students
Michael has compiled a collection of his advice for students, covering important topics such as attending conferences, applying for internships, and using version control. Do I mention that he even included advice for running a conference committee meeting? Whether being an early-stage or an experienced researcher, almost anyone can learn something from him.
Peng Billy Xu - Navigate Through the Current AI Job Market: A Retrospect
This post is not necessarily oriented for PhD students, but Peng’s experience with interviewing for current AI jobs is helpful for us who are looking for opportunities in the industry. I particularly like how he discusses and breaks down the AI job spectrum.
Ronald T. Azuma - A graduate school survival guide: “So long, and thanks for the Ph.D!”
This post is like a graduate school handbook dedicated for CS students. It got me thinking about my views about PhD (it’s a job, period) and the qualities of a successful researcher.
Sebastian Ruder - 10 Tips for Research and a PhD3
Sebastian’s advice is brief and to the point. I resonate with his advice of working on two projects, where one of which is high-risk-high-reward. In fact, the reason this piece of advice caught my attention is that my PhD friends had advised me to only work on one project.
Shomir Wilson - Guide for Student Research
I particularly like the section “Common Obstacles” in Shomir’s blog because it acknowledges that many problems are just parts of the research journey. For instance, the first obstacle “slow start” perfectly describes my first summer semester as a PhD student. I like that Shomir went into details on how he handled them.
Swapneel Mehta - Interview Advice for Research Internships in Data Science
Incredible resource of Swapneel’s interview experiences at big tech firms for the research positions. He also gave another compilation of advice in this Tweet.
Tom Silver - Lessons from My First Two Years of AI Research
Tom’s recount of his learning is in accordance with other blog posts here (although the specificities may differ). I particularly like Tom’s analysis of the fundamental motivations of researchers (and their papers). In my opinion, his point about how these motivations shape the research papers goes beyond publications – it extends to the reviewing process, the public perception of deep learning, and even the Twitter clamor about AI.
Yannic Kilcher - Machine Learning PhD Survival Guide 2021
Yannic’s video was released just in time when I started my PhD. The Venn Diagram for topic selection helps me objectively evaluate whether a certain research topic is suitable for me.