Expectation setting

Caveats:

• This is targeted at MARL researchers (with an emphasis on transportation researchers in MARL) but is somewhat generic except for some of the specific expected skills.

It is quite hard to understand what a successful first year or two of graduate school looks like. This document is intended to help you set a standard so that you are neither too hard on yourself, stressed out about meeting some imaginary expectation that you could not possibly achieve, nor too easy on yourself because you misunderstand how much hard work is required. On the other hand, if you're struggling to hit these expectations we should have a chat to figure out what's going on and find a way to help you! The goal is for you to succeed and these expectations are intended to be a useful guideline, not a source of stress.

Year one

In your first year you are taking classes and so you should have the following baseline expectations for yourself. You can easily go above and beyond them if it turns out to be too little but you should be happy and proud if you hit these expectations and not stressed out if you haven't done more.

• Time: You should be spending about 15-20 hours on research per week. Inevitably, not all of this is going to wind up being productive, so don't be hard on yourself if you're losing time to say, navigating a codebase or figuring out how to run code on the clusters. That's being productive too! This is also an internal expectation, I'm not going to be clocking your hours; the expectation is that you are in graduate school and are self-motivated.

• Learning: A lot of your learning is going to be course-work but one thing to understand is that the coursework is just getting you the basic level of understanding you need. There's a surface level understanding you get out of a course and a deeper understanding when you really know something and it's important to learn to distinguish the two; there's a big difference between being able to look something up when you need it and having it deeply internalized and available at your fingertips. In terms of practical outcomes I think at the end of your first year you're in a good spot if you have
  • fluency in an ML library of your choosing (preferably pytorch or jax. The libraries are increasingly similar so if you figure out how to write pytorch code you’ll figure out the others without too much trouble.)
  • comfort with a simulation environment that's relevant to your research
  • able to run experiments on the cluster with ease
  • a good understanding of the topics in Sutton and Barto, taken an optimization course, a controls course
  • read most of the papers on my recommended reading list. This'll get customized a bit based on your interests!
  • find a seminar series you like and attend it religiously. This has a few benefits:
    • You'll meet new folks and get to know about their research.
    • You'll get comfortable with the process of what it feels like to go from not understanding something at all to seeing patterns to getting it. Warning: If you're not understanding the seminars for the first few months that's normal for most people (seminars are generally targeted at people with familiarity with a particular research area and will skip over some introductory information) and so you shouldn't get discouraged. If you’re still not getting anything out of the seminar after three months or so, shoot me a message and we’ll chat about it.
    • You'll hear about some research that might inspire a direction for you.

• Paper output: In your first year you're going to be working on one paper that we outline and design together; it'll likely be a combination of your stated interests and whatever else the lab is focusing on. This may be a paper on which someone else is the primary author and you’re just helping out to get your feet wet. I'll be pretty involved in helping you design this though your input is expected and I expect the project to evolve in unexpected directions due to your insight. Note: a graduate student failure mode is thinking that you can just focus on your coursework until you reach some presumed level of mastery and only then will you be ready for research. This is not the case, research is its own skill and you can only get better at it by actively doing it.

• Fellowships: You should be applying for 2-3 fellowships this year; we can discuss which ones exactly. Getting one of these will free you up to pursue your interests in a way that is less restricted by what my funding looks like and can be a really straightforward way to make your PhD even more enjoyable.

• Research notebook: You should be keeping a thorough research notebook and be giving some of the suggestions in personal rules of productive research a chance. Some of them won't work for you but I expect you to give it a try! I genuinely believe that you cannot think deeply without writing; you may come to disagree with me about the choices outline there but I expect you to give it a chance. In the end, I'd love to know what worked for you and what didn't.

• Meetings: We’ll be meeting weekly most of the time, occasionally bi-weekly when things get hectic. I can’t read your mind and it’s not super-productive for me to sit and read your research notebook with you so it’d be good if you have your research in some kind of digestible form, whether that’s a latex document you share with me a day ahead of the meeting (watching me sit and read a document and react in real-time is a waste of your time) or an informal presentation to talk through where you’re at. Over time we’ll refine what constitutes a productive research summary.

Year Two

• Problem list: You should be starting to formulate a set of problems you'd be psyched to spend 2-3 years tackling and be refining this regularly.

• Fellowships: If you have a fellowship by this point, great success. Otherwise, apply again!

• Learning:
  • You should be extremely comfortable in a language of your choosing (probably python). If we're working on hardware you should be getting comfortable in C++ as well or something else that can run in real-time.
  • You should be taking some advanced coursework on more specific subject areas and using a few courses to shore up any key missing bits of your knowledge that you've found. I can't make generic recommendations, at this point it's going to be specific to you though we should discuss it!

• Papers: we should have wrapped up a paper in the first year. At this point you'll probably have a ton of ideas for projects and we should be refining a project that's a mix of my interests and yours with a slight lean towards your interests. A reasonable expectation is to write at least a paper a year going forwards, sometimes two depending on the background you come in with.

Year Three

Here the direction setting moves way more to your side! By now you should have a clear sense of what topic you'd like to focus on for the remainder of your PhD. You should have a well-developed toolbox and be ready to start exploiting it (though of course the learning doesn’t actually ever stop)! By this point you're setting the direction, not me. The only expectations at this point are:

• You have a clear milestone for what problem you'd like to throw yourself at for 2-3 years. You should have thought quite deeply about this by now because backtracking from it will be tricky. It's not impossible so don't stress out if you discover that it's a dead end or that you hate the problem / topic but it's better if you have thought this through fully by now.

• This is a great time (or earlier) to have gotten involved with the organization of a seminar series. This’ll get you introduced to a bunch of leaders in your chosen field, grad students with shared interests, etc. Doing this was one of the most useful things I did during my PhD.

• Learning: one trap that can happen here is to focus 100% on paper writing and forget that you have to keep learning or you’ll be stuck with the same skills forever. There’s some good advice here in An Opinionated Guide on ML Research and Principles of Effective Research

Other advice

Some other useful pieces of advice that I like on expectations for graduate student behavior: Anant Sahai

How to be a good ML researcher by John Schulman: An Opinionated Guide on ML Research

Some good advice on how to balance explore and exploit by Michael Nielsen: Principles of Effective Research