How to Write a Neurips Paper [0]

a look behind the scenes

I have been planning to write this series for a while now, and the beginning of 2021 is an arbitrarily auspicious time to start. This series serves two purposes: First, as an account/guide on the process of developing original research for publication, written by someone who has done it a few times. Second, to share some funny stories of my time as a PhD student / postdoc at MIT, and keep it very personal. I remember Philip Guo’s the PhD grind being very informative when I started grad school, and I wanted to make something similar for other PhD students / researchers. However, I will focus specifically on the process of research, a universal, yet at first confusing activity that everyone must go through. Also, I want to tell my story in a more light-hearted tone, as my PhD journey had been, for the most part, fun.

paper is a consequence of your research

The paper in question is Program Synthesis with Pragmatic Communication published at Neurips 2020. A published paper is the tip of the iceberg: When you see a published work, it is always the case that a team of people have labored behind the scenes, building up a critical mass of collective expertise in their subject area. Once a critical mass of insights and expertise is reached, the team will curate the most presentable portion of these insights and turn it into a publication. Therefore, rather than focusing on the superficial publication, I want to explain the process of developing sufficient domain expertise, in which a publication becomes a natural consequence.

a look behind the scenes

The process of building domain expertise is a latent variable, difficult to infer from the paper alone. Fortunately, I have curated in this folder some research notes (~60 pages), detailing my personal journey in this process: from conception to having sufficient expertise to publish. These notes span from Oct-2020 to June-2021, interrupted by conferences and job hunts, and overall estimated 6 months efforts. Specifically, these notes are from the perspective of the first author, where he is responsible for coming up with the idea, applying it to a known problem, and asking others to join the project in order to cover the expertise gaps. This process is similar to the original research everyone must undertake toward their PhD thesis. This publication is my first foray into a new line of program synthesis research under the framework of collaborative communications, its personal importance cannot be understated, and I want to share with you how it happened.

contents of this series

This series will serve as a behind the scene look into the research process. I will go through a few pages of these notes at a time, and explain my thought processes back then. Each chapter will be very flexible, depending on what I would feel important upon re-examining these notes, and the kinds of funny stories associated with them. It will contain roughly the following topics:

1. performing voodoo rituals to procure original research ideas
2. evaluations to vet the research idea
3. patching of the gaps of knowledge toward the goal
4. making a minimal viable artifact to gain confidence
5. asking others to destroy your artifact to stay grounded
6. making and iterating on the research artifact
7. recruiting others to round out the remaining knowledge gaps
8. working with the team to collect experiment results
9. writing the paper towards publication
10. writing the paper rebuttals
11. presenting your finished work to others

please check the chapters list at the end for a list of up-to-date topics :)

a warning

This is a single roll-out of my own research process, which is both biased and likely inappropriate for direct applications to your research. Please enjoy responsibly and do not overfit to it.

stay tuned !

I will be updating this series once every couple weeks, depending on how much free times I have. I will be announcing the new updates on my twitter, so if you can follow it that’d be great :)

thanks for reading and high five!

— evan

chapters list :

• [0] overview : a look behind the scenes
• [1] getting a research idea : series of voodoo rituals
• [2] vetting a research idea : be honest and disciplined
• [3] acquiring knowledge : set a goal first
• [4] building a prototype : <under construction>