From reviewer to author (academic practices)

.

Part I

Something quite amazing happened last year. About this time, or actually a little later, I reviewed A Paper for the 2016 CHI conference. I thought the topic of the paper was extremely interesting, but, the authors hadn't read The Book. They spent a lot of time inventing the wheel again and again and much of it was due to not having read The Book. Another problem with the paper was that while the topic was very interesting, it was hard to see that the authors had managed to connect the topic to topics that were relevant to the CHI conference in a satisfying way. It was an interesting topic and an interesting paper, but was really CHI the right venue for the paper in its current form? My "verdict" as a reviewer was that the paper wasn't ready for CHI in its current form, but I also gave very detailed and exhaustive feedback to the authors on how to take the paper further. And that was that, or so I thought.

Some months later I was over in the US for a public event as well as for planning the then-upcoming 2016 Computing with Limits workshop (conference) that I would co-chair together with Barath Raghavan and Bonnie Nardi. After the public event, Marcel Pufal, a ph.d. student that I knew very well from my sabbatical at UC Irvine (we had shared the same open lab environment for half a year) approached me. He started by asking "Is there perhaps a chance that you recently reviewed a paper that was submitted to the CHI conference about topic X?" I was floored because I couldn't understand how he could know that. Isn't the information about which papers I have reviewed supposed to be secret? It turned out Marcel had written The Paper and he had drawn the conclusion that I might have been reviewer #3 due to two clues. He felt he recognised certain ("Swenglish"?) phrases or turns of words or something (could have been a consistent error when I speak/write English due to it not being my first language), and, he also vaguely recalled that I at some point had recommended he should read The Book.

The bigger surprise came when he invited me to become a co-author at the next attempt at rewriting and submitting the article in question! I've never been recruited as a co-author in quite that way before. The only reason I don't divulge the actual topic of the article or who the other two co-authors are is that the article we wrote together currently is under review. It might be the case (especially if the article is accepted) that I will go back to this blog post at a later point in time and de-anonymize the text above.

I've been pondering the fact that I was recruited as a co-author in this very way. It's of course very flattering to put some effort into writing a helpful (but tough) review and then get "instant feedback" and credit for it by getting an offer of co-authorship in return. In this particular case I already had a relationship with the paper authors, but it could theoretically happen that you reach out and get in touch with someone you don't know.

Part II

Some time later (end of April 2016) I submitted a paper to the ICT4S conference. I got four positive reviews back in mid-June, but Reviewer #2 stood out in terms of having provided very detailed and helpful comments. I couldn't follow up on all the feedback the reviewer gave me due to several reasons:
- I didn't have the time to incorporate/act on all the feedback before the deadline for the camera-ready version.
- I would have had to read up pretty extensively in new/bordering areas to be able to fulfil the reviewer's requests.
- The focus would then have shifted and my paper would have become a different paper. I'm not sure it would have been the paper I wanted to write to the conference, but perhaps instead some other paper.
- The paper would have overflown all boundaries. Acting on all the advice and suggestions would have required an article-length treatment (rather than a shorter papers-length treatment) of the issues in question.

So I followed some of the suggestions I got and put other suggestions/advice aside. They might come in handy if I wanted to extend the paper into a journal article. Which got me thinking.

So a few weeks after the conference, I sent this e-mail to the program chairs:


Hello ICT4S program chairs.

I got two very thorough reviews of my ICT4S paper and especially Reviewer #2 had a lot of very good comments (see below). The problem for me was that:

- I had limited time to take the reviews into account and incorporate them in the paper

- Had I done a thorough job of doing so, it would have become another paper or a paper that would have been way too long.

In the acknowledgements, we credit the reviewer(s) as follows: "We would like to thank an anonymous ICT4S reviewer for the insightful comments and also take the opportunity to express our regets that we could not follow through on more of the excellent advice due to time constraints”.

As it is, I am thinking of extending the paper into a full length article and I would like to discuss this with Reviewer #2. If we get along and can manage to create a shared vision of what such an article could look like, I would then like to extend an invitation the the reviewer to become a co-author. I know this is irregular but I have recently myself been recruited to become a co-author in just this way.

So, I wonder if you could forward this mail to the reviewer in question?

Yours,

/Daniel

Only a day or two later I got an answer:



Hi Daniel,

The ICT4S organisers have passed on a message from you.

I AM REVIEWER #2!

...

I would be more than happy to co-author on a future version of the paper.

And the mail "of course" came from someone I already knew, namely Oliver Bates at Lancaster University. We haven't actually done anything more yet but that's primarily because we both worked frantically towards the CHI conference deadline up until only a few days ago. Also we're not really in a hurry. But I do look forward to starting up a writing project with Oliver. We know each other - we are actually organising a workshop together at the end of October - but we have never written anything together before. Also, I found the review very useful and could also see that the reviewer (Oliver!) knew about topics that will come in handy when we start to work on extending the paper into an article.

Analysis

Isn't this just the coolest way ever to recruit a co-author? It's like going on a blind date where your best friend promises you will hit off. It's actually probably even better since there is not outside manipulation, pressure or suggestion behind the match-up (best friend or no best friend). And it makes for a good story too!

What is best though is that these examples harbour the potential to change the practice of reviewing paper. What if people knew that a really helpful review could (sometimes) lead to a gig as a co-author of the paper in question? Wouldn't that keep reviewers on their toes and with the added beneficial effect of generating truly excellent reviews a lot more often than today? And vice versa, couldn't we then sometimes encounter the very best co-authors for our papers in the pool of reviewers of an earlier version of the paper?

Another reason I like this idea is because it is very democratic. You would act solely based on the quality of the review - rather than making a decision that were based on the title or the reputation of the reviewer!

I hope I will have inspired at least one or a few readers to go ahead and recruit a reviewer (or at least reach out in an attempt). If so, I'd like you to go back to this blog post and write a comment or alternatively send me a mail and tell me about it! Perhaps this is the start of a new academic practice? Perhaps it's not really a new practice - perhaps you have already experienced it or perhaps this is not at all that unusual in some specific academic field? Do tell!
.