In my first post in this three-part series on the biomedical system, I discussed the fundamental flaw in using ‘the best’ indiscriminately, without acknowledging that, more often than not, there is no universal, Platonic ideal. In this post, I’m going to explore how choosing our ranking system requires a deeper conversation about where it is that we, as a scientific community, want to be in twenty or thirty years.
In some ways, choosing between scientists is a lot like judging the Westminster dog show. Essentially, judges have to find a way to compare apples and oranges. And then rank the huge diversity. At Westminster, the judges have to say that this Siberian husky  is better than that beagle; at the NIH, they have to say that this scientist should be funded while that one should not be.
This is a hard problem. Really hard.
So how could we rank scientists to decide on ‘the best’? Well, there are few approaches. One class of approaches is to consider only the scientist, and judge him/her by one of several metrics, such as:
- Efficiency (e.g. average cost per publication);
- Scientific significance;
- Technological innovation;
- Commitment to teaching;
To me, though, this straightforward approach misses the most important  difference between a dog show and the NIH: while Westminster chooses one winner, the NIH chooses to fund many scientists. That is, the NIH funds a portfolio, and so it must consider the strength of an individual scientist in relation to the other scientists.
You would be completely justified in stammering right about now, Doesn’t this approach just make things even more complicated?
Well, yes and no.
Yes: Now there are a lot of moving parts:
- How many scientists should be focused on training graduate students?
- How important is efficiency in respect to productivity?
- How important is a diverse (in all senses of the word) community of scientists?
- What is the right balance between foundational science and translational research?
- What is the right mixture of high-risk/high-reward and low-risk/low-reward projects?
And no, because now we can think about how the funding decisions are shaping the scientific community. Not just the scientific community of today, but also the scientific community of the future.
It’s true, this aspect puts a lot of pressure on us. However, this very fact—that our funding decisions today affect future science—also gives us a toehold into the very tricky problem of ranking apples and oranges.
The easiest way to untangle this issue is to take a step back and flip our perspective around. Rather than decide on the specific metrics for the portfolio and then watch how that affects the community over the next decade, we should instead use an approach of deciding where we want to be, as a community, in twenty, thirty years and base our metrics upon that destination. That goal, that vision, that ideal is our North Star.
After all, if you don’t know where you’re going, how can Google maps decide on the best route? And if you don’t know where you’re going, how will you know when you’ve arrived? And how on earth will you ever know if you’re off course?
I don’t presume to have the perfect answer, but I’ll put out my utopian scientific community anyway, one that I hope we can move towards during my career as a scientist. Here, we would:
- Be diverse (gender, racially, geographically, area of interest);
- Embrace the power of technological innovation;
- Encourage risk-taking;
- Support interdisciplinary interactions that often drive discoveries;
- Evaluate projects while keeping a long horizon in mind;
- Value scientific quality and rigor more than immediate scientific impact;
- Consider scientific efficiency, rather than just productivity;
- Understand the give-and-take relationship between foundational and translational (i.e. applied) research;
- Reverse the trend of the ever-increasing length of apprenticeship;
- Promote real scientific literacy in the public.
This, of course, is just one answer to the overarching question of ‘where do we want to go?’ I have no doubt that other scientists (and the general public and politicians) will give different answers.
But let’s have the conversation.
Let’s recognize that until we tackle this question, talking about ‘the best scientist’ is virtually meaningless. Let’s admit that we are doing our own sort of evolution experiment each time we fund some scientists, and not others. Let’s acknowledge that the community and the colleagues and the scientists of the next decade are the ones that will best flourish in the environment we set today.
Scientists, I think it’s time that we had an honest discussion about where we want to go. And then we can decide how to get there.
 I should be upfront here: I don’t have all (or any) of the answers. More specifically, I don’t know how, in this era of harsh, Draconian funding lines, to choose between all of the extraordinary science that is going on. I don’t know how to deal with the fact that, even if we had a perfect set of judging criteria, many scientists who should be funded are not going to be. And I really don’t know how to make that okay.
 I’m really much more a cat, than dog, person. But I do have a pretty big soft spot for Siberian huskies.
 At this point, let’s not even worry about how to actually measure these traits.
 Well, I guess, that the contestants at Westminster are dogs and scientists are human is a pretty important difference too. Details, details.
 Studies have shown that diverse workplaces are more productive. But, for me, the argument in support of diversity is a moral one of basic fairness.