Community type list for evaluations

Question

It was suggested to me to make a meta post first and see the community response for this type of question.

I would like to know what types of quantitative evaluations exist for researchers. I do not mean how students evaluate, but how departments evaluate the staff/faculty that does research. For example, is there some Impact Factor publication score that is combined with author position to give some number as a way to evaluate all the academics together.

The institute I work with is 'interdiscipline' as in, disciplines from bio,biochem, nano, robotics, UX, etc. are all in the same building and administratively similar. So the institute is looking for ways to evaluate researchers quantitatively, which is supposed to be 'fair' (but I think it is more just transparency).

Currently we have some 'total possible points', then it breaks into categories like research and outreach. Research is broken into journals, grants, books. Then outreach is broken into lectures, invited talks, internal tours, media coverage (if your in newspaper or news).

Some example of the points are, journal is X amount of points, if you are first or last author, it is 70%, if you are the only author it is 100%, and if you are middle it is 30%. Then, if the impact factor places it in the top 10% of journals, the score is multiplied by 3.

One issue we have is computer science and similar fields that publish in conferences, which are much more like 'publications' than some other disciplines.

We also have an issue with the departments like Art (kind of a techno art and research), in which grant funding is usually lower.

I think this type of question would be useful to people, but mostly in a list format where many people can continually add methods. I think it is not too broad, since it is focused on quantitative measures. Thoughts?

Answer 1

With my mod hat on I do what the AC.SE community wants, but personally I don't like lists. I especially don't like lists that are not obviously useful to a large number of people and will have people interested in maintaining the list.

That said, for this particular case, I wonder if there is a relevant non-list question. For example, the UK uses the Research Excellence Framework (REF) to rate groups of researchers and departments then use the REF framework to rate individual researchers. There probably is a less than book length single comprehensive answer that touches on the underlying theory behind things like REF.