Publications are the starting point for the entire evaluation in the case of academic and research staff. This means that evaluation of publications will be used as a basis for the overall evaluation and other activities will only modify the overall result (see the Overall result of the evaluation section).
Our publications have been subject to evaluation at national and university level for many years, although often not in a satisfactory way. The system infamously known as “coffee grinder” in Czech mainly counted the quantity of publications (the more the better), which has often led to a production of rather low-quality texts. Current evaluation methods allow us to place more emphasis on quality. This is either assessed based on bibliometric data, or by peer-review. However, neither of these methods is completely satisfactory either. Bibliometrics is fast and easy, but only works well for certain fields of research (at our faculty, these include social sciences and certain branches of linguistics), but when applied to other disciplines, its results may be distorted. On the other hand, peer-review can take into account the specificities of the discipline, but it is time‑consuming, costly and its results can be biased.
Our evaluation model combines these two approaches, while we are aware of their weak points and actively seek ways to minimise them. Our starting point is respect for the diversity of our fields and their specificities. Although we apply the same principles to the evaluation of publications in all fields, we take into account their different manifestations. Therefore, we use bibiliometrics to evaluate disciplines where it yields representative results. Where its results are less reliable, we use different formal criteria (prestige of the publisher, publication or authors of the remaining chapters of the monograph) or content-oriented criteria.
In any case, we are more concerned about quality than quantity, in accordance with the national and university-level evaluation. That is why each employee only proposes for evaluation their top five publications from the last five years and they can even succeed with a lower number (as long as quantity is compensated by quality). The evaluation is in no way intended to create pressure to achieve maximum performance. It is supposed to increase the quality of our publications and develop reasonable publishing strategies that will help us achieve good results without having to resign to our personal or family life.
The evaluation is based on a combination of formal and content-oriented criteria. The advantage of formal criteria consists in that they are easier to work with and can be applied comparably to different publications (and so, for example, different articles in the same journal receive the same rating). Therefore, they should be used as starting point wherever they are applicable. However, in many cases, they will not be sufficient and will have to be complemented with an assessment of the content. Please, see the Manual for Evaluation of Publications for a detailed description of all criteria.
The prestige of a journal can be often determined bibliometrically (the SJR/Scopus methodology is usually most suitable for our disciplines). However, in many fields, bibliometric data have to be adjusted manually (such as in the case of interdisciplinary or non-Anglo-Saxon journals, which can be more suitable for certain type of articles).
In many cases, however, bibliometrics is of little or no use. Key journals of a certain field (such as literature) may be missing in the Scopus database, and certain disciplines are not represented at all (such as Asian Studies). In general, bibliometrics is not very suitable to rate monographs.In all these cases, the prestige of the given publication must be determined in a different way – for example, according to the importance of the authors of the remaining chapters of the given collective monograph/proceedings/journal. Also, a publication prepared by authors from different countries and centres is considered to be more prestigious.
Content evaluation does not consist of a detailed peer‑review that specialists within a narrow field carry out based on close reading. Given the number of publications, this would not be practically possible. Therefore, the assessment only works with aspects that can be evaluated by academics with a slightly different specialisation (such as a linguist specialised in another language). Rather than a detailed analysis as we know it from the reviews of articles in journals, it is a quick reading of the text to see how the author proceeds, which resources they use, how they work with them and based on what methodology, what results they achieve.
Within the self-evaluation, the employee may include anything that will facilitate the assessment of quality of the given publication and help identify its scientific importance. Such description is a valuable source of information for the evaluator, who at the same time critically assesses it and evaluates how realistic it is according to the other evaluation criteria.
Responses to the publication (not only citations, but also critical reviews, reactions of other researchers, awards, etc.) can also help to assess it correctly. Citations form part of formal criteria in many fields (they are measurable according to diverse methodologies), but in our disciplines, publications are not cited quickly enough so that it makes sense to measure this aspect in a few years. Therefore, more attention is paid to the content of the responses. Direct citations from such responses are particularly welcomed,as they allow us to see how the publication is viewed by the expert community.
Each publication is evaluated separately and ranked into one of the following categories:
This ranking is based on the 17+ RVVI Methodology, but it differs from it by a clearer differentiation of publications in world and national languages. This should reduce the risk of a lower ranking of Czech publications as compared to those written in English.
“International importance” always depends on the context of the given field. International publications address international professional audience. This is reflected not only in the language of the publication, but also in the publisher and other authors publishing in the given issue of the journal, proceedings or monograph. Therefore, even with publications written in English, international value is not a given (e.g., English articles in Czech journals with mostly Czech authors are considered as national). Foreign publications in languages of lesser diffusion (such as Danish, Polish, Tibetan, etc.) are primarily considered as international, because for many “semi‑peripheric” countries, it is important to maintain direct communication channels instead of mediated publishing via an Anglo-Saxon centre. However, this only applies when the given publication contains works by authors from different countries (see chapter 5 of the Manual for Evaluation of Publications)
In addition to the evaluation of individual publications, the employee will also receive an overall rating on the scale from A to E, which corresponds to the “international” A1-D1 scale, while the “national” scale is shifted to A2-D2. In other words, a predominance of excellent national publications will result in a rating of B, while a predominance of below-average national publications to a rating of E. In this way, we want to stimulate publishing in international journals, in which our faculty is still lagging behind compared to other European countries. Exact rules for the overall evaluation of publications are specified in chapter 14 of the Manual for Evaluation of Publications.
| A1 – International excellent | |
| B1 – International significant | A2 – National excellent |
| C1 – International average | B2 – National significant |
| D1 – International below-average | C2 – National average |
| D2 – National below-average |
