Killing the Discipline: Problems and limitations on
impact factor, prestige, and bibliometric studies in
tourism research
Matar la disciplina: Problemas y limitaciones del factor
de impacto, el prestigio y los estudios bibliométricos en
la investigación turística
1
Maximiliano E. Korstanje
https://orcid.org/0000-0002-5149-1669
mkorst@palermo.edu
2
University of Palermo, Argentina
University of Leeds, UK
1
Nota de investigación. Manuscrito recibido el 18 de agosto del 2022, y aceptado tras revisión editorial
el 11 de noviembre del 2022. Turismo, desarrollo y buen vivir. Revista de Investigación de la Ciencia
Turística- RICIT. no. 16 (2022) Publicación anual. ISSN: 1390-6305 ISSN-e: 2588-0861
2
Profesor investigador. Universidad de Palermo y University of Leeds.
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
99
Abstract
Over the decades, epistemologists and theorists of tourism have devoted considerable
efforts in laying the foundations towards the discipline maturation. This so-called
maturation would take the lead when tourism situated not only as a solid object of study,
but also professional publications reach a the scientific rigorism. The number of
publications in the tourism fields has certainly been triplicated in the recent years, but
today tourism has many barriers to become in a consolidated option. This happens
because of many reasons. One of them associated to the problem of impact factor, the
invention of academic prestige and the bibliometric studies. We, in this short note of
research, explore the impossibilities, limitations and problems of the current tourism
research which remains culturally embraced to impact factor logic. To put things in
bluntly, the current obsession for impact factor in the academy is gradually leading the
discipline into an unparalleled crisis.
Key words: tourism research, investigation, higher education, impact factor.
Resumen
A lo largo de las décadas, los epistemólogos y teóricos del turismo han dedicado
considerables esfuerzos a sentar las bases para la maduración de la disciplina. Esta
llamada maduración tomaría la delantera cuando el turismo se situará no solo como un
sólido objeto de estudio, sino que las publicaciones profesionales alcanzaran el rigor
científico. El número de publicaciones en el ámbito turístico se ha triplicado ciertamente
en los últimos años, pero hoy el turismo tiene muchas barreras para convertirse en una
opción consolidada. Esto ocurre por muchas razones. Uno de ellos asociado al problema
del factor de impacto, la invención del prestigio académico y los estudios bibliométricos.
Nosotros, en esta breve nota de investigación, exploramos las imposibilidades,
limitaciones y problemas de la investigación turística actual que permanece culturalmente
abrazada a la lógica del factor de impacto. Para decirlo sin rodeos, la actual obsesión por
el factor de impacto en la academia está llevando poco a poco a la disciplina a una crisis
sin parangón.
Palabras clave: investigación turística, investigación, educación superior, factor de
impacto.
It is noteworthy that the bibliometric studies have been multiplicated over recent years in
the constellations of tourism and hospitality. For some reason, which is hard to precise
here, the mania for these studies was practically palpable just after the 2000s (Barrios et
al 2008). These works mainly focus on the different positions of authors according to two
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
100
key variables, the number of publications most of them located in top-tiered or leading
journals, and the number of citations expressed in the I-10 and Hirsch indexes (Law et al
2009; Gursoy & Sandstrom, 2016). Particularly, H-index measures the correlations
between citations and the number of publications while creating a hierarchy of scholars
worldwide (McKercher 2007; Sheldon, 1991).
The utility of bibliometric studies has been unquestionable. On one hand, it allows the
understanding of the networks of scholars as well as the trending topics. On another, it
consolidates the emergence of new discussions revolving around the future of the tourism
industry (Evren & Kozak, 2014). Echoing Jafar Jafari, the scientifization [maturation] of
tourism research depends not only on the number of publications but also on the
calibration of reliable resources to obtain mix-balanced conclusions (Jafari 2005).
From Jafari onwards, scholars strongly believed that the maturation of the disciplined
depended upon the number and impacts of publications in the fields of tourism and other
disciplines. For more than four decades, tourism-related studies strived for situating
tourism as a serious discipline (Xiao & Smith, 2006; Wardle & Buckley, 2014; Butler
2015). Henceforth, the culture of metrics has occupied a central position in the
configuration of tourism epistemology since its onset (Beckendorff & Zehrer 2013).
Having said this, some critical voices have alerted not only on the problems revolving
around the culture of publishing or perishing but also on the epistemological
discrepancies left by the culture of metrics (Korstanje 2021; 2023).
One of the pioneering scholars who claimed the problems of tourism research was
doubtless Michael C. Hall. To wit, he argues convincingly that bibliometric analysis often
emphasizes the importance of publishing or citation factors excluding other hybridized
methods. At the same time, bibliometric analysis is mainly marked by an institutional and
policy vacuum dominated by private organization evaluations. To some extent, those
protocols orchestrated to evaluate professors` performance, which is strictly based on
productivity, simply overlook the quality factor.
The problem with quantitative methods lies in the lack of what experts dubbed the
descriptive factor. Instead of measuring, description helps to shed light on the current
understanding of tourism future. To put simply, the correlation between two variables
[mainly measured by the employment of quantitative methods] does not explain the
causality of events. Of course, as Hall adheres, these types of evaluations correspond with
the obsession for gaining further funding and the monopoly of financial resources in
tourism higher education.
Per his viewpoint, there is a momentum in the discipline [following the current reasoning]
where a paper situated in a highly ranked journal has more impact or importance as a
source of measurement than its genuine contribution to the specialized literature (Hall
2011). In addition, Graham Dann called attention to the control of the Anglophone world
in knowledge production as well as peer-review processes in leading tourism journals
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
101
(Dann 2011). In consonance with this, John Tribe exerts a caustic critique of what he calls
“the indiscipline of tourism”. As Tribe eloquently observes, the burgeoning growth of
publications over recent years has not had a direct impact on the consolidation of tourism
research. What is more important, tourism research not only faces a crisis of sense but
also a great dispersion because of the lack of a shared agenda in the academic tribes (Tribe
1997; 2010).
As the previous critical argument is given, a mix-balanced discussion is at least needed.
The bibliometric culture has strengths and weaknesses. Among the strengths, the
bibliometric analysis allows the rapid identification of those factors or topics the
academic tribe is investigating. Besides, it recreates the conditions of dialogue [to resolve
the problems of the industry in the years to come] as well as cooperation toward
multidisciplinary research (Koseoglu et al 2016). In the next lines, we shall identify [if
not scrutinize] the weaknesses behind bibliometric-based studies.
At a closer look, there is a dichotomy given by the H-index as a key factor impact
evaluation. H-index never reflects the real contribution of the author to the field [or the
paper quality], but only the interplay between the number of publications and the impact
factor. For the sake of clarity let's put a concrete example, with only for papers published
in his life, Albert Einstein would have a low H- index. This reminds us that some of the
most recognized scholars who had earned international awards globally seem not to be
the most cited ones. Furthermore, being cited does not mean that the paper has a direct
contribution to the discipline. Most citations are critiques of methodological problems.
Some disciplines [like mobilities theories, anthropology, or cultural studies] are less
prone to productivity and citation factors than marketing or management.
This behooves us to consider the fact that the citation impact varies on culture, language,
and of course discipline. Under some conditions, impact factors recreate a hierarchy of
authors -most of them English native speakers- that serve as real gatekeepers [simply
because they are part of leading journals] of the produced knowledge. Scholars are simply
cited as emulating fashionable behavior instead of their contribution to the field. It is not
simplistic to say that “being cited” does not depend on self-performance. It is a question
of trust [where the attention of colleagues is captivated], publicity, and successful
strategies in placing the papers in the appropriate journals. Here, four additional problem
surfaces: the role of reviewers is not paid.
Although some journals have implemented rewards programs for reviewers to resolve
this problem, no less true is that some reviewers condition their approval to a situation of
dependency. Authors are pressed to cite reviewers` works or simply to papers previously
published by the intended journal. In this way, journals have more possibilities to promote
further positions in the impact factor list. It is important to mention that quantitative
methods are often prioritized over qualitative forms of evaluation, as Hall ultimately
lamented.
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
102
Another point of discussion in this entry is the database selection process. This includes
the sampling process where some important journals are excluded from the analysis.
Albeit some books and book chapters are highly cited, they are not selected in the
bibliometric studies. The opposite is equally true, some indexes or databases [like Scopus
or WOS] are highly preferred by authors excluding other larger ones [like google scholar].
Returning to Hall`s assumption, if we start from the premise that the citation factor does
not mean a higher paper quality, this begs the question: why Scimago or WOS are more
important than other databases in the sampling process? These indexes only represent a
small portion of the academic tribe.
The act of selecting the publications placed in the top-tiered journals has an additional
limitation. Significant material such as books, book chapters, or doctoral theses, which
certainly obtain higher citations, are systematically ignored. Scholars more cited in
Scopus probably are not the most cited in other sources or databases [i.e., google scholar].
What seems to be more important to discuss, bibliometric culture reaffirms the hegemony
of the greatest editorial corporations while cementing the logic of pay-for-journals. As a
result of this, institutional or university presses or classic journals are in decline, or simply
incorporated into the leading worldwide publishers. Pay-for-journals keep restricted
access to published material to a global audience.
Some scholars in developing countries have serious restrictions to access this material
though they paradoxically are pressed by their institutions to publish in leading journals.
In consequence, the proposed manuscript has two fatal problems, the literature is far from
being updated, and the topic is far from being original. It is safe to say that the current
bibliometric culture, which holds the glue on some asymmetries between Global South
and North, not only harms the discipline but also entraps academicians into an ivory tower
and a climate of extreme competition. In parallel, some low-skill fieldworkers are
debarred into a peripheral position in the global academic hierarchy. Some institutional
pressures lead low-resource academicians to access predatory journals which charge them
back for publishing.
Let`s explain that predatory journals are defined as fraudulent or deceptive pseudo-
journals that often violate scientific practices. In most cases, these journals manipulate
the author`s desperation [promising faster publications] charging them back excessive
fees while in others the peer review process is poor and inadequate. The rise and
popularity of these journals explain on basis of two combined factors. Universities
exerted considerable pressure on their professors to publish their works in accredited
journals. At the same time, universities are frequently evaluated according to the number
of publications of their researchers. More ranked the university, more financial
resources, and students. This leads some scholars to criticize the culture of publishing or
perishing as the cornerstone of the commoditization of higher education (Van Raan
2005).
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
103
Furthermore, the restricted access to the published material put universities between the
wall and the deep blue sea. Universities should pay significant fees to publishers for
sharing with students the material their professors ultimately publish. Universities double
pay the researchers` wages, and the access royalties. Therefore, important universities
like Cambridge, Oxford, and Chicago recently called on their researchers not to publish
works in WOS and Scopus [in a type of unparalleled boycott]. As an initiative, leading
publishers implemented a new policy of open access [billing to authors for the fees] but
paradoxically these fees are sometimes out of the possibilities of researchers.
It is important not to lose sight of the fact that the bibliometric culture has a direct impact
on the higher education system. Some universities impose strict rules and procedures,
which include the publication of two or three papers in top-tiered journals annually so
that academicians keep their tenure. Plausibly, this creates a vicious circle where
professional fieldworkers leave their classrooms in the hands of amateur [low-paid]
professors. One of the most palpable consequences associated with the fact that pre- or
post-graduate students have countless methodological limitations to finalize their theses.
No less true seems to be that the urgency for publishing put authors in a dilemma: writing
exclusively on the topic of the moment or finding some innovative unexplored themes.
In the few months that marked the Post COVID-19 context, it is estimated more than
250.00 items are found on google scholar. This happens simply because scholars are not
motivated by their object of study [or main scope] but rather by the higher journal impact
factor estimated for their publications. Here a new point emerges: Is this the precondition
towards the expansion of the discipline or its final ruin? Last but not least, this literary
piece is far from being an empirical note of research in the strict sense of the word [as I
am accustomed to writing] but it lays the foundations for embracing new methods in the
evaluation, not to the future of tourism research in the future.
References
Barrios, M., Borrego, A., Vilaginés, A., Ollé, C., & Somoza, M. (2008). A bibliometric study of
psychological research on tourism. Scientometrics, 77(3), 453-467.
Benckendorff, P., & Zehrer, A. (2013). A network analysis of tourism research. Annals of Tourism
Research, 43, 121-149.
Butler, R. (2015). The evolution of tourism and tourism research. Tourism Recreation
Research, 40(1), 16-27.
Dann, G. M. (2011). Anglophone hegemony in tourism studies today. Enlightening Tourism. A
Pathmaking Journal, 1(1), 1-30.
Evren, S., & Kozak, N. (2014). Bibliometric analysis of tourism and hospitality related articles
published in Turkey. Anatolia, 25(1), 61-80.
Problems and limitations on impact factor, prestige, and bibliometric studies in tourism research
Maximiliano E. Korstanje
RICIT no. 16 (diciembre- 2022) pp. 98-104.
104
Gursoy, D., & Sandstrom, J. K. (2016). An updated ranking of hospitality and tourism
journals. Journal of Hospitality & Tourism Research, 40(1), 3-18.
Hall, C. M. (2011). Publish and perish? Bibliometric analysis, journal ranking and the assessment
of research quality in tourism. Tourism Management, 32(1), 16-27.
Korstanje, M. E. (2021). The decline of book reviews in tourism. Published online: 15 February
2021 Free full-text access available at: www. jthsm. gr, 76.
Korstanje, M. E. (2022). Tourism imagination: a new epistemological debate. Current Issues in
Tourism, 1-13.
Koseoglu, M. A., Rahimi, R., Okumus, F., & Liu, J. (2016). Bibliometric studies in
tourism. Annals of tourism research, 61, 180-198.
McKercher, B. (2007). A study of prolific authors in 25 tourism and hospitality journals. Journal
of Hospitality & Tourism Education, 19(2), 23-30.
Law, R., Ye, Q., Chen, W., & Leung, R. (2009). An analysis of the most influential articles
published in tourism journals from 2000 to 2007: A Google Scholar approach. Journal of
Travel & Tourism Marketing, 26(7), 735-746.
Sheldon, P. J. (1991). An authorship analysis of tourism research. Annals of Tourism
Research, 18(3), 473-484.
Tribe, J. (1997). The indiscipline of tourism. Annals of tourism research, 24(3), 638-657.
Tribe, J. (2010). Tribes, territories, and networks in the tourism academy. Annals of tourism
research, 37(1), 7-33.
Van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking
of universities by bibliometric methods. Scientometrics, 62(1), 133-143.
Wardle, C., & Buckley, R. (2014). Tourism citations in other disciplines. Annals of Tourism
Research, 46(3), 163-184.
Xiao, H., & Smith, S. L. (2006). The making of tourism research: Insights from a social sciences
journal. Annals of Tourism Research, 33(2), 490-507.
