 |
 |
 |
 |
 |
 |
 |
CAT Bibliometria SPA Bibliometría |
contents
Introduction | Main bibliometric parameters | Dangers and limitations of bibliometric analyses | Methodological issues | Thirty years of bibliometric studies in TIS | Research potential
Bibliometrics, like translation and interpretation studies (TIS), is a relatively recent discipline, with its origins dating back to the 1920s, when the search began for ways to quantify and statistically describe bibliographic production. Also like TIS, the discipline developed rapidly from the 1970s onwards, when the term was coined internationally (Pritchard 1969) and its systematic study and application began.
 |
| Bibliometric concepts and BITRA. |
Although bibliometrics itself is essentially quantitative, it is often necessary to supplement the mere figures with qualitative analyses which, for example, delve into the nature of the most remarkable publications or into the reasons why some works have more impact than others. These reasons usually include socio-historical, linguistic and epistemological considerations that are part of the interpretative realm and add interest and relevance to the data by integrating them into a coherent narrative.
Bibliometrics is interesting mainly because it allows to empirically analyse large volumes of published information in terms of its dissemination patterns, behaviour and social (and academic, in the case of scientometric studies) weight. Bibliometrics applies statistical tools in search of regularities, seeking to establish trends regarding key issues such as which are the most productive hubs, what characterises the most influential works, how do scholars cooperate, what are the effects of publications on the academic community and on society in general, how are citation patterns, and if it is possible to obtain insights into the theoretical and methodological evolution of a discipline.
For a bibliometric approach to make sense, it is first necessary to have a substantial bibliographic production liable to be analysed. Notwithstanding some tentative pioneering studies, this explains why bibliometric techniques have not been applied to TIS until relatively recently. Its birth as an autonomous discipline in the 1980s means that it has taken almost four decades to reach a sufficient critical mass. In the early 2020s, databases such as BITRA recorded more than 80,000 TIS-oriented scholarly publications. This has drawn attention towards this type of long-range bibliographic analysis and might also be seen as a kind of certificate of consolidation and maturity of TIS as a discipline in the academic world.
The most frequent aims of bibliometric research revolve around the following approaches:
- Time-related (disciplinary diachrony or synchronic snapshot of a historical moment).
- Thematic (objects of study and theoretical approaches).
- Authorial (co-authorship, productivity, degree of specialisation).
- Networks (groups and their networks, institutional, national and/or publisher nodes in academic production).
- Distributional (accessibility of documents, distribution by languages and formats).
- Impact (citations, co-citations, downloads, reprints, reviews and translations).
These parameters are here presented separately for clarity. However, it is important to be aware that it is common in bibliometric studies to combine several of these approaches in order to draw a broad portrait of the aspect under study. Thus, for instance, it is unusual to provide a diachronic study merely giving growth figures, as these are often combined with other parameters, especially with those featuring a thematic and/or distributional nature.
Time-related analysis typically focuses on answering questions such as when that field of study began to spark academic interest and how its treatment evolved over a given period. As mentioned, the mere juxtaposition of figures is often accompanied by other socio-historical approaches and contextualisations that help to understand the figures.
Thematic analysis seeks to classify bibliographic production according to its main objects of study, approaches or theoretical schools, usually on the basis of frequency studies of keywords and of words in titles and abstracts.
Authorial analysis seeks to answer questions such as whether researchers are specialists or their interest is sporadic, whether they tend to work with other authors or alone, and how productivity is distributed in the discipline.
Network analysis partly overlaps with the previous one and seeks to examine parameters of stable collaboration between researchers, if these relationships and networks tend to be national or international, and which are the most productive (and therefore usually most influential) networks in terms of groups, research centres, regions and specialised publishers.
Distributional analysis examines aspects of an ancillary nature, and usually focuses on the accessibility of documents (whether they are toll- or open-access, an increasingly consolidated trend), the distribution of publishing languages - as a reflection of balances in the academic world - or formats, as clues that can even provide information about the epistemological focus of a discipline.
Finally, impact analysis is the best-known bibliometric endeavour in Academia, since it plays an increasingly decisive role in the assessment of individual research and, therefore, in the possibilities of promotion. The traditional and most common way of measuring the impact of a text, an author or a container (journal, publisher) consists of calculating the citations received by different methods, including the impact factor and the h-index. Among the academic community, until now, there has been a tendency to assume that the number of citations is directly correlated with research quality, and more and more countries apply this index in a mechanical way when evaluating their researchers, not without strong resistance. However, this supposedly direct proportional relationship is far from being a fact. In order to mitigate what is sometimes seen as a perverse use of citation impact, efforts have been made to qualify the calculation methods. This has been done, for example, with new indices, until now not very consolidated, or with different citation windows (average time it takes for a document to receive its first citation or to reach a certain percentage of citations) according to each discipline. Other complementary procedures include enlarging the bibliographical databases that act as a repertoire of citable works or incorporating the trace publications leave on the internet in terms of number of visits, readings and downloads (altmetrics).
![[título del epígrafe]](2.2-X.png "[título del epígrafe]"){kind=small}
Dangers and limitations of bibliometric analyses
The imposing mathematical basis of bibliometrics lends it an aura of asepsis and objectivity that needs to be qualified. A quality bibliometric study can provide much useful and empirical information but, as with any statistical analysis, some central elements are by no means axiomatic or self-evident, such as the definition of the object of study, the selection of methods, indicators and questions, the representativeness of the data and its interpretation. It is therefore important to be aware of some preconceptions and limitations that accompany bibliometric studies. The ones that probably stand out the most are presented below.
A chain is as strong as its weakest link
The reliability of a bibliometric analysis depends first and foremost on the completeness and representativeness of the data on which it is based. Bibliographic databases, especially in living disciplines, have gaps and biases arising from factors such as inclusion criteria, availability and visibility of publications from the compilers' point of view. In the case of TIS, reference bibliographic databases, whether made in Europe or the United States, can be labelled as Eurocentric to varying degrees (BITRA or TSB) or Anglophile (Web of Science). Or they can be so full of documentary noise (Google Scholar) or so lacking in specific labels (Scopus) that they are sometimes very difficult to use for disciplinary purposes. Here is a brief description of all of them.
Web of Science (WoS) and Scopus are generalist databases covering a wide variety of scientific disciplines. In these databases, TIS has no label or category of its own and is often grouped under "related areas", such as linguistics. A detailed manual review of the retrieved documents is therefore necessary. A major limitation in both databases is that they require a subscription, without which not all information can be accessed. Both cover only partially the humanities and social sciences, although this limitation is more prominent in Web of Science than in Scopus (Mongeon & Paul-Hus 2016). The documents collected are mainly journal articles (and only from indexed journals, with very restrictive criteria) and, to a lesser extent, books. This is an important limitation for TIS, since book chapters, books or doctoral theses comprise a far from insignificant percentage of the total number of publications in the discipline (Rovira-Esteva, Franco & Olalla-Soler 2019). A database with more than 80% of entries in English, that lacks a discipline-specific label, that virtually ignores non-articles and that includes less than 10% of the journals specialising in that discipline necessarily yields biased results.
Google Scholar covers the humanities and social sciences better (Martín, Orduña, Thelwall et al. 2018), although it suffers from its own limitations. The main one is that it is full of documentary noise, because the compilation is automatic (without human revision) and indexes all the documents it finds on the web, regardless of the type (it includes, for example, conference slide presentations or subject teaching guides) and the quality of the publication.
The International Information Network on Research into Conference Interpreting (CIRIN) is an open-access bibliographic bulletin on conference interpreting research.
Bibliography of Interpreting and Translation (BITRA) and Translation Studies Bibiography (TSB) are the two TIS-specific holistic databases. BITRA is open access, while TSB requires a subscription. BITRA mines citations from documents and indicates open accessibility; both can be described as linguistically Eurocentric, but they are efficient resources for studying our discipline.
Two basic consequences can be drawn from these potential limitations: firstly, that data should be triangulated whenever possible (not all bibliographies compile the same data and are not systematically comparable), and secondly, that these limitations should be explicitly stated so that the reader of the bibliometric analysis can contextualise the results.
Disciplinary behaviour patterns differ
Each discipline is peculiar in terms of its bibliographic behaviour and citability. For example, the citation window, the number of potential citators and the average number of citations per author tend to be distinctive, which makes it advisable to use standardised analysis parameters specific to each research area. In other words, disciplines are not directly comparable from a bibliometric perspective and therefore specific studies are needed to correct for this disciplinary bias. If a relatively minority discipline, such as TIS, is included as part of a larger discipline, such as linguistics, the vast majority of its texts and containers will necessarily appear with a lower impact than would be the case if the minority domain were considered on its own. Incidentally, this is perhaps the main argument in favour of developing specific bibliographic databases to increase the visibility of a discipline that will otherwise always be peripheral.
The confusion between quantity and quality
The idea that the number of citations is a direct and objective reflection of the quality of a publication is based on the assumption that all academic papers are on an equal footing in terms of citability, so that the difference in citations can only be due to the fact that the best studies attract more attention among experts. This is simply not the case. A paper with identical content will have completely different citation rates from the outset if written in Esperanto or in English, because there are far fewer researchers who understand Esperanto. It is therefore necessary to relativise the absolute number of citations and try to normalise it in the analysis. Here is an example.
Gideon Toury was a TIS theorist whose quality and historical value is a matter of consensus. He is also the author of the most cited work in BITRA, much more than any other scholar. Toury wrote three books, in 1977, 1980 and 1995. The 1977 volume is written in Hebrew and in March 2020 had 19 citations in BITRA (0.4 citations/year). The 1980 book, published in English by an Israeli university, is one of the foundations of the theoretical revolution that brought about modern translation studies and has 293 citations (7.3 citations/year). The 1995 book, in English and launched by a prestigious commercial Dutch publisher, is basically an orderly reformulation of the 1980 book and has 960 citations (38.4 citations/year). A mechanical application of the impact-quality relationship would tell us that the 1977 book, in which Toury's theses are firstly presented, is rather weak, the 1980 book very worthy but nothing extraordinary, while the 1995 volume, unoriginal from its very inception, is five times better than the 1980 book and the best piece of research ever accomplished in TIS. Actually, it is the variables of publishing language and publishers’ dissemination power-nothing to do with quality-that explain the different impact of these three works.
Apart from those in the previous example, perhaps the main citability variables are the reason for citation (a publication can also be cited to criticise it), the time elapsed (the more years since the publication of a text, the more likely it is to accumulate citations), the popularity of the object of study (translation analyses of Shakespeare attract many more researchers than those dealing with Rosalía de Castro) and the degree of specialisation (the more generalist, the more potential citations; hence, handbooks, general expositions and summaries tend to be the most cited documents). For example, Franco (2013) noted that the vast majority of the 50 most cited documents in TIS were generalist books written in English and published in the last quarter of the 20th century.
This does not imply that the number of citations is unrelated to quality, but it does imply that the only way for impact to be meaningful is to compare between equals, i.e., between essays with a similar degree of citability. In other words, having many citations can be considered a likely indication of interest in its class, whereas lack of citations is often due to reasons that have nothing to do with lack of quality.
Along the same lines of quality-related confusions, academic evaluation agencies often use journal impact to assess an article. To give an example, in March 2020 BITRA registered almost 450 articles in Target, one of the most prestigious journals and the one with the best impact index in TIS. In BITRA there are two Target articles with more than 120 citations detected as of that date, while over 100 articles had no citations detected. Using the journal's overall impact index automatically to characterise the individual articles means that the two articles with more than 120 citations have the same impact and therefore (according to the social imaginary) the same quality as the hundred articles with zero citations.
A database is the first instrument needed to carry out a bibliometric analysis. There are two main types: those created and supplied by third parties and ad hoc databases. The latter are created mining bibliographic and bibliometric data from a given set of publications. In order to be able to describe trends, this corpus of mined publications must be representative, both qualitatively and quantitatively. This representativeness is often enhanced when using third-party databases, which also help to limit bias stemming from researchers' interests and preconceptions.
To improve the representativeness and completeness of the database, data from different sources can be integrated. In TIS, this is recommended especially for studies based mainly on Web of Science and Scopus, due to their limitations. Currently, there are no tools available to facilitate the process of integrating data from different databases, and it is a task that has to be done manually.
Once the information has been obtained and the database for the analyses has been created, the data must be prepared. In the initial stage, it is a good idea to create the database on a spreadsheet or a database manager. The data are prepared according to the aspects to be analysed. The aim of this first phase is to repair errors in order to make the analysis as clean as possible. If, for example, author parameters are to be analysed, it is important that their names are spelled correctly and coherently (for analysis software "E. A. Nida", "Eugene Albert Nida", "Nida, E. A." or "Nida, Eugene A." are different authors). Symbols that may cause reading errors when transferring information and change formats must also be removed. After preparation, the database is transferred to a statistical analysis application. In the absence of a specific software for bibliometrics, any statistical package can be used for this purpose, and there are free and open-source programmes that allow this task to be carried out adequately.
Before proceeding with the statistical analysis, the data must be standardised. Normalisation is the process of transforming data to facilitate comparison between different sets. For example, from a 2021 perspective, two articles from 1990 and 2018, both with 15 citations, do not indicate the same impact, as the latter has reached the same number of citations in much less time.
Waltman and van Eck (2013) list several bibliometric data normalisation systems. There are two main methods. The first is to use percentiles (usually 25, 50, 75, 90 and 95 percentiles) instead of absolute counts. The second is to divide the data by the absolute maximum or by an expected value.
 |
| Figure 1. Number of authors with 1-15 publications in TIS (1961-2015). |
Bibliometrics-oriented statistics
Generally speaking, quantitative bibliometric data are discrete, with a positively asymmetric distribution (Figure 1) similar to a Pareto distribution (see Zipf's law, Lotka's law and Bradford's law [Bailón, Jurado, Ruiz et al. 2005]).
Consequently, the central limit theorem is not applicable in bibliometrics. For this reason, parametric statistics for inferential purposes should be used with caution. Similarly, means and standard deviations are statistical parameters that do not provide a reliable description of the centre and dispersion of the data. It is therefore advisable to use other parameters, such as the median and percentiles, in the case of describing the central tendency of the data, and the median absolute deviation when describing the dispersion. In terms of regression procedures, models based on the Poisson distribution are often appropriate for bibliometric data when time is used as a factor in the analysis of, for example, the number of citations (De Bellis 2009).
Moreover, when the sample is very large, the results of inferential tests tend to be statistically significant, even if the differences between the aspects compared are small and therefore not relevant. It is advisable for all statistical tests to be accompanied by effect sizes (see Mellinger & Hanson 2017 for calculating effect sizes) and to have established a minimum level of effect size beforehand to determine which significant results are relevant to the study.
Some indicators in academic production
Table 1 lists some indicators in the analysis of academic production. Production can be analysed in terms of the evolution of publications, their formats, countries and institutions with the highest scientific production and the most productive languages, to name but a few possibilities.
| Table 1. Some indicators related to academic production. | ||
|
indicator |
purpose |
estimation |
|
development through time |
to describe how any aspect of interest develops as time advances |
frequency, through cumulative ratios |
|
productivity per total publications (Vinkler 2009) |
to compare the productivity of research groups sharing similar traits in a given period |
tp = period U(tp) = total publications in that period K = mean of analysed researchers |
|
productivity per total articles (Vinkler 2009) |
to compare the journal-related productivity of research groups sharing similar traits in a given period |
tp = period P(tp) = total publications in journals in that period Pi(tp) = total publications in indexed journals in that period K = mean of analysed researchers |
|
activity index (Frame 1977) |
to determine the production ratio of a part (sub-discipline, country, language, etc.) in relation to production as a whole |
Pt = total publications of the part PT = total productions of the whole |
Indicators of impact
Table 2 shows some of the bibliometric indexes used to measure the impact of scientific production in terms of citations, but there are more.
| Table 2. Some indicators related to impact (through citations) | ||
|
indicator |
purpose |
estimation |
|
impact factor of a journal (Garfield & Sher 1963) |
to measure the citation mean per article in a given journal and year through citations received in a given period (citation window), normally between 2-5 years) |
C = total citations to articles published in a journal during the last two (or five) years P = total de artículos publicados en una revista en los dos (o cinco) años anteriores y-1 y y-2 = years as established in the citation window |
|
h-index (Hirsch 2005) |
to measure productivity and impact of a researcher or container |
The highest number of an author’s publications which have been cited h times at least |
|
citation obsolescence |
to describe bibliographic obsolescence |
yearly distribution of citations received (by a document or a group of documents) as a count and as a cumulative ratio |
|
median age of citations |
to measure the median age of the citations received by a publication in a given year |
median of the age of the citations (year of the citation– year of the citing publication) received by a document in a given year |
|
median age of cited documents |
to measure the median age of the documents cited by a publication |
median of the age of the documents (established– year of publication) cited by a document in a given year |
|
immediacy index (McVeigh 2004) |
to measure the time documents published in a given year take to be cited in that same year’s bibliography. Usually applied to journals |
Cy = Number of citations to the articles of a journal published in a given year Py = Number of articles published that same year |
|
attractivity index (Schubert y Braun 1986) |
to measure the rate of citations received by a part (sub-discipline, country, language, etc.) in relation with the total citations received by the whole |
Ct = total quotations of the part CT = total de citas del todo |
|
probability of being cited (Vinkler 2004) |
to measure the mean citations a group of publications issued in a given period can receive in a particular year |
ry(t) ⋅ Py = mean number of references in the documents published in year y Pt = number of documents published in period t |
When using citations as the unit of analysis, it is necessary to decide whether to include self-citations, as they may distort the results. Arguments for and against this can be found in Cooke and Donaldson (2014). Other impact indicators besides citations are the number of downloads, the number of positive peer reviews, reprints, translations, etc.
Authorship indicators
Generally speaking, authorship parameters are linked to productivity and impact. These indicators relate mainly to collaboration between authors (and, consequently, between institutions or countries). They are therefore analyses of co-authorship. For this type of analysis, the researcher must decide how to count the authors of a co-authored publication, since work leading to a publication is not always equally distributed. In general, two methods are proposed: full count and fractional count. In the full count, each author is recognised as full author of the publication (1 publication among 4 authors = 1 publication per author). In the fractional count, the credit for the publication can be divided equally among the authors (1 publication among 4 authors = 0.25 per author) or weighted according to appropriate factors (such as signing order, academic status, etc.). Full count is often considered more operative, since it is difficult to know how the work has been divided among the co-authors. However, the full count also has many detractors, especially for fields where authorship tends to involve dozens of people per document. Various initiatives try to establish from the start the role of each co-author. Vinkler (2009) lists several methods for fractional counting. Co-authorship is often studied using bibliometric networks (Figure 2).
 |
| Figure 2. Network of collaboration between authors in TIS from 2011 to 2015 (data source: BITRA, September 2019; counting method: full; inclusion criterion: minimum of four co-authored publications). |
These networks provide an overview of the collaboration between authors and place them in clusters according to the degree of closeness to each other. This method is also useful for visualising co-citation networks. Co-citations measure the number of times two documents are cited simultaneously in the same publication, giving an overview of the influence and impact of certain authors on others. To elaborate these networks, there is specific and free software such as VOSviewer (van Eck & Waltman 2010).
Thematic indicators
The objects of study of a discipline or a part thereof can be analysed on the basis of several aspects: the most frequent labels (in the title, abstract or keywords), the fields of research to which the documents belong, the relationship between topics, keywords or fields, etc. Thematic analyses are also particularly useful for determining the respective weight of theoretical schools or research methodologies on the basis of key terms typical of each of them. As in the case of co-authorship, bibliometric networks are very useful for visualising this type of data.
Thirty years of bibliometric studies in TIS
To review the bibliography of bibliometric studies in the discipline, BITRA entries were filtered using keywords designed to retrieve works of a quantitative nature (bibliometric, quantitative, scientometric, statistical, etc.). The “bibliometrics” label is relatively recent, and restricting the search to it would have left out many contributions featuring quantitative approaches or based on more or less sophisticated statistical calculations. Many of them were in some way pioneers in the discipline and should be considered in order to study the evolution of quantitative bibliometric approaches within TIS. To check their eligibility, the list of references was manually revised one by one, taking the abstract -when available- as the main reference. When the case was doubtful or the information scarce, the document was read in its entirety if accessible. The 117 references that passed this screening comprise the database used in the following analysis.
We can place the birth of bibliometric studies within TIS in the 1990s, with only 10 documents (8.5%) which were thematically dissimilar, with a slight bias in favour of interpreting. The first work is by Sajkevic (1992), although it focuses more on translated works than on TIS. In this decade, Franz Pöchhacker and Daniel Gile also stand out as forerunners of bibliometric studies in interpreting.
With the new millennium, new researchers such as Luc van Doorslaer, Javier Franco Aixelá, Nadja Grbić, Christian Olalla-Soler, Sonja Pöllabauer, Sara Rovira-Esteva and Ziyun Xu entered the scene. Grbić & Pöllabauer (2008b) wrote a very comprehensive paper on bibliometric concepts and methodologies, and how they had been applied to TIS up to that point. From 2001-2010, 34 papers (29.1%) were published, and the highest percentage (62.4%) corresponds to 2011-2019. A peak of 22 publications, in 2015, is due to a special issue of Perspectives on bibliometrics. There is also an upward trend in 2019, with twelve documents. This third decade shows that bibliometrics is attracting the interest of a greater number of researchers. Some have several works but the 73 papers are distributed among 78 different authors. Although the majority (61.5%) of the contributions are by a single author, almost 40% are co-authored. Of these, more than half (22.2%) are by two researchers, 10.2% by three and 5.9% by four. However, tandems do not tend to last longer than a couple of contributions, except in the cases of Franco and Rovira-Esteva who, with six and eleven publications, respectively, seem to have consolidated working teams in this field.
Just over half of the studies (51.3%) adopt a multilingual and international perspective. The rest are case studies focusing on different aspects of TIS as produced in particular languages (which may cover more than one country) or geographical areas. In this sense, China is the country with the highest number of studies (18.8%), almost all of them by its nationals, followed by Spain (9.4%). In both cases, the publication languages are almost always English and Chinese or English and Spanish, respectively. There are also two works on Spanish-speaking countries and two more on TIS in Latin America. Brazil is the next country on the list, with three specific studies.
As for the languages used in the papers, English is clearly the prevailing language, with 64.1% of the cases, followed by Spanish (17.9%), Chinese (8.5%), French (5.1%) and Portuguese (1.7%). Only two papers were found in German and Turkish. Two thirds are journal articles (65.8%); one quarter are book chapters (26.5%) and there are five working papers, two books and two doctoral theses. The only journal that stands out as a platform for publishing bibliometric studies is Perspectives, due to the 2015 special issue mentioned above. In fact, the 77 articles are scattered among 45 journals, most of them specialising in TIS, but also with some that do not, and therefore more difficult to locate. There are 34 journals with only one article on TIS-oriented bibliometrics.
In terms of the thematic scope, 57.3% of the contributions cover TIS as a whole, while others focus on one of its sub-disciplines. At the top of the list are studies on interpreting in general or one of its modalities (21.4%), followed by translation didactics and literary translation (with 5.1% each), scientific-technical translation (4.3%), audiovisual translation (2.6%), corpus linguistics (1.7%) and, at the end of the list, gender and translation, discourse analysis and translation theory, with only one contribution each.
In a few publications (10.2%), the object of study is restricted to a specific language, with none standing out. Many publications do not indicate the period of analysis, as they are descriptions of the state of the art. Others present a diachronic view to describe historical developments, and a third (33.3%) analyse time frames ranging from four to two thousand years. Out of the 39 studies specifying a time window, in 19 cases (48.7%) this is less than 10 years, 14 of which coincide with studies focusing only on journals included in particular impact indexes (mostly WoS), which are based on relatively small citation windows. There are eight papers that analyse a window of 10-30 years and 12 that go beyond this figure to cover the whole of our era.
Studies tend to cover various objects of study and methodological tools of analysis at the same time. By way of illustration, 17 contributions (14.5%) analyse parameters related to researchers (such as their affiliation, subject matters they address, number of citations, co-authorship patterns, co-citation networks and signing order). Identifying the predominant topics of study in TIS as a whole or in some sub-discipline thereof by analysing words in titles, keywords or abstracts is also a relatively frequent approach, with 22 occurrences (18.8%). Additional parameters include document typology (7), methodology or theoretical approach (6), language (5), including its correlation with the number of citations, or the number of pages (2). Grbić & Pöllabauer (2008a) and Rovira-Esteva, Franco & Olalla-Soler (2019) are two examples of ambitious works due to the range of parameters they analyse and interrelate.
Citation counts, included in 13 papers (11.1%), are relatively recent in TIS, mainly for two reasons: the low representativeness of TIS academic production in international bibliographic databases and the non-existence until 2009 of a specific database that collected citations and references together, so that data could be cross-referenced to carry out long-range disciplinary impact studies. The first works that included citation counts are by Gile (2005, 2006). Both the birth of TSB and BITRA, and the increasing inclusion of TIS specialised journals in the most prestigious indexes mark a turning point. Thus, 2009 represents the beginning of a decade in which bibliometric work is proliferating, with statistical analyses becoming increasingly sophisticated.
There are hardly any works featuring a critical stance towards the current research evaluation systems due to the precarious situation of TIS (cf. Rovira-Esteva & Orero 2011). Almost none take into account alternative indexes (such as the Google Scholar h-index) or others such as those offered by Scopus because their use is new in the specialised literature (Huang & Liu 2019). This precariousness can only be overcome with a greater number of increasingly strong bibliometric studies allowing us to gain greater self-knowledge, to establish which data represent normality in our discipline and to know where we stand in relation to others.
Nowadays, the possibilities for quantitative bibliometric analysis have bloomed for a number of reasons. On the one hand, thanks to the digitisation and democratisation of information through the Internet, together with the growing number of open-access scientific documents and databases. On the other hand, this has come to happen due to the increase in the number of TIS journals that have found their way into international impact indexes (in WoS we have gone from 7 journals in 2011 to 13 in 2020, and in Scopus from 13 to 37) and the creation of TIS-specific bibliographic databases reducing representativeness biases and providing information on the citations of many of their entries.
There are many issues that remain to be studied or whose analysis has been anecdotal which would undoubtedly be worth their own research efforts, including signing order, author profiling or qualifications, gender issues, co-authorship networks, interdisciplinarity, citation analysis, number of pages, visibility and accessibility of contributions, indexing or open access policies of journals, analysis of co-occurrences, institutions or reference groups by subdisciplines or impact in altmetric terms. The work by Grbić & Pöllabauer (2008b), Grbić (2013), van Doorslaer (2015) and Rovira-Esteva & Franco Aixelá (2018) point to unexplored avenues. This should be done not only with large and representative databases enabling longitudinal analyses, but also by combining more qualitative approaches that allow for contextualising the data and obtaining deeper results. With current resources, not only can new research be carried out, but studies could also be replicated or periodically revised to update the picture we have of our discipline.