Entry | Mixed-methods research

CHI (s) 混合方法研究 SPA Investigación con métodos mixtos

Introduction

Introduction to mixed-methods research. [Source]

Mixed-methods research (MMR) refers to an integrative research approach that combines quantitative and qualitative methods within a single study to erklären (explain) and verstehen (understanding) a phenomenon (Tashakkori & Teddlie 2010; Creswell & Plano Clark 2018). It involves collecting, analyzing and integrating both numerical (quantitative) and textual or observational (qualitative) data to leverage their complementary strengths. From both substantive and practical perspectives, MMR aligns well with Translation & Interpreting Studies (TIS). Substantively, TIS has long been recognized as an interdisciplinary or multidisciplinary field, drawing on theories and methods from disciplines such as philosophy, sociology, psychology, and (applied) linguistics to explore key issues related to translation and interpreting (T&I) performance, training, and evaluation. In practice, as a rapidly evolving field, TIS has been exposed to diverse methodological paradigms within a relatively short period of time and remains open to multiple approaches to scientific inquiry (methodological pluralism). While early debates framed TIS within the “natural sciences” versus the “liberal arts” dichotomy, several prominent TIS scholars advocate for a pragmatic stance, emphasizing methodological flexibility in TIS research (e.g., Schäffner 2004; Pöchhacker 2011).

This entry offers a review and analysis of MMR in TIS. It begins by examining philosophical and theoretical foundations of MMR, highlighting its epistemological and methodological underpinnings. It then explores the key rationales and designs of MMR, addressing essential aspects such as design features and notation systems. This entry further traces the scholarly discussions and historical developments of MMR in TIS, focusing on three key periods: the 2000s, 2010s, and 2020s. Next, it critically evaluates major challenges and limitations associated with conducting MMR within TIS, emphasizing the importance of quality and rigor in MMR-based studies. Finally, the entry concludes by outlining future directions for advancing and refining MMR practice and research in TIS.

Philosophical and theoretical foundations

Overall, the practice of MMR is grounded in a pluralistic epistemological stance, integrating both quantitative and qualitative paradigms to provide a more comprehensive and nuanced understanding of research phenomena. Specifically, the philosophical and theoretical foundations of MMR draw primarily from pragmatism, critical realism, and dialectical pluralism, each contributing to its methodological flexibility and integration of different knowledge systems (Johnson, Onwuegbuzie & Turner 2007; Creswell & Plano Clark 2018). At the core of MMR is pragmatism, which prioritizes research questions over methodological allegiance and allows researchers to use both (post-)positivist and constructivist approaches as needed (Tashakkori & Teddlie 2010). Pragmatism, largely influenced by philosophers such as John Dewey, asserts that knowledge’s worth is derived from attested practical consequences and can be evaluated based on its utility and applicability. It emphasizes “what works” in addressing real-world problems, rather than getting entangled in methodological and paradigmatic debates. This approach enables researchers to transcend the quantitative-qualitative divide and shelf what is notoriously known as paradigm war or incompatibility thesis (see Tashakkori & Teddlie 2010).

Another key philosophical underpinning is critical realism, developed by Roy Bhaskar, which acknowledges an objective reality but recognizes that human perceptions, beliefs, and experiences may also influence how reality can be understood and interpreted. Critical realism provides a middle ground between positivism and constructivism, arguing that while empirical observations can reflect real-world “realities”, they are often mediated by social, cultural, and historical contexts (Maxwell & Mittapalli 2010). This perspective is consistent with MMR, as it allows researchers to combine quantitative methods for identifying general patterns with qualitative methods for interpreting contextual meanings.

A more recent philosophical contribution to MMR is dialectical pluralism, introduced by Greene & Hall (2010), which emphasizes the value of epistemological diversity and the need for dialogue between competing perspectives and paradigms. Unlike pragmatism, which focuses on methodological utility, dialectical pluralism encourages researchers to actively engage with epistemological tensions, using them to generate richer and more comprehensive understandings and insights. In other words, this approach takes a more proactive and critical stance to face up to and entertain potential disparities and even conflicts between different methodological paradigms. By embracing multiple paradigms, dialectical pluralism fosters a more reflective and integrative approach to knowledge construction.

Common rationales and designs

John W. Creswell, on basic mixed methods research designs. [Source]

In general, the widespread adoption of MMR in the social and behavioral sciences is grounded in three key rationales. One of the most frequently cited can be termed cross-referencing, similar to the wider understanding of triangulation in Greene, Caracelli & Graham (1989), which typically involves combining different data sources, methods, or perspectives to confirm and verify results, thus mitigating the limitations of any single approach. This rationale is particularly relevant in TIS, where scholars frequently justify MMR on the basis of cross-referencing (e.g., Alves 2003). A second important rationale is supplementarity (referred to as complementarity in Greene, Caracelli & Graham 1989), which extends beyond cross-referencing by emphasizing how different methods provide unique yet supplementary insights. Rather than merely verifying results, supplementarity allows researchers to explore different dimensions of a phenomenon, using one method to elaborate on findings from another. The third rationale, expansion, is the most flexible justification for MMR (Greene, Caracelli & Graham 1998). It seeks to broaden the scope of inquiry by employing different methods to address distinct research components. Closely related to expansion are two additional rationales: on the one hand, development results from one method informing and shaping the subsequent use of another method; on the other hand, initiation occurs when different methods yield contradictory findings, prompting further investigation to reconcile or account for inconsistencies.

Apart from the basic rationales behind MMR, various criteria have been used to develop typologies of MMR designs (Johnson, Onwuegbuzie & Turner 2007; Creswell & Plano Clark 2018). One key criterion is the time orientation of the quantitative and qualitative components, distinguishing between concurrent and sequential designs. In concurrent designs, both quantitative and qualitative components are conducted simultaneously, allowing for cross-referencing of findings. In sequential designs, one component precedes the other, meaning qualitative data collection can inform a subsequent quantitative phase or vice versa. Another important criterion is paradigm emphasis, referring to the relative weight assigned to quantitative and qualitative components (Creswell & Plano Clark 2018). A study may assign equal status to both methods or prioritize one over the other (dominant versus supplementary components).

Jaroslaw Kriukow, on how not to do mixed-methods research design

Based on the above two criteria, MMR practitioners and researchers have operationalized and distinguished four common MMR designs (Crewell & Plano Clark 2018). The first type is convergent (parallel) design, in which quantitative and qualitative data are equally important and collected simultaneously, but analyzed separately before being integrated to provide a comprehensive interpretation. The second type is called explanatory sequential design, in which a quantitative phase is conducted first, followed by a qualitative phase that helps explain or expand on the quantitative results (e.g., statistical patterns). The third type is known as exploratory sequential design, in which a qualitative phase (e.g., interviews, focus groups) is conducted first to generate insights, theories, or hypotheses, followed by a quantitative phase that aims to test or generalize the qualitative findings. The final design is labeled embedded design and involves incorporating a secondary or supplementary quantitative or qualitative component within a primarily single-method study to enhance or contextualize findings. Notably, merely incorporating both quantitative and qualitative methods in a study does not automatically qualify it as a truly mixed design. A study is considered truly mixed only when there is substantive integration of quantitative and qualitative analyses and results, leading to meta-inferences that go beyond the findings of each strand in isolation (Teddlie & Tashakkori 2009).

As outlined, the four main MMR designs share a multi-strand research structure, meaning that they incorporate and systematically mix multiple strands of research within a single study. However, an exception to this multi-strand nature is found in a unique MMR design known as the conversion mixed design. Unlike other mixed-methods approaches, the conversion design is mono-strand, as it involves the collection of only one type of data – either quantitative or qualitative – which is then converted into the other format to enable integration and comparison. The process of transforming qualitative data into quantitative form is referred to as quantitizing, while the reverse – converting numerical data into qualitative categories or narratives – is known as qualitizing (Creswell & Plano Clark 2018). Because the conversion design does not necessarily entail the full integration of quantitative and qualitative results to generate a higher-order meta-inference, it is sometimes classified as a quasi-mixed design, representing a less rigorous form of MMR compared to fully integrated multi-strand designs.

To facilitate clarity in research design classification, notational systems have also been established (Creswell & Plano Clark 2018). These systems use symbols to indicate the structure of mixed-methods designs. For instance, a plus sign (+) denotes a concurrent design, where quantitative and qualitative components are conducted in parallel (e.g., Qual + Quan). An arrow (→) indicates a sequential design, where one phase follows the other (e.g., Qual → Quan or Quan→ Qual). Uppercase letters (QUAL, QUAN) signify that a method is dominant or given equal priority in the study. Lowercase letters (qual, quan) suggest that the corresponding method plays a secondary or supportive role. These frameworks help researchers systematically design, implement, and report mixed-methods studies with greater transparency and rigor.

Mixed methods, in Translation & Interpreting Studies

The 2000s: Emerging discussions on methodological integration

Systematic discussions on MMR began to take shape in the early 2000s. A good example is the publication of the volume titled Translation research and interpreting research: Traditions, gaps and synergies (Schäffner (ed.) 2004), where T&I scholars critically engaged with methodological orientations in the field. One of the key takeaways from this volume was the recognition that advancing TIS research required bridging the methodological divide between the “natural sciences” and “liberal arts” traditions. TIS researchers emphasized the potential benefits of methodological integration, advocating for research approaches that leverage both empirical rigor and interpretive depth.

Following this foundational discourse, an increasing number of TIS researchers advocated for incorporating both quantitative and qualitative methods to investigate T&I-related topics and phenomena (Hansen 2005). For example, in her discussion on empirical research in TIS, Hansen (2005) highlighted the flexibility of integrating quantitative and qualitative approaches, noting that various combinations and integrations were possible. She further stressed that there is no universally optimal way to combine methods, underscoring the context-dependent nature of methodological choices in T&I research. This period laid the groundwork for greater acceptance and experimentation with MMR in TIS, setting the stage for more structured and systematic applications of mixed-methods designs in the following decades.

The 2010s: Advocacy for and analysis of MMR practice

Graham Crow on methodological pluralism. [Source]

In the early 2010s, TIS scholars began to explicitly use the term “mixed-methods” to advocate for mixing of quantitative and qualitative methods in TIS research. This decade saw a shift from early theoretical and methodological discussions toward more structured advocacy and analysis of MMR practice. In a thought-provoking article on research methodology, Pöchhacker (2011) examines epistemological and ontological developments in interpreting studies and entertains their implications for methodological orientations. Particularly, he emphasized “multiple and mixed-methods approaches” as an epitome of a pragmatic research stance, arguing that such approaches have the potential to capture the complex nature of interpreting. In a similar vein, Tiselius (2011) critically evaluates research paradigms in conference interpreting studies in her article “Mixed-method design in interpreting studies: An untapped resource”. Drawing on her own research project, she highlights the value of MMR and justifies its role in advancing interpreting research methodology. Likewise, in an encyclopedia entry, Hild (2015) offers an overview of MMR in interpreting research projects, providing a concise yet systematic description of its methodological underpinnings, design features, and critical considerations.

Beyond theoretical discussions, empirical analyses of MMR practice in TIS research emerged during this period. Liu (2011) conducted a data-based review of 48 empirical studies published in Interpreting between 2004 and 2009. She found that a good portion of these studies adopted a mixed-methods approach, incorporating both quantitative and qualitative data collection and analysis. She also observed that multiple data collection methods were frequently used to capture different types of data, reinforcing the growing methodological pluralism in TIS.

This growing scholarly emphasis on MMR culminates in dedicated discussions of planning and implementing MMR studies in two textbooks on translation research methodology (Hale & Napier 2013; Saldanha & O’Brien 2013). Particularly, Saldanha and O’Brien (2013) advocated for MMR by emphasizing its cross-referencing and supplementarity benefits. They also described sequential and concurrent MMR designs and illustrated their application with examples from published research. Particularly, the authors highlighted the integration of corpus linguistics and critical discourse analysis as an effective way to explore linguistic features in translated and interpreted texts. They argue that MMR can uncover general linguistic patterns in translated language while providing rich sociolinguistic insights into the mechanisms underlying these patterns.

Sequential designs for mixed methods research, by David Morgan. [Source]

By the end of the 2010s, two methodological review articles offered an evidence-based analysis of MMR practice in TIS (Han 2018; Meister 2018). Meister (2018) examined 10 PhD dissertations completed within the Swedish-Finnish context between 1998 and 2012, given that these studies employed more than one analytical method and drew on multiple theoretical frameworks to address their research questions. She observed that, while these studies integrated different methods, most authors did not engage with the philosophical assumptions underlying their research, failed to justify their methodological choices, and did not explicitly reference MMR as their approach. Meister cautioned that the lack of explicit methodological discussion might stem from a shortage of methodological literature specifically aimed at early-career scholars, limiting their ability to conceptualize and articulate MMR frameworks within TIS.

In his turn, Han (2018) conducted a systematic content analysis of 312 empirical studies on interpreting, published between 2004 and 2014 across 36 mainstream T&I journals. This first large-scale investigation examined the prevalence, rationale, and design of MMR in the field. He found that 48% (n = 149) of studies employed MMR, with 16.3% (n = 51) using mono-strand designs (one data collection method analyzed both qualitatively and quantitatively) and 31.4% (n = 98) using multi-strand designs (multiple data collection strands integrated into a single study). Among mono-strand studies, 30 employed a mono-strand conversion design, where data were transformed for analysis but not integrated, while 21 used a truly mixed Survey (Qual + Quan) design, combining quantitative and qualitative data collection and analysis. Most multi-strand studies (n = 92) were truly mixed, integrating qualitative and quantitative components to derive meta-inferences.

The prevalence of truly mixed studies (36%) suggests an increasing tendency toward methodological pluralism in interpreting research, likely due to its interdisciplinary nature and absence of a dominant research paradigm. The primary rationale (69%) for MMR was complementarity (referred to as “supplementarity” in the present entry), followed by expansion (16%) and triangulation (11%; termed “cross-referencing” in the entry). Four prototype MMR designs were also identified: (1) parallel mixed, (2) sequential mixed, (3) conversion mixed, and (4) Survey (Qual & Quan), with conversion mixed (33%) and Survey (Qual & Quan, 19%) being the most common. Researchers also adapted these prototypes using addition, substitution, and embedding techniques, with parallel designs (n = 32) being more frequent than sequential designs (n = 13) probably due to resource constraints.

Overall, the 2010s witnessed a growing scholarly endorsement of MMR in TIS, moving beyond conceptual debates to more structured methodological analyses and practical guidance on its implementation. This decade laid the foundation for the systematic incorporation of MMR designs in TIS research, setting the stage for further methodological diversification in the 2020s.

The 2020s: Diversified applications

The beginning of the 2020s has witnessed application of MMR in a diverse array of TIS realms. In audiovisual translation and media accessibility, Black (2022) conducted a mixed-methods investigation into children’s reception of standard and integrated subtitles, by employing eyetracking, recognition memory tests, and interviews. Similarly, Hermosa (2022) demonstrates the use of physiological instruments within a mixed-methods framework for media accessibility research, by describing three MMR designs (i.e., convergent, explanatory, and exploratory) and examples of their application.In cognitive TIS, Stasimioti & Sosoni (2021) use keylogging and eyetracking data to assess the temporal, technical and cognitive effort exerted by translators during the post-editing of neural machine translation output. Their findings are further cross-referenced with a qualitative analysis of edit types to evaluate post-editing quality. Most recently, Rojo & Muñoz (2025) contributed an edited volume on research methods in cognitive TIS, offering a comprehensive examination of both qualitative and quantitative methods applied to examine cognitive aspects of T&I. This volume provides good food for thought as regards mixing and integrating substantively different research methods in future cognitive T&I research.

In machine translation (MT) and post-editing, Pérez (2020) uses mixed methods to examine translators’ perceptions of post-editing MT, blending survey data with qualitative interviews to uncover their fears, worries, and preferences. These studies underscore the importance of combining quantitative data with qualitative insights to understand the evolving role of MT in the translation industry. In interpreting studies, MMR has also proven valuable. In line with Saldanha & O’Brien’s (2013) suggestion, Gu & Gao (2023) apply MMR to explore ideology in interpreting, where corpus-based Critical Discourse Analysis (CDA) is employed alongside quantitative text analysis. In addition, Napier & Hale (2023) advocate for MMR in court interpreting research, showcasing a longitudinal multi-stage project that investigates the feasibility of deaf signers serving as jurors. Their work demonstrates how mixed methods can address the complexity of interpreting practices.

Challenges of mixed-methods research

Given the relatively recent adoption of MMR in TIS and based on the analysis of MMR practices in the field, TIS researchers face several key challenges related to MMR design, implementation and reporting. The overarching issue is ensuring and monitoring the quality and rigor of MMR studies. Hild (2015), Han (2018) and Meister (2018) have highlighted the methodological weaknesses and vulnerabilities of MMR studies within TIS. In particular, in his data-based methodological evaluation, Han (2018) underscores the need for periodic reviews to track patterns, trends, and methodological advancements in MMR practice within TIS. Associated with this overall challenge are six specific challenges that may undermine the quality and rigor of MMR in TIS:

Philosophical, theoretical and methodological choices remain often implicit.
Quasi-mixed designs tend to prevail.
Integrating quantitative and qualitative data is inherently difficult.
Scarce funding and support undermine resource- and skill-intensive research, such as MMR.
MMR design, implementation, data analysis, and results need to be transparent and more detailed.
Quantitative MMR findings are difficult to generalize.

Let us address these challenges with some more detail:

Paradigms: Positivism, interpretivism, critical realism for grad students. [Source]

(1) In MMR within TIS, researchers often do not explicitly state their philosophical worldviews. In addition, many studies fail to provide a clear and transparent rationale for choosing a particular type of MMR design. Moreover, researchers frequently neglect to use established notation systems to clearly label their MMR designs. For instance, few researchers explicitly identify their paradigmatic orientation or discuss the relationship between their philosophical worldview and research design (Han 2018; Meister 2018). This aligns with Pöchhacker’s (2011) observation that the ontological, epistemological, and methodological foundations of TIS research have received limited scholarly attention. Han (2018) and Meister (2018) also found that very few studies explicitly justify their use of MMR or include terms such as mixed-methods or mixed-methods study in article titles or keywords. Moreover, many researchers fail to apply established notation systems to clearly label their MMR designs. While the absence of explicit justifications and standardized labeling may not necessarily compromise the rigor of MMR studies, enhancing methodological transparency in design rationale, paradigmatic positioning, and notation use would contribute to greater awareness, consistency and standardization in MMR reporting within TIS. This, in turn, would facilitate comparability, replication and scholarly engagement with mixed-methods approaches in the field.

(2) Quasi-mixed designs, particularly the conversion mixed design, are relatively prevalent in TIS. In this design, data from one methodological strand (qualitative or quantitative) is transformed into the other format (i.e., qualitative data into numerical codes or vice versa) to facilitate further analysis. Saldanha & O’Brien (2013) and Hild (2015) have highlighted the widespread use of conversion mixed designs in TIS, particularly in cases where qualitative verbal data (e.g., think-aloud protocols or textual analysis) is quantitized into numerical categories for frequency-based statistical analysis. While this practice enables statistical comparison, it often lacks the depth of integration required for a truly mixed design. Empirical findings confirm the disproportionate reliance on conversion mixed designs in TIS. According to Han’s (2018) methodological analysis, 37% (n = 37) of the 113 MMR studies examined employed a conversion mixed design—a figure significantly higher than in other disciplines. Comparatively, Alise & Teddlie (2010) found no instances of conversion mixed design in psychology, sociology and nursing, and only one case in education, and Hashemi & Babaii (2013) reported only seven cases of conversion design in a sample of 205 articles from seven major applied linguistics journals from 1995 to 2008. Since quasi-mixed designs lack the full methodological rigor of traditional MMR approaches, the high representation of conversion mixed designs in TIS raises methodological concerns (Hild 2015; Han 2018). While conversion designs can serve specific analytical purposes, they should be used judiciously and combined with more robust integration strategies to ensure greater rigor and conceptual depth in MMR studies.

MMR data integration. [Source]

(3) Closely related to the issue of quasi-mixed designs is the methodological difficulty of truly mixing and integrating quantitative and qualitative data to derive meta-inferences (Hild 2015; Han 2018). In TIS, quantitative data—often derived from corpus-based studies, eyetracking experiments, or statistical measurements of T&I performance—provides numerical representations of patterns, correlations and general trends. In contrast, qualitative data, gathered through interviews, think-aloud protocols, ethnographic observations and textual analyses, offers in-depth insights into translators’ cognition, decision-making processes, and socio-cultural influences. The challenge lies in bridging these two forms of data to generate meaningful meta-inferences, rather than simply juxtaposing results from separate analyses. While the integration of quantitative and qualitative data has been widely debated and demonstrated in other disciplines (e.g., Hitchcock & Onwuegbuzie 2022), TIS has seen relatively little discussion on how to achieve true methodological and analytical integration (Hild 2015). Current strategies and frameworks for integration—such as joint displays and meta-matrix approaches—need to be critically examined, piloted and adapted to meet the unique requirements of TIS research. Without careful methodological planning, MMR in TIS risks resulting in parallel, non-integrated analyses, rather than a cohesive mixed-methods approach that genuinely enhances our understanding of T&I-related phenomena. Strengthening methodological awareness and fostering disciplinary discussions on data integration strategies will be crucial for advancing MMR in TIS.

(4) There is a conflict between the resource- and skill-intensive nature of MMR and the limited funding and institutional support available for TIS research. On the one hand, MMR is inherently resource-intensive, as it involves multiple research phases, diverse methodologies, and complex integration processes. Conducting rigorous, MMR studies requires substantial time, financial resources, methodological expertise, and often interdisciplinary collaboration. On the other hand, TIS is still a relatively young discipline, often operating within constrained budgets and limited institutional infrastructure, making it difficult to sustain the demands of MMR. Many TIS researchers work in small teams or as independent scholars, without access to the financial and human resources necessary for large-scale MMR projects. Another barrier is the need for specialized training in both qualitative and quantitative methodologies. Acquiring expertise in statistical analysis, experimental design, ethnographic methods, or computational techniques requires access to advanced courses, workshops, and mentorship, which are not always readily available to TIS researchers (e.g., Han, Lu & Zhang 2023). As a result, many scholars resort to quasi-mixed designs or less demanding MMR approaches that require fewer resources.

Empirical evidence supports this trend. Han (2018) found that about 60% and 30% of the 113 MMR studies relied on conversion and Survey (Qual & Quan) designs, respectively. Specifically, the Survey (Qual & Quan) design refers to a unique MMR design in TIS wherein a single research strand begins with a survey, primarily structured as a questionnaire that includes both closed- and open-ended questions. The closed-ended questions generate quantitative data (e.g., frequency counts, scale ratings) suitable for statistical analysis, while the open-ended questions elicit qualitative data (e.g., written feedback) for thematic or content analysis. He also found that a larger proportion of the 113 MMR studies examined used parallel (n = 38, 33.6%) than sequential designs (n = 17, 15%). These findings suggest that TIS researchers tend to favor designs that minimize time and effort in data collection. On the one hand, conversion and survey (Qual & Quan) designs are essentially the mono-strand approaches requiring less methodological complexity. On the other hand, parallel designs are typically less demanding than sequential designs, as they allow simultaneous data collection, whereas sequential MMR designs require at least two distinct time periods for data collection and analysis. Given these constraints, conducting a rigorous MMR study remains a significant challenge, particularly for under-resourced TIS researchers. Expanding institutional support, research funding, and methodological training opportunities will be critical to strengthening MMR practices in TIS and enabling more robust, fully integrated mixed-methods studies in the field.

Introduction to the Open Science Framework. [Source]

(5) Transparent and detailed reporting of MMR design, implementation, data analysis, and results is often lacking, especially in relation to the qualitative component of MMR studies. Empirical analyses indicate that qualitative findings in MMR-based studies are frequently underreported or even omitted altogether. For instance, in a review of 113 MMR articles in interpreting studies, Han (2018) observed that some studies explicitly stated in their methodology sections that qualitative components were conducted and analyzed, yet failed to report the qualitative results in their findings. Similarly, Hild (2015) underscores the importance of transparency in key aspects of the MMR research process, particularly in demonstrating the trustworthiness of qualitative data coding. There are two main reasons that may explain this lack of detailed reporting in TIS MMR studies. One reason is that TIS researchers may place greater emphasis on quantitative data and statistical results, treating qualitative data as secondary, particularly in embedded designs, where one method plays a supporting role to the other. The other reason is that many scholarly journals impose strict word limits, making it challenging to include detailed qualitative findings, which typically require more extensive descriptions than quantitative data. While increasing journal page limits may not be feasible in the near future, MMR researchers in TIS are encouraged to utilize open data repositories such as the Open Science Framework (OSF). The OSF is a free, open-source platform that supports transparent, reproducible, and collaborative research, allowing scholars to store, archive, and share detailed quantitative and qualitative datasets, coding frameworks, and supplementary results. This approach can enhance the rigor, credibility, and accessibility of MMR studies in TIS, facilitating peer review, replication, and broader scholarly engagement.

(6) The final challenge facing MMR researchers in TIS is related to the generalizability of statistical findings in the quantitative phase of an MMR study. Given that empirical TIS studies often rely on relatively small and homogeneous samples, particularly in observational and experimental research (Mellinger & Hanson 2017), there is a legitimate concern regarding the stability and external validity of quantitatively derived results. This issue is particularly problematic for MMR, as meta-inferences rely on integrating quantitative and qualitative findings. If the quantitative component lacks generalizability, it may undermine the broader conclusions drawn from an MMR study. To mitigate this challenge, MMR researchers in TIS should prioritize larger and more diverse samples whenever possible, integrating multi-site and multi-phase data to enhance representativeness and robustness. In addition, transparent reporting of effect sizes, confidence intervals, and potential biases is essential (Mellinger & Hanson 2017), as such information facilitates meta-analyses, allowing researchers to statistically synthesize quantitative findings for greater transferability and generalizability. Strengthening methodological rigor in the quantitative component will ultimately enhance the credibility and impact of MMR studies in TIS.

Future directions

With the technological advancement and the inherently interdisciplinary nature of TIS research, the field is expanding rapidly in terms of its scope and depth. Increasingly, TIS research projects incorporate multi-strand, multi-modal, and multi-lingual data to analyze T&I-related source texts, examine T&I processes, and evaluate T&I products. In this evolving research landscape, MMR is expected to play an even greater role, enabling T&I scholars to uncover emerging insights and explore new frontiers. To enhance our understanding of MMR practice in TIS, we need regular evidence-based methodological reviews and syntheses focusing on TIS publications to monitor design features, assess methodological developments, and ensure quality and rigor in MMR-based studies, such as Han (2018). Similar reviews are needed to systematically examine MMR in TIS and to analyze more recent trends, particularly after 2015. Such methodological reviews would serve multiple crucial purposes: (a) assessing the quality and rigor of MMR studies in TIS; (b) identifying best practices in research design, data integration, and analysis; (c) exposing methodological weaknesses and areas for improvement; and (d) mapping the evolution of MMR in TIS to guide future research.

Beyond methodological advancements, several key research areas in TIS stand to benefit from thoughtfully integrating quantitative and qualitative methods and analyses. Two such areas include the analysis of multimodal data in cognitive T&I studies and translation reception research. For example, qualitative thematic analysis of translators’ think-aloud protocols collected during the process of post-editing MT output can complement and shed insight into statistical results of translators’ eye-movement patterns such as gaze fixations, saccades, and scan paths. Similarly, in reception studies of subtitling, while such methods as eyetracking experiments, comprehension tests, cognitive load measurements, and large-scale surveys can be used in the quantitative arm of an MMR project to measure reception effects, in-depth interviews, focus groups, and ethnographic observations can be adopted in the qualitative arm of the MMR project to illuminate audiences’ subjective reception such as viewing challenges, preferences, emotional engagement, cultural attitudes, and viewing behaviors.

Another promising area for MMR in TIS is the use of exploratory sequential MMR designs to develop measurement instruments (e.g., psychological scales, questionnaires, and rating rubrics). As T&I research increasingly emphasizes accurate and reliable measurement, there has been a growing effort to develop field-specific instruments tailored to T&I-specific constructs and phenomena (e.g., Schaeffer, Huepe, Hansen-Schirra, et al. 2019). The exploratory sequential MMR design is particularly well-suited for scale development and validation because it follows a structured, phased approach: a) a qualitative phase is initiated to explore and understand a given T&I-related concept or phenomenon; and b) a quantitative phase is followed in which qualitative results from the first phase of the study inform the design of a quantitative instrument (e.g., a scale or questionnaire) to test or validate the qualitative findings on a larger scale. This approach ensures that measurement instruments are grounded in authentic, context-specific insights from qualitative research, while also undergoing rigorous psychometric validation in the quantitative phase.

Similarly, explanatory sequential MMR design can be used to shed insight into quantitative-statistical results and patterns for several emerging research topics. For instance, recent studies on automatic evaluation of T&I quality rely on large datasets of T&I samples to analyze the statistical relationship between human and machine-generated scores (e.g., Han & Lu 2025; Han, Lu & Chen 2025). While these studies primarily use quantitative methods (e.g., correlation analysis, regression models) to compare machine and human scoring patterns, they often overlook qualitative investigation into cases where machine scores deviate significantly from human ratings. An explanatory sequential MMR study could first conduct a statistical analysis to examine overall correlations and divergences between human and machine scores, followed by a qualitative analysis of T&I samples that exhibit scoring discrepancies, investigating specific linguistic, stylistic, or contextual factors that are not amenable to accurate machine scoring. This approach would help identify weaknesses in current automated scoring models and inform improvements in AI-driven evaluation tools. For another example, emerging efforts to investigate reception of translated work or media product through “big data” usually involve computational analysis of large amounts of textual data, including online discussions, reviews, or social media posts (e.g., Kotze, Janssen, Koolen et al. 2021), to reveal general sentiments, topics, and linguistic features. However, big data studies often lack deeper qualitative insights into the contextual meaning and nuances behind statistical results. A follow-up qualitative analysis, for example, using purposive sampling of highly engaged discussions, could be conducted to contextualize and explain the underlying reasons behind statistical patterns.

Lastly, with the growing importance of meta-research in TIS, T&I researchers have performed meta-reviews to generate comprehensive understandings of the state of the art on a diverse array of important topics (e.g., Han, Lu & Zhang 2023). An emerging opportunity in this area is the application of mixed-methods systematic reviews (MMSR) – a research synthesis approach that systematically integrates both quantitative and qualitative evidence to provide a comprehensive and nuanced understanding of a research topic (Heyvaert, Hannes & Onghena 2017). Unlike traditional systematic reviews, which often focus on either statistical meta-analysis (quantitative) or narrative synthesis (qualitative), MMSR strategically combines both to generate meta-inferences that integrate findings from both strands to reveal broader theoretical and methodological implications. By adopting MMSR approaches, T&I scholars can enhance the methodological rigor of systematic reviews, foster evidence-based decision-making, and bridge the gap between empirical findings and theoretical insights.

Research potential

As TIS continues to evolve, MMR offers a powerful framework for addressing complex research questions by integrating quantitative and qualitative approaches. While challenges such as methodological rigor, resource constraints, and data integration remain, MMR presents significant opportunities for advancing cognitive T&I research, reception studies, T&I quality assessment, field-specific instrument development, and meta-research in the field of TIS, as has been described in detail in the prior section. By fostering greater methodological transparency, training, and systematic reviews, future MMR studies can further enhance the depth, validity, and impact of T&I research.