Tag: design-based research

Extending the Discussion – Week 4: Educational Research Methods

A.M. Ternullo

Extending the Discussion –   Week 4 Educational Research Methods

Early discourse in educational technology research were focused on the difference between quantitative, experimental research and qualitative, descriptive research. Quantitative research designs are privileged in that discussion as though they illuminate generalizable truths, while qualitative methods may be viewed as illuminating specific, local truths. The discourse has since shifted to adopting mixed-methods approaches so the right tool can be employed for the research task at hand (Cobb, 2003; Foster, 2023; Jacobsen et al., 2023). Design-based research seems to be emerging in the discourse as a top contender for “gold standard” status of research in educational technology.

Design-based research does not privilege one qualitative or quantitative study. Rather, the process of research, the question posed, and the desired outcome of the research should shape and determine what processes are applied to gain an understanding (Jacobsen et al., 2023; Sandoval, 2014). Research is an iterative process – and when a researcher starts out looking at a topic, the questions asked are not fully formed and shaped because information is gathered during the research process (Jacobsen et al., 2023). Since the question evolves based on the phase and researcher’s knowledge, the methodologies employed may also need to evolve as the study progresses (Jacobsen et al., 2023). Cobb (2002) pointed out that a “primary goal for a design experiment is to improve the initial design by testing and revising conjectures as informed by ongoing analysis …” (p. 11.) Even though Cobb is speaking specifically to student learning, this goal underscores the iterative process of specifically educational research that may be overlooked in strictly quantitative or qualitative research designs, where the questions do not evolve much during the process.

Jacobsen et al. (2023) analyzed two student dissertations to illustrate the iterative process of design-based approaches in educational research. The methods to achieve understanding aren’t as important as having an open mind for this iterative process. The goal of methodological alignment should be to make sure that the questions asked by researchers can be “operationalized at each phase” of the process and are “precise” so the questions can be answered proficiently by the research (p.5). Qualitative methods should be applied when the question calls for it, just as quantitative methods should. Results from all aspects of investigation should be analyzed, compared and contrasted, and synthesized to make meaning.

The most compelling aspect of design research for me so far is that it breaks down silos of scientific vs. non-scientific, qualitative vs. quantitative, and hard vs. soft science. It opens up the discourse to focus not on how educational researchers approach questions, but what questions we are asking and what value those answers will have on the field of educational technology.

References

Cobb, P., Confrey, J., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9-13.

Foster, C. (2023). Methodological pragmatism in educational research: From qualitative-quantitative to exploratory-confirmatory distinctions. International Journal of Research & Method in Education, 1-16. https://doi.org/10.1080/1743727x.2023.2210063

Jacobsen, M., & McKenney, S. (2023). Educational design research: Grappling with methodological fit. Educational Technology Research and Development. https://doi.org/10.1007/s11423-023-10282-5

Sandoval, W. (2013). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18-36. https://doi.org/10.1080/10508406.2013.778204

Annotation – Educational design research: Grappling with methodological fit

A.M. Ternullo

Jacobsen et al. (2023) present a conceptual framework to assess methodological fit based on educational design research (EDR) — a term that includes all research approaches that enhance practice and advance scientific understanding. Jacobsen at al. situate the framework in the current discussion and debate of educational methodologies. Researchers seeking out theoretical or practical research must  identify problems worth study which are legitimate, researchable, and research-worthy in theoretical and/or practical terms.  

Jacobsen et al. further explore three orientations of EDR trajectories, which they define as research for interventions, research on interventions, and research through interventions. Research for interventions add to theoretical knowledge and design work. Research on interventions aims to provide information on an intervention’s characteristics. Research through interventions focuses on implementation processes of an intervention. These trajectories are usually combined and used for comparative analysis in EDR work. Jacobsen et al. use two recent dissertations to examine and illustrate the EDR trajectories they describe.

The discussion of the conceptual framework comes to a close through a discussion of why methodological fit is so challenging for researchers. Jacobsen et al. point out methodological fit depends on a variety of factors such as: the researcher’s research expertise in the area, expertise in methodologies, concerns of the researcher and other practical considerations. Four specific challenges to applying the correct methodological fit are identified: asking beginner level questions, focusing on state-of-art, rather than state of practice, insufficient measures for causal inferences, and absence of synthesis. Jacobsen et al. conclude the field of educational research needs more EDR examples to show how valuable this type of research can be.

Jacobsen et al.’s conceptual framework’s main strength comes first from the way the discussion is situated in the current discourse of methodological framework. The analysis of two dissertations to illustrate the concepts of the orientations of EDR trajectories was very strong. Elements of research design were shown at various stages of the dissertation process to illustrate and highlight the iterative nature of creating questions to shift focus of the orientation EDR trajectory. Jacobsen et al. also point out the pitfalls of this type of research for the novice researcher – which is what doctoral students are – and underscore the significance of support and mentorship from faculty if students pursue this avenue of research.

As a doctoral student knowing a dissertation is on the horizon, this conceptual framework is helpful thinking about potential topics and approaches. The most interesting sections of the article for me were the determinants of research-worthy problems and the orientations of EDR trajectories. This also connects back to Salomon & Perkins (2005) and the discussion of a concept of, with, and through technology, though this time focusing on specific intervention, which may or may not be technology. EDR is a complex, but rich way to analyze topics using mixed-methodologies that are brought to bear on a research topic as the research grows.

References

Jacobsen, M., & McKenney, S. (2023). Educational design research: Grappling with methodological fit. Educational Technology Research and Development. https://doi.org/10.1007/s11423-023-10282-5

Salomon, G., & Perkins, D. (2005). Do technologies make us smarter? Intellectual amplification with, of and through technology. In R. J. Sternberg, & D. D. Preiss (Eds). Intelligence and technology: The impact of tools on the nature and development of human abilities (pp. 71-86). Mahwah, NJ: Lawrence Erlbaum Associates.