TRACE for Evidence-to-Claim Coherence in Undergraduate Thesis Writing: Development and Descriptive Evaluation
Keywords:
undergraduate research; evidence-to-claim coherence; claim calibration; inference errors; PhilippinesAbstract
Evidence-to-claim coherence requires thesis conclusions and recommendations to remain traceable to research questions, analytical results, evidence, warrants, and methodological boundaries. Grounded in scholarship on argumentation, validity, feedback literacy, and rubric assessment, this study developed the Thesis Reasoning and Argument Coherence Evaluation Framework (TRACE) for undergraduate thesis writing. TRACE integrates an Evidence–Claim Traceability Matrix, a six-dimensional Claim Calibration and Coherence Rubric, a Taxonomy of Research Inference Errors, and a secondary Argument Coherence Index (ACI). Framework development combined conceptual synthesis, exploratory examination of 42 anonymized thesis chapter sets, expert-guided refinement, and rater calibration. A subsequent descriptive evaluation involved 120 third-year undergraduates in four intact thesis-writing sections at a public teacher-education institution in Central Luzon, Philippines. Two sections received TRACE-supported instruction; two received equal-duration conventional thesis-writing instruction without TRACE tools. Ten doctoral-level experts in research methodology, academic writing, educational assessment, instrument development, curriculum, measurement, or thesis supervision appraised high-level framework statements (S-CVI/Ave = .92). Three trained raters scored baseline drafts and final outputs; average-measure absolute-agreement coefficients were ICC (2,3) = .854 and .883, respectively. All sections had higher final means, but one comparison section showed a trajectory comparable to that of the TRACE sections. Very low internal consistency among final dimension scores (? = .09) argued against a homogeneous-scale interpretation of the ACI. TRACE is recommended for cautious formative and diagnostic use, with the dimension profile primary and the ACI provisional.
https://doi.org/10.26803/ijlter.25.8.38
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.testingstandards.net/open-access-files.html
Arias-Hermoso, R., Garro Larrañaga, E., & Imaz Agirre, A. (2025). Assessing academic and disciplinary literacies: Rubric validation to measure argumentation, comparison and source-based writing skills. Journal of Language and Education, 11(3), 60–75. https://doi.org/10.17323/jle.2025.27560
Baelen, R. N., Gould, L. F., Felver, J. C., Schussler, D. L., & Greenberg, M. T. (2023). Implementation reporting recommendations for school-based mindfulness programs. Mindfulness, 14, 255–278. https://doi.org/10.1007/s12671-022-01997-2
Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299
Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305–314. https://doi.org/10.1037/0033-2909.110.2.305
Braun, V., & Clarke, V. (2025). Reporting guidelines for qualitative research: A values-based approach. Qualitative Research in Psychology, 22(2), 399–438. https://doi.org/10.1080/14780887.2024.2382244
Browne, K. M., Drewes, A., Smalley, G., & Lichtenwalner, S. (2026). Data description, claim, evidence, reasoning (DCER): An instructional framework to develop undergraduates’ data literacy skills and scientific reasoning. Journal of College Science Teaching, 55(3), 235–250. https://doi.org/10.1080/0047231X.2026.2633359
Chien, S.-C., & Li, W.-Y. (2024). Perceptions of supervisors and their doctoral students regarding the problems in writing the doctoral dissertation results section. English for Specific Purposes, 76, 14–27. https://doi.org/10.1016/j.esp.2024.06.003
Curtis, K., Chong, S.-W., & Kong, M. S. (2025). A qualitative synthesis of research into the use of exemplars in the English for Academic Purposes context to develop student feedback literacy. Research Synthesis in Applied Linguistics, 1(2), 302–339. https://doi.org/10.1080/29984475.2025.2495271
Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of Marketing Research, 38(2), 269–277. https://doi.org/10.1509/jmkr.38.2.269.18845
Du, H., & List, A. (2024). Evidence-based reasoning: Results from an intervention. Applied Cognitive Psychology, 38(5), Article e4238. https://doi.org/10.1002/acp.4238
Duncan, R. G., & Chinn, C. A. (2025). Evaluating the quality of argumentation: The role of epistemic ideals and reliable processes. Cognition and Instruction, 43(3), 201–232. https://doi.org/10.1080/07370008.2025.2497240
Flowerdew, L., & Petri?, B. (2024). A critical review of corpus-based pedagogic perspectives on thesis writing: Specificity revisited. English for Specific Purposes, 76, 1–13. https://doi.org/10.1016/j.esp.2024.05.003
Golparvar, S. E., Hu, G., & Seyedi, S. E. (2025). Cohesion in the discussion section of research articles: A cross-disciplinary investigation. English for Specific Purposes, 77, 1–19. https://doi.org/10.1016/j.esp.2024.08.004
Grainger, P., Scott, J. J., Mulgrew, K., Dean, M., Carey, M. D., & Johnston, C. (2025). Developing a formative assessment criteria-referenced tool (FACT) for doctoral students. Educational Research and Evaluation, 30(7–8), 748–765. https://doi.org/10.1080/13803611.2025.2556048
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3
Grohnert, T., Gromotka, L., Gast, I., Delnoij, L., & Beausaert, S. (2024). Effective master’s thesis supervision: A summative framework for research and practice. Educational Research Review, 42, Article 100589. https://doi.org/10.1016/j.edurev.2023.100589
Harsch, C., Koval, V., Kanistra, P. V., & Delgado-Osorio, X. (2024). Validating an integrated reading-into-writing scale with trained university students. Assessing Writing, 62, Article 100894. https://doi.org/10.1016/j.asw.2024.100894
Hu, G., & Bonsu, E. M. (2025). A cross-disciplinary study of value arguments in doctoral theses submitted to universities in Hong Kong. English for Specific Purposes, 79, 1–16. https://doi.org/10.1016/j.esp.2025.02.002
Jose, L. S. (2026). Improving methodological coherence in thesis proposal design: Development and preliminary validation of the Research Coherence Alignment Framework. International Journal of Learning, Teaching and Educational Research, 25(6), 588–615. https://doi.org/10.26803/ijlter.25.6.25
Jönsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002
Jusslin, S., & Hilli, C. (2024). Supporting bachelor’s and master’s students’ thesis writing: A rhizoanalysis of academic writing workshops in hybrid learning spaces. Studies in Higher Education, 49(4), 712–729. https://doi.org/10.1080/03075079.2023.2250809
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Kurt, B., & Kafes, H. (2025). The role of disciplinary enculturation in stance-taking in L2 academic writing. Acta Psychologica, 260, Article 105720. https://doi.org/10.1016/j.actpsy.2025.105720
Levitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 26–46. https://doi.org/10.1037/amp0000151
Lymer, G., Lindwall, O., & Greiffenhagen, C. (2024). Student writing in higher education: From texts to practices to textual practices. Linguistics and Education, 80, Article 101247. https://doi.org/10.1016/j.linged.2023.101247
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46. https://doi.org/10.1037/1082-989X.1.1.30
McNeish, D., & Stapleton, L. M. (2016). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51(4), 495–518. https://doi.org/10.1080/00273171.2016.1167008
McShane, B. B., Bradlow, E. T., Lynch, J. G., Jr., & Meyer, R. J. (2024). “Statistical significance” and statistical reporting: Moving beyond binary. Journal of Marketing, 88(3), 1–19. https://doi.org/10.1177/00222429231216910
Musaeus, P., Prilop, C. N., Dikilita?, K., Sundset, M. A., Svensen, C., Ershova, T., Boud, D., Lindberg, A. B., Bjælde, O. E., Gray, R. M., Jr., ?stencio?lu, T., Kvernenes, M., Lassesen, B., & Raaheim, A. (2026). Teacher feedback literacy in higher education: Navigating enablers and constraints. Assessment & Evaluation in Higher Education, 51(2), 315–331. https://doi.org/10.1080/02602938.2025.2591556
Panadero, E., Delgado, P., Zamorano-Sande, D., Pinedo, L., Fernández Ortube, A., & Barrenetxea-Mínguez, L. (2025). Putting excellence first: How rubric performance level order and feedback type influence students’ reading patterns and task performance. Learning and Instruction, 99, Article 102168. https://doi.org/10.1016/j.learninstruc.2025.102168
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199
Praestegaard Larsen, B. (2025). Research process model for bachelor’s thesis. Journal of Learning Development in Higher Education, (34). https://doi.org/10.47408/jldhe.vi34.1379
Qiu, H., Cook, A. J., & Bobb, J. F. (2024). Evaluating tests for cluster-randomized trials with few clusters under generalized linear mixed models with covariate adjustment: A simulation study. Statistics in Medicine, 43(2), 201–215. https://doi.org/10.1002/sim.9950
Quinlan, K. M., & Pitt, E. (2025). Evaluative feedback isn’t enough: Harnessing the power of consequential feedback in higher education. Assessment & Evaluation in Higher Education, 50(4), 577–591. https://doi.org/10.1080/02602938.2024.2430596
Rutkowski, D., Rutkowski, L., Thompson, G., & Canbolat, Y. (2024). The limits of inference: Reassessing causality in international assessments. Large-scale Assessments in Education, 12, Article 9. https://doi.org/10.1186/s40536-024-00197-9
Teng, M. F., & Ma, M. (2024). Assessing metacognition-based student feedback literacy for academic writing. Assessing Writing, 59, Article 100811. https://doi.org/10.1016/j.asw.2024.100811
Toulmin, S. E. (2003). The uses of argument (Updated ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511840005
Yan, D. (2024). Rubric co-creation to promote quality, interactivity and uptake of peer feedback. Assessment & Evaluation in Higher Education, 49(8), 1017–1034. https://doi.org/10.1080/02602938.2024.2333005
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Luijim Santos Jose, Jacqueline Gumallaoi, Jonalyn Sad-ayan-Lacambra, Jessica Ayawan, Imelda Binay-an, Cynthia Carino, Olga Tresmanio
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published by IJLTER are licensed under a Creative Commons Attribution Non-Commercial No-Derivatives 4.0 International License (CCBY-NC-ND4.0).
