Volume 10, Issue 3 (9-2025)
IJREE 2025, 10(3): 1-15 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Azmoode M. (2025). EFL Students’ Rating Accuracy in Assessing Reading Comprehension Subskills across Genres: A Diagnostic Perspective. IJREE. 10(3),
URL: http://ijreeonline.com/article-1-1002-en.html
URL: http://ijreeonline.com/article-1-1002-en.html
Department of foreign Languages, Science and Research University, Tehran, Iran
Abstract: (608 Views)
This study aimed to examine EFL learners’ rating accuracy in assessing reading comprehension subskills in various genres. To this end, 60 English translation students participated in this study. The instructional treatment was based on the learners’ challenging reading subskills in four genres. Taking the instructor’s ratings as the yardstick, during a 12- week course, the accuracy of the learners’ self- and peer- assessments was investigated. Data analysis, using MANOVA, confirmed that there was a statistically significant difference between the accuracy of self-, peer- and instructor-ratings. More specifically, the two groups were inaccurate in assessing the main idea/ supporting details and cause/effect subskills in all genres. However, for assessing fact /opinion subskills only the self-assessment group was inaccurate in descriptive genre. Obtaining such detailed diagnostic information about learners’ performance can help instructors in elevating weaknesses in language skills.
Keywords: reading comprehension, reading subskills, reading genres, rating accuracy, diagnostic assessment
References
1. Alderson, J. C., Clapham, C. M., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
2. Alderson, J. C., Brunfaut, T., & Harding, L. (2015). Towards a theory of diagnosis in second and foreign language assessment: Insights from professional practice across diverse fields. Applied Linguistics, 36(2), 236- 260. [DOI:10.1093/applin/amt046]
3. Alfallay, I. (2004). The role of some selected psychological and personality traits of the rater in the accuracy of self- and peer- assessment. System, 32(3), 407-425. [DOI:10.1016/j.system.2004.04.006]
4. Alibakhsh, G. (2013). Construction and validation of self-assessment inventory for English for academic purposes: A case of Iranian tertiary students. RALs, 4(2), 93-109.
5. Aminu, N., Hamdan, M., & Russell, C. H. (2021). Accuracy of self evaluation in a peer learning environment: an analysis of a group learning model. SN Social Science, 1(185), 1- 17. [DOI:10.1007/s43545-021-00152-3]
6. Aryadoust, V. (2019). A review of comprehension subskills: A scientometrics perspective. System, 88, 1-16. [DOI:10.1016/j.system.2019.102180]
7. Ashton, K. (2014). Using self-assessment to compare learners' reading proficiency in a multilingual assessment framework, System, 42, 105-119. [DOI:10.1016/j.system.2013.11.006]
8. Azmoode, M., Kiany, G. R., & Abbasian, G. R. (2024a). Diagnostic assessment of interactional competence in paired speaking tests: Investigating rating accuracy of Iranian EFL learners. Journal of Language and Translation, 4(1), 19-33. [DOI:10.30495/ttlt.2024.709364]
9. Azmoode, S. M., Kiany, G. R., & Abassian, G. R. (2024). On the effect of diagnostic self-, and peer- assessment on reading comprehension: Examining EFL learners' diagnostic rating accuracy across various genres. Journal of Modern Research in English Language Studies, 11(2), 177-202. [DOI:10.30479/jmrels.2023.18703.2204]
10. Bachman, L. F. (4002). Fundamental considerations in language testing. Oxford University Press.
11. Bailey, A. L., & Heritage, M. (2018). Self-Regulation in learning: The role of language and formative assessment. Harvard Education Press.
12. Butler, Y. G. (2018). The role of context in young learners' processes for responding to self- assessment items. The Modern Language Journal, 102(1), 1-20. [DOI:10.1111/modl.12459]
13. Benson, P., & Voller, P. (2014). Autonomy and independence in language learning. Routledge. [DOI:10.4324/9781315842172]
14. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment Evaluation and Accountability, 21(1), 5-31. doi:10.1007/s11092-008-9068-5 [DOI:10.1007/s11092-008-9068-5]
15. Butler, Y. G., & Lee, J. (2006). On-task versus off-task self-assessments among Korean elementary school students studying English. The Modern Language Journal, 90(4), 506-518. https://doi: 10.1111/j.1540-4781.2006.00463.x [DOI:10.1111/j.1540-4781.2006.00463.x]
16. Butler, Y. G., & Lee, J. (2010). The effects of self-assessment among young learners of English. Language Testing, 27(1), 5-31. [DOI:10.1177/0265532209346370]
17. Carrell, P. L., & Connor, U. (1991). Reading and writing descriptive and persuasive texts. Modern Language Journal, 75(3), 314-324. [DOI:10.1111/j.1540-4781.1991.tb05361.x]
18. Chen, Y. M. (2008). Learning to self-assess oral performance in English: longitudinal case study. Language Teaching Research, 12(2), 235-262. [DOI:10.1177/1362168807086293]
19. Davis, F. B. (1944). Fundamental factors of comprehension in reading. Psychometrika, 9(3),185-197. [DOI:10.1007/BF02288722]
20. Dörnyei, Z., & Ushioda, E. (2013). Teaching and researching motivation. Routledge. [DOI:10.4324/9781315833750]
21. DuBravac, S., & Dalle, M. (2002). Reader question formation as a tool for measuring
22. comprehension: Narrative and expository textual inferences in a second language. Journal of Research in Reading, 25(2), 217-231. [DOI:10.1111/1467-9817.00170]
23. Earl, L. M. (2012). Assessment as learning: Using classroom assessment to maximize student learning. Corwin Press.
24. Elahi, M. S. (2016). Assessing and improving general English university students' main sub-skills of reading compression: A case of university of Bojnord. Sino-US English Teaching, 13(4), 245-260. https://doi:10.17265/1539-8072/2016.04.002 [DOI:10.17265/1539-8072/2016.04.002]
25. Esfandiari, R., & Jafari, H. (2021). Morphological complexity across descriptive expository, and narrative text types in Iranian lower- intermediate language learners. Issue in Language Teaching, 10(1), 237-267. [DOI:10.22054/ilt.2021.59736.580]
26. Esfahani, K. M., Rashtchi, M., Abousaidi, R. A., & Mowlaie, B. (2022). Promoting metacognitive awareness in writing assessment tasks through planning, monitoring, and evaluation: Achievements and perceptions. Iranian Journal of English for Academic Purposes, 11(3), 50-68. dor: 20.1001.1.24763187.2022.11.3.4.9
27. Farhady, H., & Daftarifard, P. (2006). On the scalability of the components of the reading comprehension ability: A progress report. In H. Farhady (Ed.), Twenty-five years of living with applied linguistics: Collection of articles (pp. 189-204). Rahnama publisher.
28. George, D., & Mallery, P. (2020). IBM SPSS statistics 26 step by step: A simple guide and reference. Routledge. [DOI:10.4324/9780429056765]
29. Goodman, K. S. (1967). Reading: A psycholinguistic guessing game. Journal of the Reading Specialist, 6(4), 126-135. [DOI:10.1080/19388076709556976]
30. Goral, D. P., & Bailey, A. L. (2019). Student self-assessment of oral explanations: Use of language learning progressions. Language Testing, 36(3), 1- 27. [DOI:10.1177/0265532219826330]
31. Green, A. (2018). Assessment for learning in language education. Iranian Journal of Language Teaching Research, 6(3), 9-18. https://files.eric.ed.gov/fulltext/EJ1192609.pdf
32. Han, C., & Riazi, M. (2018). The accuracy of student self-assessments of English-Chinese bidirectional interpretation: A longitudinal quantitative study. Assessment & Evaluation in Higher Education, 43(3), 386-398. [DOI:10.1080/02602938.2017.1353062]
33. Jang, E. E. (2009). Demystifying a Q-Matrix for making diagnostic inferences about L2 reading skills. Language Assessment Quarterly, 6(3), 210-238. https://doi.org/ 10.1080/ 15434300903071817
https://doi.org/10.1080/15434300903071817 [DOI:10.1080/ 15434300903071817]
34. Javidanmehr, Z., & Anani Sarab, M. R. (2019). Retrofitting non- diagnostic reading comprehension assessment: Application of the G- DINA model to a high stakes reading comprehension test. Language Assessment Quarterly, 16(3), 294-311. https://doi.org/ 10.1080/15434303.2019.1654479 [DOI:10.1080/15434303.2019.1654479]
35. Karakoc, A. I. (2019). Reading and listening comprehension Subskills: The match between theory, course books, and language proficiency tests. Advances in Language and Literary Studies (ALLS), 10(4), 166_171. [DOI:10.7575/aiac.alls.v.10n.4p.166]
36. Khalifa, H., & Weir, C. J. (2009). Examining reading: Research and practice in assessing second language reading. Cambridge University Press.
37. Kim, A. (2015). Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability. Language Testing, 32(2), 227-258. [DOI:10.1177/0265532214558457]
38. Kim, Y. H., & Jang, E. E. (2009). Differential functioning of reading subskills on the OSSLT for L1 and ELL students: A multidimensionality model-based DBF/DIF approach. Language Learning, 59(4), 825-865. doi:10.1111/j.1467-9922.2009.00527.x [DOI:10.1111/j.1467-9922.2009.00527.x]
39. Lave, J. (1993). The practice of learning. In S. Chaiklin & J. Lave (Eds.), Understanding practice. Perspectives on activity and context. Cambridge University Press. [DOI:10.1017/CBO9780511625510.002]
40. Lee, Y. W. (2015). Diagnosing diagnostic language assessment. Language Testing, 32(3) 299 -316. doi:10.1177/0265532214565387 [DOI:10.1177/0265532214565387]
41. Lee, S. K., & Chang, S. H. (2005). Lerner involvement in self- and peer- assessment of task- based oral performance. Second Language Research, 41, 711-735. file:///C:/Users/SMA/Downloads/10.+2231654.pdf
42. Lee, Y. W., & Sawaki, Y. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly, 6(3), 239-263. [DOI:10.1080/15434300903079562]
43. Liu, H. H. (2014). The conceptualization and operationalization of diagnostic testing in second and foreign language assessment. Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics, 14(1), 1-12.
44. Liu, H. H., Alderson, J. C., & Brunfaut, T. (2015). Diagnostic assessment of reading and listening in a second or foreign language: Elaborating on diagnostic principles. Language Testing, 32(3), 317-336. [DOI:10.1177/0265532214564505]
45. Lu, L. (2018). An analysis of peer-assessment in Chinese as a second language classroom presentation. Chinese Language Teaching Methodology and Technology, 1(3), 18. https://engagedscholarship.csuohio.edu/cltmt/vol1/iss3/3
46. Mazloomi, S., & Khabiri, M. (2016). Diagnostic assessment of writing through dynamic self-assessment. International Journal of English Linguistics, 6(6), 19-31. doi:10.5539/ijel.v6n6p19 [DOI:10.5539/ijel.v6n6p19]
47. Ma, W., & Winke, P. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time? Foreign Language Annals, 52(1), 66-86. [DOI:10.1111/flan.12379]
48. Markey, M. (2020). Using diagnostic assessment to investigate challenges in second language reading. The Reading Matrix: An International Online Journal, 20(1), 45-65. https://readingmatrix.com/files/22-045p942v.pdf
49. Nalbantoğlu, Y, F. (2017). Reliability of scores obtained from self-, peer-, and teacher-assessments on teaching materials prepared by teacher candidates. Educational Sciences: Theory & Practice, 17, 395-409. http://dx.doi.org/10.12738/ estp.2017.2.0098
50. Ng, M. C. W. (2018). Assessment for/as learning in Hong Kong English language classrooms: A review. IJREE, 3(3), 1-12. doi:10.29252/ijree.3.3.1 http://ijreeonline.com/article-1-104-en.html [DOI:10.29252/ijree.3.3.1]
51. Oscarson, M. (2013). Self-assessment in the classroom. In A. J. Kunnan (Ed), The companion to language assessment (pp. 712-729). Wiley-Blackwell [DOI:10.1002/9781118411360.wbcla046]
52. Ou, W. J. A. (2022). Writing accessible theory in ecology and evolution: Insights from cognitive load theory. BioScience, 72(3), 300-313. [DOI:10.1093/biosci/biab133]
53. Oxford, R. L. (2016). Teaching and researching language learning strategies: Self-regulation in context. Routledge. [DOI:10.4324/9781315719146]
54. Paleczek, L., Seifert, S., Schwab, S., & Gasteiger-Klicpera, B. (2015). Assessing reading and spelling abilities from three different angles-correlations between test scores, teachers' assessment and children's self-assessments in L1 and L2 children. Procedia - Social and Behavioral Sciences, 174, 2200-2210. https://doi.org/ 10. 1016/j.sbspro.2015.01.876
https://doi.org/10.1016/j.sbspro.2015.01.876 [DOI:10. 1016/j.sbspro.2015.01.876]
55. Pang, N. S. (2020). Teachers' reflective practices in implementing assessment for learning skills in classroom teaching. ECNU Review of Education, 1-21. doi:10.1177/2096531120936290 [DOI:10.1177/2096531120936290]
56. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self- regulated learning. Educational Psychology, 36(2), 89-101. https://doi.org/ 10.1207/S15326985EP3602_4 [DOI:10.1207/S15326985EP3602_4]
57. Ravand, H. (2015). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 34(8), 782-799. https://doi.org/ 10.1177/0734282915623053 [DOI:10.1177/0734282915623053]
58. Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research, and Evaluation, 11(10), 1-13. doi: [DOI:10.7275/9wph-vv65]
59. Rost, D. H. (1993). Assessing different components of reading comprehension: Fact or fiction? Language Testing, 10(1), 79-92. https://doi.org/ 10.1177/026553229301000105 [DOI:10.1177/026553229301000105]
60. Rouhi, A., Jafarigohar, M., Alavi, M., & Hosseini. Y. (2015). Task difficulty of macro-genres and reading strategies and reading comprehension. International Journal of Asian Social Science, 5(11), 656-677. https://doi.org/ 10.18488/journal.1/2015.5.11/1.11.656.677 [DOI:10.18488/journal.1/2015.5.11/1.11.656.677]
61. Şahin, A. (2013). The effect of text types on reading comprehension. Mevlana International Journal of Education, 3(2), 57-67. https://doi.org/ 10.13054/mije.13.27.3.2 [DOI:10.13054/mije.13.27.3.2]
62. Santos, L., & Semana, S. (2015). Developing mathematics written communication through expository writing supported by assessment strategies. Educational Studies in Mathematics, 88(1), 65-87. http://doi.org/10.1007/s10649-014-9557-z [DOI:10.1007/s10649-014-9557-z]
63. Sawaki, Y., Kim, H. J., & Gentile, C. (2009).Q-matrix construction: Defining the link between constructs and test items in large-scale reading and listening comprehension assessments. Language Assessment Quarterly, 6(3), 190- 209. [DOI:10.1080/15434300902801917]
64. Schunk, D. H. (2004). Learning theories: An educational perspective. Merrill Prentice.
65. Singh Negi, J., & Laudari, S. (2022). Challenges of developing learner autonomy of english as a foreign language (EFL) learners in underprivileged areas. IJREE, 7(2), 65-80. doi:10.52547/ijree.7.2.65 [DOI:10.52547/ijree.7.2.65]
66. http://ijreeonline.com/article-1-689-en.html
67. Song, M. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25(4), 435-464. [DOI:10.1177/0265532208094272]
68. Spearritt, D. (1972). Identification of subskills of reading comprehension by maximum likelihood factor analysis. Reading Research Quarterly, 8(1), 92-111. [DOI:10.2307/746983]
69. Surbakti, R., Umboh, E. S., Pong, M., & Dara, S. (2024). Cognitive load theory: Implications for instructional design in digital classrooms. International Journal of Educational Narrative, 2(6), 483-493. [DOI:10.70177/ijen.v2i6.1659]
70. Suzuki, Y. (2015). Self-assessment of Japanese as a second language: The role of experiences in the naturalistic acquisition. Language Testing, 32(1) 63-81. doi:10.1177/0265532214541885 [DOI:10.1177/0265532214541885]
71. Tengberg, M. (2018). Validation of sub-constructs in reading comprehension tests using teachers' classification of cognitive targets, Language Assessment Quarterly, 1-13. [DOI:10.1080/15434303.2018.1448820]
72. Toledo, P. F. (2005). Genre analysis and reading of English as a foreign language: Genre schemata beyond text typologies. Journal of Pragmatics, 37, 1059-1079. https://doi.org/ 10.1016/j.pragma.2005.01.002 [DOI:10.1016/j.pragma.2005.01.002]
73. Vacca, R. T. (1980). A study of holistic and subskill instructional approaches to reading comprehension. Journal of Reading, 23(6), 512-518. https://www.jstor.org/stable/40028836
74. Vygotsky, L. (1978). Mind in society. The development of higher psychological processes. Harvard University Press.
75. van Kraayenoord, C. E., & Paris, S. G. (1997). Australian students' self-appraisal of their work samples and academic progress. The Elementary School Journal, 97(5), 523-537. [DOI:10.1086/461879]
76. Xiao,Y., & Lucking, R. (2008). The impact of two types of peer assessment on students' performance and satisfaction within a Wiki environment. Internet and Higher Education, 11, 186-193. [DOI:10.1016/j.iheduc.2008.06.005]
77. Yin, J. (2018). A review on researches of "genre-based teaching approaches" in recent 20 years in China. International Journal of Secondary Education, 6(1), 16-23. https://doi.org/ 10.11648/j.ijsedu.20180601.14 [DOI:10.11648/j.ijsedu.20180601.14]
78. Yoshida, M. (2012).The interplay of processing task, text type, and proficiency in L2 reading Reading in a Foreign Language, 24(1), 1-29. https://files.eric.ed.gov/fulltext/EJ974102.pdf [DOI:10.64152/10125/66671]
79. Zhou, L., & Siriyothin, P. (2011). Effects of text types on advanced EFL learners' reading comprehension. Journal of Language and Culture, 30(2),
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
This work is licensed under a Creative Commons Attribution-NonCommercial (CC BY-NC 4.0)