Measuring Malaysian Lower Secondary Learners ’ Reading Ability on a CEFR-Aligned Text

To capture learners ’ interest in reading and help them understand the content of their reading, it is important to write at a level that aligns with their reading abilities. This will motivate learners to read and comprehend the material. A CEFR-aligned textbook is a valuable tool for improving the English language skills of lower secondary learners in English Language classrooms. Therefore, it is significant to assess learners ’ reading competence in text comprehension using a reading text from a CEFR-aligned textbook. These assessments were conducted based on a narrative text from the CEFR-aligned textbook using two different techniques: miscue analysis and retelling. The sample comprised 20 ‘ C ’ grade learners randomly selected from Malaysian lower secondary schools. The data on miscues were analysed quantitatively, following Goodman, Watson, and Burke ’ s In-Depth Procedure (2005). The quality of retelling was evaluated based on the criteria set by Irwin and Mitchell ’ s (1983) 5-point scoring method. The findings revealed that all 20 learners achieved the target descriptor scale B1, indicating that they can read simple texts on topics relevant to their field and interests with sufficient comprehension. This study implies that CEFR-aligned reading texts are appropriate for lower


INTRODUCTION
Reading helps develop vocabulary and spelling skills, which are essential in oral and written communication. It is a language-receptive process that begins with the author's linguistic surface depiction and ends with the reader's production of meaning. In acquiring reading skills, the need for learners to focus on the words, sentences, and meaning in reading material is fundamental for comprehension.
Reading helps students discover new concepts and information. They learn the required information, improving their vocabulary and other language skills. The improvement in language and vocabulary helps learners in other components of language skills. For example, if learners possess rich vocabulary, they can understand more as they can listen better. Then, they can write and speak better because of their rich word power. In schools, learners have various reading levels, the good, the average, and the weak. Reading ability is the capacity to read, understand, and interpret written words in various contexts. The difference in their reading ability level is due to factors such as learners' motivation, interest, social background, and the difficulty level of the reading texts (Kennedy, 2010).
The use of vocabulary and sentence structures determines text difficulty. If the vocabulary is within the learner's vocabulary size, it will enhance their understanding of the text. Otherwise, text comprehension would be hindered. Sikorová (2011) asserts that a textbook can be a helpful teaching tool if learners can read and comprehend it. Conversely, Kovač and Mohar (2022) emphasised that a challenging reading book may obstruct learning and make pupils less motivated. Additionally, too simple or complex content could dull learners and make them less motivated.
Nowadays, learners are more into browsing the internet and playing with their mobile phones. They read fewer printed materials unless they are forced to read. However, the expanding availability of digital materials and the lengthening of learners' reading sessions have started to impact their reading habits. Yusuf et al. (2019) claimed that the country's reading culture has declined due to the widespread use of digital gadgets. The young find it much easier to access digital devices compared to books. Thus, learners read less of printed reading materials. This leads them to a poor grasp of vocabulary and sentence structures.
Learners in classrooms fail to pay attention during reading activities due to an inadequate grasp of vocabulary and sentence structures. Learners' negative attitude towards reading printed material impacts their reading ability level. According to Woody et al. (2010), learners read fewer printed reading materials, and this affects learners' reading proficiency levels. Students' altered attitudes affect the teaching and learning environment in schools regarding reading printed materials. For example, in classrooms where the reading activity is conducted, learners ignore their friend who is reading aloud.
An English language textbook aligned with the CEFR (Common European Framework of Reference) was implemented in secondary schools in 2017, particularly for lower secondary students, to enhance the lower secondary students' competency level. Therefore, the CEFR-aligned material must be evaluated for its applicability to students. CEFR is a framework that includes a description of communicative proficiency organised into six levels and a descriptive scheme for analysing what is needed in language learning and usage. The descriptions show what a specific language learner level is capable of doing.
Identifying the reading ability level of a student can assist teachers in selecting appropriate reading content and allowing students to maintain their interest in reading. This would help learners progress from their current reading ability in text comprehension to the next level. The study answers the following research questions: 1. What are students' strengths and weaknesses in reading a CEFR-aligned reading passage? 2. What are the learners' reading ability levels in text comprehension based on CEFRaligned reading texts?
The study contributes to the literature by bridging the gap by examining students' reading behaviour using a CEFR-aligned reading text to determine if they can read and comprehend the material assigned to them.

Reading Ability
Reading ability is the capacity of a reader to communicate with a writer to obtain knowledge and information (Brown, 2001). Gillet and Temple (2012) identify three levels of reading proficiency: 1) The Independent Level -At this level of difficulty, the learners can read material without assistance. In general, reading comprehension is excellent; 2) The Instructional Level -The content is not particularly easy at this level, but it is still manageable. The learner is challenged in this area and will gain the most from education. Although comprehension is strong, assistance is required to grasp some ideas; 3) The Frustration Level -The language or concepts are too tough to read correctly at this level. Major ideas are lost or misconstrued, resulting in poor comprehension.
Many factors contribute to learners' levels of reading ability. One of the aspects is that they spend less time reading printed materials because learners are occupied with electronic devices. This causes a decline in learners' reading habits. As a result, learners' exposure to new vocabulary and sentence structures decreases. In classrooms, when learners are instructed to read by the teacher, they do not perform well because they lack word power and less exposure to sentence structures. Kovač and Mohar (2022) consider text difficulty a crucial issue in text comprehension, especially its effect on readers' ability to evaluate comprehension. Weaver and Bryant (1995) proposed the optimum effort hypothesis, which argues that students could better predict their comprehension when text materials matched their reading level rather than being too simple or too complex. In other words, learners' capacity to anticipate comprehension depends significantly on the complexity of the texts. Therefore, learners' comprehension can be predicted most accurately when the text difficulty level corresponds to their reading proficiency.

Miscue Analysis
According to the essential assumption underlying miscue analysis, reading is a psycholinguistic activity that occurs from an interaction between the mind and language (Goodman et al., 2005). By analysing readers' miscues in oral reading, Goodman (2001) devised a study approach called miscue analysis to dig into readers' underlying processes and strategies. Miscue analysis is a student-centred examination of reading processes and comprehension ability (Goodman et al., 2005). A deviation or departure from the text is referred to as a miscue. These differences indicate the difficulties readers face and the cueing methods they use to deduce meaning from text.
The miscue patterns reveal three primary language cueing systems and distinct cognitive approaches in the readers' word attack capabilities and comprehension abilities: 1) The grapho-phonics system, which deals with spelling and phonic relationships; 2) the syntax system, which deals with grammar and structure of the language, and; 3) the semantic system, which deals with the meaning of words, phrases, and sentences (Goodman, 2001). Miscue analysis includes substitution, insertion, omission, self-correction, repetition, hesitancy, and reversal. It is a great way to learn more about a learner's reading skills and pinpoint specific weaknesses.

Retelling
Retelling has emerged as a viable method for evaluating reading proficiency due to its effectiveness, emphasis on active text reconstruction, format uniformity across text kinds, and formative evaluation of reading skills (Kucer, 2010). Retelling is a valuable teaching and evaluation tool since it allows the reader to concentrate on specific aspects of the story structure. Retellings may be classified into three typesunassisted, aided, and specified -with the unassisted recounting happening shortly after the reading (Goodman, 2001). The method of unaided retelling is utilised in this study because it allows learners to narrate anything they remember about the text they read. Learners must use various linguistic skills to absorb the text's information, recall it accurately, and restructure it meaningfully in their language output to retell it successfully. Recounting is the ability of the learners to accurately state important story aspects in their oral retelling after reading a piece, such as characters, place, initial events, problem, and conclusion (Cohen & Cowen, 2011).
Oral retelling measures a wide variety of comprehension abilities that can be used during instruction and intervention, unlike other reading comprehension tests (Reed & Vaughn, 2012). Learners must mentally construct comprehension from reading and then recount information about the text as they comprehend it (Goodman, 2001). It helps learners organise, summarise, and process facts. To do so, learners must recall every detail, select the most crucial details, and correctly recount the story. It is a post-reading practice in which learners, orally or in writing, describe what they remember after reading (Lapp et al., 2010).
According to Lapp et al. (2010), retelling may reflect the purpose that learners see in the stories they read and the difficulty that they have in putting the numerous components of a story together into a cohesive whole. Researchers have suggested significant findings regarding recounting. According to Morrow (1985), retelling is an active method that engages adults and children in reconstructing a text. Retelling a story aloud helps learners grasp and remember what they have learned (Morrow, 2005). The researchers have proposed significant results concerning recounting. Retelling a story aloud improves understanding and retention of the information (Morrow, 2005).

Research Design
A qualitative study was conducted to determine learners' reading ability levels in the lower secondary classes. A qualitative approach was needed because it provides the depth and details of learners' reading ability through miscue analysis and retelling techniques.

Research Site
The research was conducted in five non-residential lower secondary schools in Perak, Malaysia. Five districts were selected. They were Batang Padang, Manjung, Hilir Perak, Kuala Kangsar, and Hulu Perak. Two schools from each district were chosen randomly.

Participants
The sample consisted of 20 participants, with five selected from each school. The selection of participants from the schools in each district was based on a simple random sampling method that included both genders from various races, including Malays, Chinese, and Indians, as well as different social backgrounds. The participants all had a grade of 'C' in their English language, which corresponds to an academic grading system in Malaysia that ranges from 40 to 59 marks.

Instrument
A reading passage titled "The Story of the Mobile Phone Novel" from the CEFRaligned textbook (page 21) was chosen for miscue analysis, and a set of four 'whquestions (information questions)' was used for the retelling session. This particular text was chosen because learners were familiar with mobile phones. The following sub-sections discuss the appropriate steps involved in the data analysis of the assessments employed in this study.

Miscues analysis
The passage was prepared as a coding sheet for easier marking of miscues. The passage was read aloud by the participants, recorded, and replayed for the markings of miscues on the coding sheet. Miscues looked for and analysed were self-correction (correcting the word to understand the passage), insertion (inserting a new word into the phrase), omission (omitting letters or words), repetition (repeating a word), substitution (using a different word instead of the original word). The miscues are considered as participants' strengths if they do not impair the sentence syntactically or semantically (Goodman et al., 2005).
Participants' strengths and weaknesses were analysed based on the miscue percentages. Miscues such as self-correction and repetitions were considered as participants' strengths because they did not distort the meaning and structure of the sentences. Miscues are viewed as weaknesses when they distort the sentences' meaning and structural components. The percentage for each miscue is calculated by the total number divided by the overall number of the miscue patterns and multiplied by 100, adapted from Janan (2011) for this study.

Retellings
Miscue analysis was followed by unaided retellings. Participants were tested on their level of text comprehension. There were four 'wh-questions' based on the passage "The Story of the Mobile Phone Novel" to test the participants' comprehension. Example of questions: 1. Where did the story happen? 2. What was amazing in the story? 3. Are Akiko's phone novels different from paperback books? If yes/no, how? and 4. What did you learn from the story? Irwin and Mitchell's (1983) 'judging of the richness of retelling' was used to analyse the participants' scores. It gave the participants a comprehensive assessment of their ability to recognise overall text structure, primary ideas, and important information, summarise, and generalise beyond the text. A score was given to each participant's retelling, ranging from one to five. At level 1, the participants can only relate details, the lowest level, while they can relate a few key concepts at level 2. At level 3, participants can relate big ideas. At level 4, participants can summarise, and at level 5, they can extrapolate beyond the text by connecting the information in the tale to their everyday lives. For this study, a 5-point scale indicates the participants' degree of text comprehension. The higher the richness level in retellings, the higher the participants' understanding of the text. If the retelling level is lower, the text comprehension is poorer.
After each retelling, the score was circled in the retelling analysis sheet. The total score was counted in the form of percentages. It indicated the participants' comprehension level of the text. After evaluating the participants' text comprehension skills, the total marks were totalled, and the descriptive statistical analysis results were expressed as percentages for the participants' scores. To demonstrate the degrees of retelling in text comprehension, the data is shown in bar charts to exhibit the levels of retelling. Table 2 shows the score of retelling.
To check the reliability, two instructors with 20 years of experience served as inter-rater to ensure the data gathered was reliable. They listened to the recorded retellings of a participant and double-checked the individual's verbatim transcript. The two inter-rater graded the retellings in the retelling analysis sheets. To establish dependability between the inter-rater, the researcher employed a percentage agreement. The inter-rater grades for each retelling were added together, and the result was divided by the number of ratings and multiplied by 100 to get the percentage. The percentage of agreement was 83%, which is an acceptable figure indicating that the inter-rater was in accord.

Miscues Percentages
This section reports the findings for the first research question: What are students' strengths and weaknesses in reading a CEFR-aligned reading passage? Table  1 shows participants' miscues. The words listed in Table 1 were the major miscues committed among the 20 participants. Firstly, in the sentence 'Akiko wasn't an experienced writer and didn't expect anyone to be interested in her tale', the word 'experience' is self-corrected by P1 and P12 substituted by 'experiencing'. The use of the word 'experiencing' has distorted the structure and the meaning of the sentence. For the word 'interested', P2 inserted the syllable suffix '-ing' and read it as 'interesting', which did not match the structure and was ungrammatical. P3 read the word as 'interest' whereby the prefix '-ed' is omitted. This distorted the structure and the meaning of the sentence.
In the sentence 'They write in a simple, chatty style and use lots of abbreviations and emoticons, P1 self-corrected the word 'abbreviations', whereas P5, P11, and P20 repeated it. They read the word slowly and managed to read after the fourth attempt. Four participants (P4, P5, P9, and P12) had problems with the word 'emoticons'. P4 omitted the initial 's', and the rest three participants repeated the word. They paused at the syllable 'emo' and continued with the syllable 'cons'. The complete word was then pronounced. However, the omission of the initial 's' by P4 maintained the sentence's meaning but was syntactically ungrammatical.
Next, in the sentence 'She worked on whenever she could, mostly while she was relaxing at home', four participants (P3, P7, P15, and P16) repeated the word 'whenever', and they had the same behaviour in reading the word. They paused at the word 'when' and continued with the word 'ever'. They were very cautious in their attempt to read the word. After the third try, they read the complete word. P7 and P15 self-corrected the word; firstly, they read as 'wheneve' and then read as 'whenever'. In this sentence, 'They thought they were a bad influence on teenage girls, who preferred them to proper literature', the word 'influence' was self-corrected by P3 and P8. P3 pronounced the first two syllables of the word and paused, then continued to pronounce the whole word, whereas P8 gave a first try and read the second time correctly.
For the sentence 'Akiko was one of the first of a new generation of novelists in Japan who write novels', five participants (P9, P12, P13, P18, and P20) made miscues in reading the word 'generation'. P9, P12, and P18 repeated the word to gain an understanding of the word, whereby P13 and P20 self-corrected the word. P13 and P20, when correcting themselves, managed with the first two syllables, 'gene' of the word, then continued with the third and fourth syllable, 'ration'. For the word 'originally' in the sentence 'Many of the best-selling paperback books in Japan were originally mobile novels!', P14 repeated the word and self-corrected, whereas P19 repeated the word but did not self-correct.
For the word 'preferred' in the sentence 'They thought they were a bad influence on teenage girls, who preferred them to proper literature', P12 and P18 self-corrected. Both participants had difficulties pronouncing the word. The 'r' sound seemed prominent in this word. Thus, the participants had to correct themselves to understand the word, whereas P15 paused at this word and repeated it to ensure proper pronunciation. For the word 'literature', P4 and P6 had difficulties pronouncing it. P4 paused at the syllable 'lit' and then continued with the syllable 'erature'; the same goes with P6. As for the sentence 'Some people were critical of mobile novels at first', P7 and P10 self-corrected the word 'critical'. Table 2 shows the number of each miscue performed by the participants. P1 performed two self-correction miscues (2/13x100=15%). This revealed the participants' strengths in oral reading because the miscues did not distort the sentences' meaning semantically or syntactically. P2 committed one miscue, which was insertion (1/1x100%=100%), and it was the participants' weakness because the miscue made the sentence ungrammatical. P3 made two miscues, omission (1/1x100% =100%) and self-correction (1/13x100=7%). The omission miscue was considered the participants' weakness, and self-correction was the participants' strength. P4 made three miscues, two omissions (2/2 x100%=100% -weaknesses and one repetition (1/20x100=5%-strength). P5 made two repetition miscues (2/20x100%=10%). Both miscues were regarded as strengths.

Retellings
The finding for the second research question "What are the learners' reading ability levels in text comprehension based on CEFR-aligned reading texts?" is shown in Table 3. Upon reading the text, the participants answered a total of four comprehension questions. For the first comprehension question, all 20 participants scored 5. For the second question, 12 participants scored 4, and the rest scored 5 (P4, P5, P6, P7, P15, P16, P19, and P20). For question 3, all the participants scored 4; for the last question, P9, P10, P11, P17, P18, P19, and P20 scored 5. The overall percentage was rated good, with all participants scoring in the range of 80-95. As the scores for the retellings were above 80, which falls under the 'A' category, the reading passage is within the comfortable range of the participants' proficiency.
Even though the participants had difficulty pronouncing several words mentioned in Table 1 and could not understand them, such as "abbreviations", "emoticons", and "literature", they were nevertheless able to answer the information questions correctly. Figure 1 shows a bar chart of the participants' scores; two participants scored 95%, ten participants scored 90%, and the rest eight scored 85%. The target descriptor scale B1 was attained by all 20 participants, who can read simple texts about topics relating to their field and interests with a sufficient degree of understanding (CEFR BI descriptor).

The Students' Strengths and Weaknesses in Reading a CEFR-Aligned Reading Passage
This section of the discussion is based on the first research question: "What are students' strengths and weaknesses in reading a CEFR-aligned reading passage?" This research question was answered by analysing the miscues made by the participants. The findings revealed 37 miscues in participants' oral reading. Out of 37 miscues, only four weaknesses were identified; the rest 33 were participants' strengths. As the percentages of strengths outperformed the weaknesses in participants' oral reading, the CEFR-aligned reading passage can be concluded to be within the comfortable reading level for the participants, which suggests that it is within the range of participants' reading ability.
It is apparent from the findings that although the participants are from different geographical areas, they had the same reading behaviour, whereby repetition and selfcorrection were the common miscues they produced. The participants did sound frustrated throughout these miscues as they tried to grasp the pronunciation of the words; they did it slowly to ensure that they were reading a word correctly. Besides, they kept track of their progress and were confident about their reading. While reading slowly, repeating, and self-correcting the difficult words, participants recalled words from their schemata to sense or guess if they had encountered them. Nation (2019) also agreed that slow reading might result in better comprehension.
Self-correction is crucial in reading process because it helps students read more effectively. Prompts were also beneficial, and by pausing they enabled students to selfcorrect. This self-correction helps students read more effectively. Elicitation, a metalinguistic cue, and a clarification request are all prompts that invite self-correction (Stahl, 2011). Self-correction and repetition are other essential factors in determining which students can read a more challenging text. Learners who invest time and effort to review, reread (repeat), and self-correct have a personal interest in their reading (Stahl, 2011). Therefore, it can be concluded that participants understood the paragraph well.

The Students' Comprehension Levels Based on a CEFR-Aligned Reading Passage
The second research question, "What are students' comprehension levels based on a CEFR-aligned reading passage?" is discussed in this section. Based on the retellings of the passage "The Story of the Mobile Phone Novel", the findings revealed that all 20 participants had good scores. The scores range from 80 to 95. For question 3 'Are Akiko's phone novels different from paperback books? If yes/no, how?', none of the participants scored 5 for this question because they could not accurately compare paperback books and phone novels. However, students obtained a score of 4 to 5 for the rest of the questions.
According to Bloom's Taxonomy, all four questions provided to the participants were at the knowledge and understanding level. Despite participants having a grade of 'C', they could generate responses at levels 4 and 5. Participants at level 4 could produce a summing remark, and at level 5, they could generalise beyond the text by relating the story's substance to their daily lives. The results revealed that participants' retelling is comprehensive beyond their language proficiency level. It can be concluded that the CEFR-aligned text intended to the lower secondary learners corresponds to their reading ability. Regarding text difficulty, the reading passage does not contain syntactic complexity and lexical items which hinders participants' comprehension of the text in the CEFR-aligned textbook. As in this study, the text consisted of words which students were unfamiliar with. For example, the word 'abbreviations' were beyond learners' ability to pronounce and comprehend. However, unfamiliar vocabulary can be inferred if students know an adequate number of other words in the text (Webb et al., 2023). Previous studies have shown that the success rate of guessing vocabulary from the context is promising under this circumstance (Hamada, 2009;Nassaji, 2003).
To develop and expand learners' word lists, teachers must expose them to various words in the classroom. They will require various materials and instructional techniques to address students' comprehension of words and retelling demands in the classrooms. The teachers must provide a model of reading the passage aloud to the students so they can read it independently. Therefore, learners' fluency and phonetics skills can be improved instead of just word drills. On the learners' part, they must put in more effort to read widely to improve their reading comprehension level. They must discover ways to read regularly to upgrade their proficiency from where they currently are to a higher level. It takes dedication to get them through, and assigning texts appropriate for their reading level and interests can be profitable because they will enjoy reading them and gain more linguistic knowledge (Gopal & Mahmud, 2019).

CONCLUSION
The culmination of reading is comprehension, and to achieve this, the texts provided to the students must be within their readability level (Gopal et al., 2021). In assessing the reading ability of Malaysian lower secondary learners based on the CEFR-aligned textbook, the reading passages catered for the learners are within the reading proficiency of the learners. This is demonstrated by the type of errors made and the retelling evaluation, which revealed an independent level of comprehension.
Nonetheless, in ESL reading circumstances, fluency issues are evident. Therefore, fluency practice should be enhanced in ESL reading lessons, with a focus on phonics and phonemic awareness. This will help students improve their pronunciation and articulation, thereby minimising reading hurdles.
One limitation of this study is that the findings may not be applicable to the entire nation, as it only covers a few districts in Malaysia. Additionally, time constraints were also a limiting factor. The reading and retelling sessions were conducted within a fortyminute time limit of forty minutes during participants' rest intervals. For future research, follow-up studies could involve more research techniques, such as observations or observational notes in miscue analysis and retellings. Including different research techniques will help determine whether students can read and comprehend printed texts, thereby revealing their levels of text comprehension.