Emerging digital factual storytelling in English language learning: Investigating multimodal affordances

Wawan Gunawan, Silmy Arizatul Humaira, Tri Indri Hardini, Mahardhika Zifana

Abstract


Attention has been given to multimodal texts to investigate their potential meaning affordances that facilitate learning and raise awareness of ideological meanings. However, how learners learn to make meaning by integrating intermodal relations involving language and visual images, especially in the context of learning English as a foreign language (EFL), has not been researched much. This study investigated the emerging process of meaning-making in digital factual storytelling practices in a senior high school in Indonesia. Fifty-six students participated in this study and each produced a digital storytelling video (DST). The DST videos were analyzed using an intersemiotic complementary framework to reveal the occurrences and typical intermodal meaning-making practices, as shown in the student digital storytelling (DST) videos on autobiography. The analysis focused on describing the emerging modes of making experiential meanings of the intermodal verbal-visual relations. It was found that the process of meaning-making was dominantly constructed in an exposition manner involving the verbs within clauses for identifying and describing. The emerging meanings resulting from the multimodal affordances allow storytellers to track with the potential meanings projected by the images. The selection of digital images helps students explore words when telling a story, shifting the conventional genre of autobiography. This study indicates the need to emphasize the purpose of text and to afford multimodal features to support the achievement of communication purposes.

Keywords


digital storytelling; experiential meaning; intermodal text; intersemiotic meaning

Full Text:

PDF

References


Alghamdi, M., & Reed, M. C. (2019). The value students and instructors place on multimodal composition within academic life. Arab World English Journal (AWEJ), 10(1), 283-297. https://doi.org/10.24093/awej/vol10no1.24

Arslan, S. (2020). Multimodal writing to promote global competence for EFL learners. Sakarya University Journal of Education, 10(3), 589-608. https://doi.org/10.19126/suje.777878

Bateman, J. (2014). Text and image: A critical introduction to the visual/verbal divide. Routledge.

Bull, G., & Kajder, S. (2005). Digital storytelling in the language arts classroom. Learning & Leading with Technology, 32(4), 46-49.

Chan, E. (2011). Integrating visual and verbal meaning in multimodal text comprehension: Towards a model of intermodal relations. In S. Dreyfus, S. Hood, & M. Stenglin (Eds.), Semiotic margins: Meaning in multimodalities (pp. 145-147). Bloomsbury Academic.

Coffin, C. (2006). Historical discourse: The language of time, cause, and evaluation. Continuum.

Cohn, N. (2020). Visual narrative comprehension: Universal or not? Psychonomic Bulletin & Review, 27(2), 266-285. https://doi.org/10.3758/s13423-019-01670-1

Derewianka, B., & Jones, P. (2016). Teaching language in context (2nd ed.). Oxford University Press.

García-Pinar, A. (2024). Initiating language engagement with multimodal learning tasks. Studies in English Language and Education, 11(3), 1316-1332. http://dx.doi.org/10.24815/siele.v11i3.35450

Gunawan, W. (2018). Re-appropriation of ideational meanings through drawings: A case of 5-year-old child experience in learning to mean. Indonesian Journal of Applied Linguistics, 8(2), 244-253. https://doi.org/10.17509/ijal.v8i2.13300

Halliday, M. A. K., & Matthiessen, C. M. I. M. (2004). An introduction to functional grammar (3rd ed.). Arnold.

Halliday, M. A. K., & Matthiessen, C. M. I. M. (2013). Halliday’s introduction to functional grammar. Routledge.

Hoecker, R. (2014). Visual narrative and trauma recovery. Narrative Inquiry, 24(2), 259-280. https://doi.org/10.1075/ni.24.2.05hoe

Hull, G. A., & Nelson, M. E. (2005). Locating the semiotic power of multimodality. Written Communication, 22(2), 224-261. https://doi.org/10.1177/0741088304274170

Jiang, L. (2017). The affordances of digital multimodal composing for EFL learning. ELT Journal, 71(4), 413-422. https://doi.org/10.1093/elt/ccw098

Jiang, L. (2018). Digital multimodal composing and investment change in learners’ writing in English as a foreign language. Journal of Second Language Writing, 40, 60-72. https://doi.org/10.1016/j.jslw.2018.03.002

Jiang, L., & Gao, J. (2020). Fostering EFL learners’ digital empathy through multimodal composing. RELC Journal, 51(1), 70-85. https://doi.org/10.1177/0033688219898565

Kang, J. (2021). Adolescent English language learners’ digitally mediated multimodal compositions: Multimodal enactment across different genres of writing in the EFL context. English Teaching: Practice & Critique, 20(2), 196-209. https://doi.org/10.1108/etpc-07-2020-0069

Kang, J. (2022). Developing multimodal communicative competence: Adolescent English learners’ multimodal composition in an after‐school programme. Literacy, 56(4), 355-370. https://doi.org/10.1111/lit.12294

Kim, D., & Li, M. (2021). Digital storytelling: Facilitating learning and identity development. Journal of Computers in Education, 8, 33-61. https://doi.org/10.1007/s40692-020-00170-9

Kim, Y., & Kang, S. (2020). Writing to make meaning through collaborative multimodal composing among Korean EFL learners: Writing processes, writing quality, and student perception. Computers and Composition, 58, 102609. https://doi.org/10.1016/j.compcom.2020.102609

Knapp, P., & Watkins, M. (2005). Genre, text, grammar. University of New South Wales Press, Ltd.

Kress, G. (2000). Multimodality: Challenges to thinking about language. TESOL Quarterly, 34(2), 337-340. https://doi.org/10.2307/3587959

Kress, G. (2003). Literacy in the new media age. Routledge.

Kress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.). Routledge.

Kress, G., & van Leeuwen, T. (2020). Reading images: The grammar of visual design. Routledge.

Lai, C. J. (2024). Enhancing multimodal output in CLIL education: The impact of VR games on fourth-grade students’ English poster designs and presentations in Taiwan. Humanities and Social Sciences Communications, 11(1), Article number 1465. https://doi.org/10.1057/s41599-024-03999-y

Lee, S. Y., Lo, Y. H. G., & Chin, T. C. (2021). Practicing multiliteracies to enhance EFL learners’ meaning-making process and language development: A multimodal problem-based approach. Computer Assisted Language Learning, 34(1-2), 66-91. https://doi.org/10.1080/09588221.2019.1614959

Lim, V. F. (2004). Developing and integrative multi-semiotic model. In K. O’Halloran (Ed.), Multimodal discourse analysis: Systemic functional perspectives (pp. 220-246). Continuum.

Martinec, R., & Salway, A. (2005). A system for image-text relations in new and old media. Visual Communication, 4(3), 337-371. https://doi.org/10.1177/1470357205055928

Meier, S. (2022). Digital storytelling: A didactic approach to multimodal coherence. Frontiers in Communication, 7, 906268. https://doi.org/10.3389/fcomm.2022.906268

Mestre-Mestre, E. M. (2015). The construction of meaning in the second language classroom: A multimodal discourse analysis. Procedia - Social and Behavioral Sciences, 173, 228-233. https://doi.org/10.1016/j.sbspro.2015.02.057

Mestres, E. T., & Pellicer-Sánchez, A. (2019). Young EFL learners’ processing of multimodal input: Examining learners’ eye movements. System, 80, 212-223. https://doi.org/10.1016/j.system.2018.12.002

Mills, K. A., Stone, B. G., Unsworth, L., & Friend, L. (2020). Multimodal language of attitude in digital composition. Written Communication, 37(2), 135-166. https://doi.org/10.1177/0741088319897978

Morell, T. (2015). International conference paper presentations: A multimodal analysis to determine effectiveness. English for Specific Purposes, 37, 137-150. https://doi.org/10.1016/j.esp.2014.10.002

Norris, S. (2012). Multimodality in practice: Investigating theory-in-practice-through-methodology. In S. Norris (Ed.), Multimodality in practice (pp. 222-226). Routledge. https://doi.org/10.4324/9780203801246-18

Olugbara, C. T., Letseka, M., & Olugbara, O. O. (2022). A systematic review of digital storytelling as an educational tool for teaching and learning in Southern Africa. In J. Olivier, A. Oojorah, & W. Udhin (Eds.), Multimodal learning environments in Southern Africa (pp. 165-195). Palgrave Macmillan. https://doi.org/10.1007/978-3-030-97656-9_9

Pan, X., & Zhang, Z. (2020). An empirical study of application of multimodal approach to teaching reading in EFL in senior high school. International Journal of Emerging Technologies in Learning, 15(2), 98-111. https://doi.org/10.3991/ijet.v15i02.11267

Peng, J. E., Zhang, L., & Chen, Y. (2017). The mediation of multimodal affordances on willingness to communicate in the English as a foreign language classroom. TESOL Quarterly, 51(2), 302-331. https://doi.org/10.1002/tesq.298

Purwaningtyas, T. (2020). Didactic symbol of visual images in EFL textbook: Multimodal critical discourse analysis. Pedagogy: Journal of English Language Teaching, 8(1), 51-63. https://doi.org/10.32332/pedagogy.v8i1.1959

Ravelli, L., Adami, E., Boeriis, M., Veloso, F. O., & Wildfeuer, J. (2018). Visual communication: Mobilizing perspectives. Visual Communication, 17(4), 397-405. https://doi.org/10.1177/1470357218794838

Robin, B. R. (2008). Digital storytelling: A powerful technology tool for the 21st-century classroom. Theory into Practice, 47(3), 220-228. https://doi.org/10.1080/00405840802153916

Salsabila, S., Nurami, N., Oktaviana, R. T., Muljanto, S., & Kurnia, A. (2023). Seeing how multimodal sources are used in EFL classrooms. English Education and Applied Linguistics Journal, 6(2), 102-106. https://doi.org/10.31980/eeal.v6i2.50

Shin, D. S., & Cimasko, T. (2008). Multimodal composition in a college ESL class: New tools, traditional norms. Computers and Composition, 25(4), 376-395. https://doi.org/10.1016/j.compcom.2008.07.001

Smith, B. E., Pacheco, M. B., & Khorosheva, M. (2021). Emergent bilingual students and digital multimodal composition: A systematic review of research in secondary classrooms. Reading Research Quarterly, 56(1), 33-52. https://doi.org/10.1002/rrq.298

Towndrow, P. A., & Pereira, A. J. (2018). Reconsidering literacy in the 21st century: Exploring the role of digital stories in teaching English to speakers of other languages. RELC Journal, 49(2), 179-194. https://doi.org/10.1177/0033688218754943

Unsworth, L., & Herrington, M. (2022). Visualization type and frequency in final-year high school science examinations. Research in Science Education, 53, 707-725. https://doi.org/10.1007/s11165-022-10094-2

Unsworth, L., & Mills, K. A. (2020). English language teaching of attitude and emotion in digital multimodal composition. Journal of Second Language Writing, 47, 100712. https://doi.org/10.1016/j.jslw.2020.100712

Wahyuni, W., Sujoko, S., & Sarosa, T. (2018). Improving students’ speaking skill through project-based learning (digital storytelling). English Education, 6(2), 161-161. https://doi.org/10.20961/eed.v6i2.35943

Wang, H. (2022). International English learners’ perspectives on multimodal composing and identity representation via multimodal texts. SAGE Open, 12(2), 21582440221103526. https://doi.org/10.1177/21582440221103526

Wong, D. (2019). Digital storytelling using both keyboard and pen. In D. Wong (Ed.), Digital humanities and new ways of teaching (pp. 195-218). Springer. https://doi.org/10.1007/978-981-13-1277-9_11

Wood, S., & Stubbs, K. (2000). Targeting text. Blake Education.

Yang, J., & Zhang, Y. (2014). Representation meaning of multimodal discourse: A case study of English editorials in The Economist. Theory and Practice in Language Studies, 4(12), 2564-2575. https://doi.org/10.4304/tpls.4.12.2564-2575

Yang, W., Chan, A., & Gagarina, N. (2024). Remote online language assessment: Eliciting discourse from children and adults. Frontiers in Communication, 9, 1463182. https://doi.org/10.3389/978-2-8325-5142-4

Yeh, H.-C. (2018). Exploring the perceived benefits of the process of multimodal video making in developing multiliteracies. Language Learning & Technology, 22(2), 28-37. https://doi.org/10125/44642

Zhang, Y., Peng, J., & Zheng, Y. (2024). Teachers’ perceptions of implementing digital multimodal composing in tertiary classrooms: Voices from Chinese EFL teachers. International Journal of Applied Linguistics, 34(4), 1265-1282. https://doi.org/10.1111/ijal.12560




DOI: https://doi.org/10.24815/siele.v12i1.41596

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Print ISSN: 2355-2794, Online ISSN: 2461-0275

This work is licensed under a Creative Commons Attribution 4.0 International License.


View Journal Stats