One of the biggest reasons multiliteracies is not implemented in the classroom is that it is difficult to assess. And if you cannot assess your students' learning, then how do you know they are learning? I'd like to propose a formative assessment, which is more like an activity. It also happens to be one of my favorite multiliteracies activities that I would like to implement fully. I have done bits and pieces of it, and that experience has helped me create this example.
Purpose and Context
For the past few years multiliteracies has been a secondary research interest of mine. I will define it in the construct section of this posting. One of the key articles I discovered made a point of concluding the implications of its investigations at the very end of the article, “And finally, there should be alignment between curriculum, pedagogy and assessment. Assessment criteria should include future-oriented new literacies if the latter are to become part of the normal classrooms.” The authors, Tan, Bopry and Guo (2010), are all members of the National Institute of Education in Singapore. In the introduction of their article they “provide [three] guidelines for the teaching and assessment of [English] throughout the country.” The first guideline includes electronic sources where students can find “a wide range of fiction and non-fiction texts.” However the third guideline is tied stronger to the construct of multiliteracies, stating that students should be able to “think through, interpret, and evaluate fiction and non-fiction texts from print and electronic sources to analyze how language is used to evoke responses and construct meaning; how information is presented; and how different modes of presentation create impact.”
In their case study, they focused on Alicia and her class of Year Two (14 year olds) English Language classroom students. Although Alicia incorporated the multiliteracies approach to her classroom, she felt that multiliteracies “was less important than conventional literacy” as reflected in “the language-dominant assessment that was still in place in the education system.” From these findings, the authors interpret that the (construct) validity of the assessment at the time was put into question because it underemphasized the development of multiliteracies.
The purpose of this paper is to develop an assessment that carries more validity in that it balances the constructs of multiliteracies and conventional literacy, which can be inferred from the Singapore Ministry of Education’s guidelines above.
The context of the assessment is the public education system of Singapore because the system is already in place to embrace this shift to a multiliteracies approach to language teaching. Another context is a reading and writing course in ESL program adjunct to a university in the United States for a few reasons. One is that I could not find research on the multiliteracies approach in ESL classrooms in the United States’ public school system. Another is that some university adjunct programs are open to modifying their courses or curriculum for the purposes of research. In particular, I have had experience with this curriculum modification at the University of Wisconsin at La Crosse.
This proposed assessment will act as a midterm formative assessment of learners’ multiliteracies and second language (L2) reading skills. It will also be a hybrid formative-summative assessment of learners’ L2 writing skills. The target proficiency level of the assessment will be intermediate-mid and intermediate-high (ACTFL) students taking into consideration the language skills mid-level ESL programs students at an American university and the age or grade in Singapore that best matches this proficiency level. I expect the range of students in a Singapore classroom to have a broader range of language skills since students are placed in classes according to their age or grade rather than their proficiency level.
Construct
I will first put forth Tan, Bopry, and Guo’s (2010) general description of multiliteracies and then I will provide a more specific description of the construct for this midterm assessment.
General Description
Multiliteracies encompasses traditional literacy (reading and writing) as well as “multiple literacies related to multimedia technology such as visual literacy and critical multimedia literacy.” The author’s cite Gee who states that literacy practices are “almost always fully integrated with, interwoven into and part of, the very texture of wider practices that involve talk, interaction, values and beliefs” (Gee, 2008: 45). Tan, Bopry, and Guo (2010) specifically looked at “the power relations inherent in the texts they commonly encounter in the modern world in which they live.”
Specific Description
According to Stuart A. Selber, author of Multiliteracies for a Digital Age (2004), there are three categories of multiliteracies: functional literacy, critical literacy, and rhetorical literacy. The focus of this formative midterm assessment is on critical literacy. On pages 102-103, Selber provides a table (see appendix) that illustrates the various power moves associated with technological regularization. Because this is a formative assessment, the midterm will act as a gauge to check how aware students are with any and all of the 11 power moves. More specifically the assessment is based on the power move of polarization, which is defined as “different versions of essentially the same artifact are created for no reason other than to reflect race, class, gender, or achievement categories.”
Wikipedia (http://www.wikipedia.org/) describes itself as “a free, web-based, collaborative, multilingual encyclopedia project supported by the non-profit Wikimedia Foundation.” In a sense, Wikipedia offers different versions of essentially the same artifact, an encyclopedia topic or article, created the purpose of reflecting language user communities. Because Wikipedia is collaborative anyone can author or co-author an article. Prior to the assessment students should become familiar with these questions when approaching any wiki:
Because Wikipedia offers different versions of the same article based on language, students may be able to find issues with reliability and accuracy more easily. For example,
As an interesting side note, Wikipedia offers an article on its own reliability at http://en.wikipedia.org/wiki/Reliability_of_Wikipedia. Looking at the sidebar as of February 15, 2013, there are only five other languages in which this article is written: French, Bahasa Indonesian, Portuguese, Russian, and Chinese. Why are there only only five since other Wikipedia articles have a lot more than that? This type of question may help students understand the relationship between Wikipedia and their first language.
Linguistic Constructs
Reading and writing are the two other constructs being assessed. More specifically, the assessment targets reading of homework instructions in L2, multimodal online reading in both L1 and L2, and paragraph composition in L2.
The construct of reading homework instructions refers to one’s ability to comprehend the instructions written in English to help students carry out the assignment. This is best observed in the completion of the assignment. The quality of the assignment depends on all three constructs.
The construct of multimodal online reading refers to “the process of receiving and interpreting information encoded in language form via the medium of [the internet]” (Urquhart & Weir, 1998:22).
The construct of paragraph composition in English refers to one’s ability to combine a series of sentences that are unified under a common idea or topic in a logical manner. This construct is elaborated upon in the writing rubric.
Instructional Objectives
The instructor or instructors who administer this assessment will observe the following objectives in their curriculum. With this assessment task, students of (your class) should be able to:
Test Specifications
The assessment comes in three stages: searching, reading and writing, and discussion.
In the searching stage, students are required to find a Wikipedia article that is published in both English and their first language. Once they have found this article, they need to seek approval from their instructor before moving on to the next stage. If they fail to find a page in during class time, the instructor has a choice of actions depending on the student’s reasons for failure:
In the reading and writing stage, students will read the Wikipedia article of their choice in both English and their first language with the goal of comparing the two. Students will write two paragraphs comparing and contrasting the two versions of the same Wikipedia article. In the third paragraph, students will write their ideas, thoughts, and opinions on why they believe the two versions are more similar or different. The instructor should allow students to express as much of their opinions or beliefs as they want as this paragraph will help indicate the depth of students’ multiliteracies skills.
In the discussion stage, students will work in groups to discuss their findings. This stage will be graded separately as classroom participation. The purpose of this stage is to give the assignment a sense of closure and to help the instructor perceive the class as a whole in terms of critical thinking development. The instructor is free to guide the students towards elements of critical thinking if he or she feels it necessary. The discussion should not impact the grade of the writing assignment as it is not part of the assessment’s construct or the objectives of the course.
Here is the link to the handouts that can be used for this assessment.
References
Bachman, L.F. & Palmer, A.S. (1996). Language Testing in Practice. Oxford, UK: Oxford University Press.
Hughes, A. (2003). Testing for Language Teachers, Second Edition. Cambridge, UK: Cambridge University Press.
Urquhart, S. & Weir, C. (1998). Reading in a second language: Process, product and practice. New York: Longman.
Selber, S.A. (2004). Multiliteracies for a Digital Age. Carbondale, IL: Southern Illinois University Press. Tan, L., Bopry, J. & Guo, L. (2010). “Portraits of New Literacies in Two Singapore Classrooms.” RELC Journal, 41(1), 5-17.
Purpose and Context
For the past few years multiliteracies has been a secondary research interest of mine. I will define it in the construct section of this posting. One of the key articles I discovered made a point of concluding the implications of its investigations at the very end of the article, “And finally, there should be alignment between curriculum, pedagogy and assessment. Assessment criteria should include future-oriented new literacies if the latter are to become part of the normal classrooms.” The authors, Tan, Bopry and Guo (2010), are all members of the National Institute of Education in Singapore. In the introduction of their article they “provide [three] guidelines for the teaching and assessment of [English] throughout the country.” The first guideline includes electronic sources where students can find “a wide range of fiction and non-fiction texts.” However the third guideline is tied stronger to the construct of multiliteracies, stating that students should be able to “think through, interpret, and evaluate fiction and non-fiction texts from print and electronic sources to analyze how language is used to evoke responses and construct meaning; how information is presented; and how different modes of presentation create impact.”
In their case study, they focused on Alicia and her class of Year Two (14 year olds) English Language classroom students. Although Alicia incorporated the multiliteracies approach to her classroom, she felt that multiliteracies “was less important than conventional literacy” as reflected in “the language-dominant assessment that was still in place in the education system.” From these findings, the authors interpret that the (construct) validity of the assessment at the time was put into question because it underemphasized the development of multiliteracies.
The purpose of this paper is to develop an assessment that carries more validity in that it balances the constructs of multiliteracies and conventional literacy, which can be inferred from the Singapore Ministry of Education’s guidelines above.
The context of the assessment is the public education system of Singapore because the system is already in place to embrace this shift to a multiliteracies approach to language teaching. Another context is a reading and writing course in ESL program adjunct to a university in the United States for a few reasons. One is that I could not find research on the multiliteracies approach in ESL classrooms in the United States’ public school system. Another is that some university adjunct programs are open to modifying their courses or curriculum for the purposes of research. In particular, I have had experience with this curriculum modification at the University of Wisconsin at La Crosse.
This proposed assessment will act as a midterm formative assessment of learners’ multiliteracies and second language (L2) reading skills. It will also be a hybrid formative-summative assessment of learners’ L2 writing skills. The target proficiency level of the assessment will be intermediate-mid and intermediate-high (ACTFL) students taking into consideration the language skills mid-level ESL programs students at an American university and the age or grade in Singapore that best matches this proficiency level. I expect the range of students in a Singapore classroom to have a broader range of language skills since students are placed in classes according to their age or grade rather than their proficiency level.
Construct
I will first put forth Tan, Bopry, and Guo’s (2010) general description of multiliteracies and then I will provide a more specific description of the construct for this midterm assessment.
General Description
Multiliteracies encompasses traditional literacy (reading and writing) as well as “multiple literacies related to multimedia technology such as visual literacy and critical multimedia literacy.” The author’s cite Gee who states that literacy practices are “almost always fully integrated with, interwoven into and part of, the very texture of wider practices that involve talk, interaction, values and beliefs” (Gee, 2008: 45). Tan, Bopry, and Guo (2010) specifically looked at “the power relations inherent in the texts they commonly encounter in the modern world in which they live.”
Specific Description
According to Stuart A. Selber, author of Multiliteracies for a Digital Age (2004), there are three categories of multiliteracies: functional literacy, critical literacy, and rhetorical literacy. The focus of this formative midterm assessment is on critical literacy. On pages 102-103, Selber provides a table (see appendix) that illustrates the various power moves associated with technological regularization. Because this is a formative assessment, the midterm will act as a gauge to check how aware students are with any and all of the 11 power moves. More specifically the assessment is based on the power move of polarization, which is defined as “different versions of essentially the same artifact are created for no reason other than to reflect race, class, gender, or achievement categories.”
Wikipedia (http://www.wikipedia.org/) describes itself as “a free, web-based, collaborative, multilingual encyclopedia project supported by the non-profit Wikimedia Foundation.” In a sense, Wikipedia offers different versions of essentially the same artifact, an encyclopedia topic or article, created the purpose of reflecting language user communities. Because Wikipedia is collaborative anyone can author or co-author an article. Prior to the assessment students should become familiar with these questions when approaching any wiki:
- Who is the author? Who are the authors?
- Do they cite any sources? How?
- How can you tell if the sources and the authors are reliable?
- What is their purpose?
- Why do you think this wiki page is currently the final version?
- How would you check if this wiki page is accurate?
- Who could help you determine the accuracy of the wiki page?
- How would you edit the wiki page to make it more reliable or more accurate?
Because Wikipedia offers different versions of the same article based on language, students may be able to find issues with reliability and accuracy more easily. For example,
- Why does one page have more content than the other?
- Why does one page have more sources than the other?
- Why does one page have more multimedia than the other?
- Why does one page have more internal hyperlinks than the other?
- Why does one page have more external hyperlinks than the other?
- For the English page, can you determine if it is written in American or British or another standard of English? How?
- For your L1 page, can you determine what standard or dialect the page is written in? How?
- How do the tones differ between the two Wiki pages?
- Is one page just a direct translation of the other? How can you tell?
- Which page was written first? How do you know?
- Can you find any language errors or mistakes on either page? What does this tell you about the authors?
- Which page do you find easier to read? Why?
- Which page do you find easier to trust as reliable? Why?
As an interesting side note, Wikipedia offers an article on its own reliability at http://en.wikipedia.org/wiki/Reliability_of_Wikipedia. Looking at the sidebar as of February 15, 2013, there are only five other languages in which this article is written: French, Bahasa Indonesian, Portuguese, Russian, and Chinese. Why are there only only five since other Wikipedia articles have a lot more than that? This type of question may help students understand the relationship between Wikipedia and their first language.
Linguistic Constructs
Reading and writing are the two other constructs being assessed. More specifically, the assessment targets reading of homework instructions in L2, multimodal online reading in both L1 and L2, and paragraph composition in L2.
The construct of reading homework instructions refers to one’s ability to comprehend the instructions written in English to help students carry out the assignment. This is best observed in the completion of the assignment. The quality of the assignment depends on all three constructs.
The construct of multimodal online reading refers to “the process of receiving and interpreting information encoded in language form via the medium of [the internet]” (Urquhart & Weir, 1998:22).
The construct of paragraph composition in English refers to one’s ability to combine a series of sentences that are unified under a common idea or topic in a logical manner. This construct is elaborated upon in the writing rubric.
Instructional Objectives
The instructor or instructors who administer this assessment will observe the following objectives in their curriculum. With this assessment task, students of (your class) should be able to:
- To develop their critical literacy of online content, specifically wikis
- To develop their print and online reading skills
- To develop their paragraph composition skills
- To demonstrate the three skills through their writing
Test Specifications
The assessment comes in three stages: searching, reading and writing, and discussion.
In the searching stage, students are required to find a Wikipedia article that is published in both English and their first language. Once they have found this article, they need to seek approval from their instructor before moving on to the next stage. If they fail to find a page in during class time, the instructor has a choice of actions depending on the student’s reasons for failure:
- Deduct points from class participation
- Select a Wikipedia article for the student
- Allow the student to work with a tutor to select a Wikipedia article
- Allow the student to select the Wikipedia article from home by seeking the instructors’ approval via email
- Any combination of the above
In the reading and writing stage, students will read the Wikipedia article of their choice in both English and their first language with the goal of comparing the two. Students will write two paragraphs comparing and contrasting the two versions of the same Wikipedia article. In the third paragraph, students will write their ideas, thoughts, and opinions on why they believe the two versions are more similar or different. The instructor should allow students to express as much of their opinions or beliefs as they want as this paragraph will help indicate the depth of students’ multiliteracies skills.
In the discussion stage, students will work in groups to discuss their findings. This stage will be graded separately as classroom participation. The purpose of this stage is to give the assignment a sense of closure and to help the instructor perceive the class as a whole in terms of critical thinking development. The instructor is free to guide the students towards elements of critical thinking if he or she feels it necessary. The discussion should not impact the grade of the writing assignment as it is not part of the assessment’s construct or the objectives of the course.
Here is the link to the handouts that can be used for this assessment.
References
Bachman, L.F. & Palmer, A.S. (1996). Language Testing in Practice. Oxford, UK: Oxford University Press.
Hughes, A. (2003). Testing for Language Teachers, Second Edition. Cambridge, UK: Cambridge University Press.
Urquhart, S. & Weir, C. (1998). Reading in a second language: Process, product and practice. New York: Longman.
Selber, S.A. (2004). Multiliteracies for a Digital Age. Carbondale, IL: Southern Illinois University Press. Tan, L., Bopry, J. & Guo, L. (2010). “Portraits of New Literacies in Two Singapore Classrooms.” RELC Journal, 41(1), 5-17.
Comments