Minutes from week 2

Tuesday 8/28/07

Lourdes: Has everyone joined the blog and posted a profile?

Class is divided into 4 groups (per the four divisions in the article) to discuss Hyland & Hyland (Feedback on second language students' writing).

Groups will look at:

1. effectiveness


2. context (recent realization; what does this mean for different research traditions?; how much 'context' can we account for in any particular study?)


3. footnote chasing (is there any study that jumps out as recommendable for this class?)

Group presentations begin at 11:15; Lourdes asks that we only look at effectiveness for now (will address context and any suggested readings on Thursday).

Group 1: Teacher written feedback

• Direct v. Indirect


• Long-term effect v. short-term


• Cultural factors


• Metalinguistic / metacognitive awareness


• Lourdes: how about in terms of proof of effectiveness? Can students explain what feedback means? Are they able to understand the teacher's comments and revise in a way that is good?


• This is how we think about how effectiveness is defined. What answer we give will determine how we go about our studies.


• Any dispute over findings goes back to how people define effectiveness in the first place...




• Proficiency


• Assessment (need more long-term studies)

Lourdes: what we should be discussing is, 'what evidence is gathered to demonstrate effectiveness?' Effectiveness is defined as the ability to revise.
In the short term, the student merely does something with feedback; when talking about long-term, the student performs subsequent writing that has nothing to do with the original work.

Group 2: Content & Form

• research not conclusive


• hard to claim direct causal relationship between teacher's feedback and students' revisions


• varied context and social factors (non-contextual and non-social)


• teacher's stance about content and form (content - form distinction?)


• types of feedback (Ferris 1997) ex: praise, criticism, suggestion, etc.

Lourdes: low-level errors at local level, or a problem that goes beyond form? When you have to re-write whole ideas, that's content and not form.

Could also do a study on this: it would be easy to look at real feedback given by real teachers and classify them as form, content, or neither.

Go back to the article: In how many different ways did researchers find effectiveness?

Group 3: Peer / Self feedback

• Indicator: product/skill
  
  • accuracy (problem: how much? when?)
  
  
  • fluency (problem: measurement)
  
  
  • organizational / linguistic complexity
  
  


• Disposition
  
  • learner autonomy (problem: evidence)
  
  
  • low affective filter
  
  
  • value & belief
  
  
  • awareness of audience
  
  


• Process
  
  • interaction
  
  
  • negotiation
  
  


• Awareness of outcomes

Lourdes: in this area of peer feedback, it's easier to see how effectiveness has been defined.

Ex: 'after peer feedback, students became better readers and editors.' So, this is another way to define 'effectiveness'. But, it's different from the typical measure: 'students become more accurate...'

Some types of error correction may be effective, but maybe things are already there that are optimal and we don't want error correction to mess with that.

Group 4: computer-mediated feedback

1. computer-mediated communication


• asynchronous


• dialogic role, scaffolding


• builds metacognitive awareness through archiving


• syncrhonous


• students more focused on feedback


• CMC v. face-to-face


• corpora-based


• proficiency: corpora helpful for beginners and to develop independent writers

Lourdes: Local and global changes are examples of defining effectiveness.

What is the quality of comments? Too local? Too global? Too confusing?

No study look at everything, buut some studies are vague in defining what 'effectiveness' is.

Keep in mind how effectiveness and context are defined in articles we're reading for Thursday.

Thursday 8/29/07

Lourdes: keep article postings down to earth, consider your audience. Can you summarize in plain language?

Don't forget to mention enjoyability: was the study so dense that you couldn't get through it?

(Lourdes proceeds to briefly touch on each article reviewed on the blog.)

Recasts against the world! Since Long, everyone seems to be taking up recasts and comparing them to something else.

Good idea for a study: seeing how concordancing works with student self-editing.

If you want to do a design/proposal for this class, at least pilot part of it.

McDonough & Mackey (2006) — HOT article: correction may have an effect that goes to the type, not just the token.

Ellis: type frequency may be more important than token frequency; this idea has a lot of support from the psycholinguistics literature in L1.

Lourdes tends to agree with Ellis: people who strongly disagree tend to be UG'ers, which Lourdes is not.

However, Lourdes doesn't care much for the HUGE role that Ellis ascribes to L1.

Brief mention of where Skehan and Robinson differ (with Robinson leaning towards emergentism).

This semester we need to look at socio-cultural theory, because studies that go this way are different.

In SLA textbooks, socio-cultural theory (Lantolf and Vygotsky)is now inclued as legitimate SLA.

*Be aware of the three [didn't catch what "three" Lourdes was talking about — David], and be aware of which one a study aligns itself with because it's important (to know where a study an author situates him/herself).

Class will meet in the faculty computer lab next Thursday as Lourdes will be having lunch with the new chancellor. We (students) will discuss how to make a wiki and what we want to do with it. Harvest ideas from our blog and add them to the wiki — even only as themes at this point. For ex: "We need a page on recasts..."

Remember that our bibliographies are on a topic. Grouping: recasts w/ children, recasts and... etc.

(class reorganizes into same groups as Tuesday to look at context with respect to Hyland and Hyland.)

Group presentations on context
Definition: a combination of factors releated to the institution and writing program as well as those that teachers and students bring to the interaction.

Some elements/factors of context that were mentioned include:

• socio-political issues (teacher-student relationship)


• available resources and class size


• institutional attitudes towards L2 writers


• exams, program philosophy


• student factors
  
  • attitudes, needs, preferences, cultural differences, proficiency, developmental stage, level of engagement


• teacher factors
  
  • approach, variation, commenting strategies, language and style in feedback, beliefs, profile


• indicators of context
  
  • institutional: location, face-to-face, role for teacher, policy
  
  
  • learner: level
  
  
  • affective context: motivation


• computer conferencing
  
  • student-centeredness
  
  
  • teacher acts a a facilitator


• automated


• corpus: websites that offer more specific words and contexts


• classroom realities (students' backgrounds, needs, and preferences


• social-cultural perspective on learning


• ownership of writing


• culture: homogeneity, heterogeneity


• power relationship: group dynamics


• society: what standards dictate good teaching?

Lourdes: indicators of context are easier to identify in the article than effectiveness.

Institutional, ownership, agency: if we talk about these factors, where is the social context?

Motivation, individual differences, style: what about these?

Some studies just define context as, "this is an EFL class," or, "the subjects are adults".

We'll be aware of how context is addressed in the studies we look at.

(Last 10 minutes of class used to discuss potential project ideas.)

Lourdes: study idea: what are teachers basing their decisions on? It's also important to look at times when correction doesn't happen, and why...

Sang-Ki: what is the difference between aquisition studies and feedback studies?

Yuki: error correction is acquisition.

Lourdes: quasi-synonomous terms:

• error correction


• (I didn't catch this one)


• negative evidence


• negative feedback

Within L2 studies of writing, there is a small group that model themselves after the oral mode; others try other things.

Next class: read second article and post your review/evaluation to the blog.
Thursday: go to lab and work on wiki.

Notes on Storch (2007) and pair work

Storch, N. (2007). Investigating the merits of pair work on a text editing task in ESL classes. Language Teaching Research, 11, 143–159.

While the merits of group-work in second language classrooms has already been established, this study aims to address the gap in research comparing small group with individual work. The three research questions of the study are as follows:

1) Do learners working in pairs complete an editing task more accurately than learners who complete the task individually?
2) What is the nature of the learners' talk when working in pairs on an editing task?
3) Do learners reach grammatically correct decisions when deliberating over grammar?

For the study, four intact, upper-intermediate ESL classes at an Australian university were employed. Class A completed an editing task in pairs while students in class B completed the task individually. To investigate whether students prefer working on grammar-based tasks in pairs, students in classes C and D were given the option to work individually or with a partner. The task itself called for editing an authentic piece of writing (done by an ESL student from a previous session) that was manipulated to contain 19 error items typical of students at this level.

The results show a slight increase in accuracy for students working in pairs (who, on average, took longer to complete the task and made more amendments to the text), but the difference is not statistically significant. While the difference in mean scores with regard to those students who had the option to work alone or with a partner is greater — accounted for, perhaps, by higher motivation — it, too, is not statistically significant. This may not seem like a strong sell to encourage more group work in the second language classroom, but the study also finds that weaker students in particular may be at a disadvantage when working alone, and that pair work seems to provide learners with the most opportunities to engage interactively with the language and initiate self-repair.

Compared to other studies I've read, this one was relatively straightforward and rather pleasant to read, actually. It was of particular interest as I promote a lot of group work in my own classes, and it was nice to be reminded of the real benefits to learners that derive from this type of activity — even if the study itself didn't turn up significant evidence in favor or group work on the whole. Again, this is only one study.

As for it's usefulness for this class, I would have to say that there isn't a great deal pertaining to error correction per se, and therefore give it a gentle 'thumbs down'. If you want more evidence of why pair work is effective for certain learners dealing with a certain task type, I'd say give it a look over; otherwise, I think you'll be satisfied with the summary of research results stated above.

Evaluation of Ferris (2006) by Sachiko

Ferris, D. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp.81-104). New York: Cambridge University Press.


Debate about the value of providing corrective feedback on L2 writing has been prominent in recent years as a result of Truscott (1996) who claimed that it is both ineffective and harmful and should therefore be abandoned. Dana Ferris is probably one of the most well-known researchers who have rigorously refuted Truscott by providing additional evidence on positive effects of error feedback in L2 writing. This book chapter is her latest work that summarizes research findings on: (1) the effects of error feedback on improvement in the short run and in the long run; (2) the impact of teacher marking strategies on students’ revisions; and (3) the relationship between error types and error treatment. Data for the study was collected in six sections of an ESL composition class where students (N=92) completed four three-draft persuasive essays over the semester. In order to investigate students’ long-term development, all three drafts of the first and fourth essays with teacher feedback were analyzed. The analysis of students’ responses from one draft to the next of a particular paper revealed that the student writers were able to successfully edit errors marked by teachers (80% of all the errors were successfully corrected). However, the longitudinal analysis showed that students’ short-term ability to edit certain types of errors (article errors, lexical errors, and sentence errors) did not always translate to long-term improvement (only “verb-form” demonstrated the progress over the semester). The analysis of teachers’ marking patterns illustrated that students utilized direct feedback more effectively than indirect feedback, since it involved mere transcribing or copying the teacher’s suggestion into the next draft of their papers. However, less explicit forms of feedback also led to accurate revisions most of the time. The analysis of student revision outcomes by error types elucidated that teachers responded to different types of errors in different ways. Specifically, “treatable” errors (errors that could occur in a rule-governed way: verb tense or forms, subject-verb agreement, run-ons, fragments, etc) were more often marked indirectly, while “untreatable” errors (no set of rules students can consult to avoid or fix errors: word choice, idioms, and the sentence structure) were overwhelmingly marked directly.

Ferris’s work fills an important niche in attempting to examine the longitudinal effects of teacher feedback and the effect of different types of feedback (direct/indirect) on different error types (treatable/untreatable). She provided evidence that students’ immediate revisions did not always lead to their longer term development. This evidence implies that students’ ability to utilize teacher feedback to successfully edit one draft for the next of a particular paper is not equivalent to their successful acquisition of the linguistic construct addressed by the feedback, and thus gives us a significant implication regarding the relationship between correction and learning. Ferris’s attempt to classify error types into treatable and untreatable ones is also invaluable. She found that treatable errors were more likely to be marked indirectly than untreatable errors which were more often marked directly. Of particular importance is that the indirect feedback that students received on verb errors may have helped them over time because verb form demonstrated the long-term improvement in accuracy. This means that indirect feedback can encourage learner reflection and therefore trigger the guided learning and problem-solving processes (Lalande, 1982). The evidence suggests possible feedback approach that could be employed by L2 writing teachers (i.e. varying feedback approaches for treatable and untreatable error types, and providing primarily indirect feedback for treatable errors).

Ferris’s study thus provides valuable insights into the effects of error feedback on L2 writers’ texts. However, as I read Ferris’s analysis and discussion of the data, I found myself unsatisfactory in a way I often am when I read feedback research pieces that are non-contextual. I hence doubt whether this article would be useful for our discussion in class. I think that there are at least two contextual factors that should have been addressed more clearly by Ferris: individual differences and classroom realities. First, by highlighting the fact that 80 percent of the total errors (4,590 out of 5,705 errors) were successfully corrected, Ferris claims a strong relationship between teachers’ feedback and successful student revisions. However, 80 percent of the cases does not say anything about how individual students responded to teacher feedback, given that 92 students participated in the study. In fact, the individual differences became apparent in the result of the quantitative analysis, indicating that the standard deviation of the total errors was extremely large. The same is true of a teacher variable. Ferris investigated three teachers’ marking strategies, and identified that their direct feedback accounted for 45 percent of the total corrections, while their indirect feedback with code accounted for 41 percent of the total. However, this triggers questions about individual teacher differences and consistency in treatment. There might have variation among the three teachers in their proportions of direct and indirect feedback. Second, classroom realities, that is, the relationship between class instruction and teacher feedback is unclear. Given that the study was conducted in an ESL composition classroom and the task was a persuasive essay, it is legitimate to speculate that a focus of the instruction was on writing as well as grammar, but Ferris’s primary concern in this study was put on grammatical accuracy alone. Such research design is skeptical in that the effect of teacher feedback is examined without regard to what was really taught in class. The issue of context that has been overlooked in research into feedback has been pointed out and discussed in class, and thus I do not think this article would serve as a topic for more productive and constructive discussion in class. My evaluation of this article is therefore “not recommended.”


References
Lalande, J. F. II (1982). Reducing composition errors: An experiment. The Modern Language Journal, 66, 140-149.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46, 327-369.

Evaluation of Sheen (2007) by Sang-Ki (2nd evaluation assignment)

Sheen, Y. (2007). The effect of focused written corrective feedback and language aptitude on ESL learners’ acquisition of articles. TESOL Quarterly, 41, 255-283.


What are the purposes of giving feedback on learner errors? This is surely an important question to be addressed when evaluating studies on error feedback. I understand that the purposes can vary substantially depending on the language mode, that is, whether the feedback is given orally or given in a written format. This is because written feedback could demand less cognitive load for learner memory than oral feedback, since feedback given in a written format will be available for a longer period of time than the other type of feedback. Thus, by giving written feedback, teachers may expect their students to benefit more not only in terms of the depth of its effectiveness, but they also expect that the benefit would reach wider contexts of language use.

Most of the oral feedback studies to date have focused on the correction of form-related errors, with an (exclusive) aim of improving the accuracy of a focused linguistic element (e.g., Doughty & Varela, 1998; Iwashita, 2003; Long, Inagaki, & Ortega, 1998; Mackey & Philp, 1998). Conversely, written feedback studies have usually had varied foci other than the improvement of accuracy in student writings; they have also intended to improve writing content, organization, and the overall quality and fluency of writing products. This difference in purposes may account for why “in contrast to the SLA research that in general has shown that oral CF [corrective feedback] is effective, L2 writing researchers have not been able to convincingly demonstrate that written CF leads to improvement in grammatical accuracy in new pieces of writing” (p. 257).

The researcher’s main question in the current study is whether written feedback, if given in a focused manner, improves the accuracy in writing. In order to employ the focused type of feedback by design, teacher written feedback was given only on the article-related errors (i.e., inappropriate uses of a and the) as target to be acquired by the 91 adult intermediate-level research participants. The two factors presumed by Sheen to affect the efficiency of written feedback were (1) two types of written feedback: direct-only correction (provisions of corrected forms) versus direct metalinguistic correction (provisions of corrected forms and metalinguistic comments) and (2) individual differences in language-analytic ability. The patterns of major findings turned out to be predictable. The focused, metalinguistic feedback was found more beneficial for accuracy improvement, and the positive effects appeared to last over a month. In addition, significant benefits from the higher levels of language-analytic ability were found, and the benefits were most prominent within the direct metalinguistic correction condition. The helpful role of metalinguistic information in grammar learning would be in line with what has generally been found from the accumulated oral feedback studies (e.g., Bitchener, Young, & Cameron, 2005; Carroll & Swain, 1993; Ellis, Loewen, & Erlam, 2006).

I enjoyed reading this article, and I think this article is useful for a couple of reasons at least. First, this study is the first to my knowledge which looks into the effectiveness of written feedback on accuracy improvement in student writings, with methodologically sound designs (e.g., true control group design with a focused target rule, and the like). Second, this study rightfully switches our attention to the mediating roles of individual differences in grammar learning through written feedback (i.e., individual differences in language-analytic ability). An improvement of the study design could be made, however, if the individual difference factor could serve as an independent variable. (The current study examines only the correlations between language-analytic scores and acquisition scores.) That is, having different independent groups with differential language-analytic abilities and giving them various types of written feedback could shed more light on the potential causal role of the individual difference variable across different feedback types.

Again, if I have rating scales, I would vote for the “recommended” option.

(Personally, I prefer this article to my previous choice (Sheen, 2006) as our course reading.)

Evaluation of Weissberg (2006): Watanabe, Y.

Weissberg, R. (2006). Scaffolded feedback: Tutorial conversations with advanced L2 writers. In K. Hyland., & F. Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp. 246-265). New York: Cambridge University.

Weissberg (2006) investigated the underlying mechanism of oral tutor feedback to L2 students’ writing in one-on-one writing center sessions. Borrowing socio-cultural framework, he examined linguistic and discourse features of scaffolding specific to writing tutorials. Two NNS graduate students each participated in four one-on-one writing conference sessions. The tutorial conversations were audio recorded and analyzed, utilizing inductive analysis (IA). IA is “an iterative process consisting of repeated cycles of data analysis and hypothesis revision” (Weissberg, 2006, p. 252). In other words, themes and discourse categories, and their underlying structures are identified and revised through repetitive reading of the transcripts.

Weissberg found three levels of categories in writing tutorial discourse: a surface discourse level (i.e., overt conversation moves which structure scaffolding: confirmation checks, information questions, etc.), semantic content level (i.e., topical episodes: grammar, citations, writing process, etc.), and pragmatic level (i.e., the goals of the tutoring session: identification of problem areas, evaluation, reflection, etc.). It was concluded that the most salient components of scaffolding were (a) the tutor’s “lexical, ideational, and affective” (p. 259) attachment to tutee’s discourse, and (b) the utilization of conversational linkages (e.g., acknowledgement) to create instructional point.

Weissberg’s study was unique in that he was concerned on the role of oral feedback on global issues of writing, such as “planning, organizing, [and] revising” (p. 252). The study focus was on the nature of interaction during tutoring, and it is unfortunate that it did not further investigate how scaffolded feedback led to advancement in learners’ writing. However, Weissberg’s list of writing tutorial discourse categories (see Table 13.1, p. 254) will be useful for those who plan to investigate such question.

Although I did enjoy the article because I am particularly interested in oral feedback on writing, I do not recommend reading this article in class. It would be a great article for our classmates to take a look as a reference, since the focus of the class is on the effect of feedback.

Evaluation of McDonough & Mackey (2006) by Sorin

McDonough, K. & Mackey, A. (2006). Responses to recasts: repetitions, primed production, and linguistic development. Language Learning, 56, 693-720.

This study examined one of the most oft-mentioned issues of recasts, whether recasts facilitate L2 development, by investigating the relationships among recasts, different responses to recasts, and L2 development. The researchers distinguished simple repetitions of a recast from primed production and compared their effects on the development of English question. Primed production was defined as a learner’s new utterance using the syntactic structure provided in the recast either immediately or a few turns later (in this paper, within six turns). In a pre- and posttest design, 58 Thai EFL university students performed a series of communicative tasks with NS interlocutors (during three treatment sessions) and completed four tests (one pretest and three posttests) over a 9-week period. L2 development was operationalized in this study as the production of higher level questions on all three posttests according to Pienemann’s developmental sequence. The results indicated that recasts and primed production were predictive of ESL question development whereas mere repetition of recasts was not significantly correlated with ESL question development.

I found this article worth reading since it was among the few studies investigating the relationships among recasts, responses, and L2 development, and showed relatively long-term effects (9-week period) of recasts. It also provided evidence that merely repeating recasts was not necessarily associated with development, and it was more beneficial to produce the form in one’s own way. The findings that learners without immediate responses to recasts were able to successfully formulate advanced questions later on supported delayed effects of recasts on L2 development, thus indicating that the absence of learner responses to recasts, which was often criticized, does not necessarily limit the effectiveness of recasts. Another interesting result of the study was that most of the learners in the control group, which did not receive any feedback, did not produce primed production involving advanced-level questions, even though their NS interlocutor’s use of the targeted form created opportunities for it, which requires further investigation.

This study would be useful to those who are interested in the effectiveness of recasts including its impact on language development, long-term (and/or delayed) effects of recasts, different types of responses to recasts, and their effects on L2 acquisition. The brief summaries of previous studies, addressing various issues regarding the effectiveness of recasts, would be helpful resources for those conducing research on these issues. Thus, I would like to recommend this article to be included in the reading packet.

Evaluation of Gaskell & Cobb (2004) by Myong Hee

Gaskell, D., & Cobb, T. (2004). Can learners use concordance feedback for writing errors? System, 32, 301-319.

This article demonstrates whether L2 learners can use concordance feedback to correct their sentence-level writing errors. Twenty adult low intermediate Chinese EFL learners in Canada participated in the present study. Each student provided a 200-word writing sample and 10 prominent errors among the samples were determined for concordance feedback. Over a 15 week semester, students handed in 10 assignments. For the teacher’s feedback, a maximum of 5 concordanced errors (hyperlinks) were given for each assignment. The findings show that (1) over 50% of the participants responded that their ability to use grammar points had improved; (2) the vast majority reported that they used concordances to correct their errors; (3) and many of them, especially seven became persistent users of the online concordance.

I felt this article very useful. It shows another way to provide feedback for students’ writings. I have learned that online concordance can be a useful tool for not only teaching vocabulary but also teaching grammatical points. Once you train your students, they can be independent users of concordance who can correct their sentence-level grammatical errors. The ultimate goal of teaching L2 writing is to make a learner an independent proficient writer. Besides, given the fact that most English courses in Korea (probably other Asian countries) are relatively big, learners’ autonomy in L2 writing is highly desirable. In this respect, this article provides good information on learner autonomy in sentence-level error correction.

I really enjoyed reading this study. It is rather a short article and easy to read. The findings interested me a lot as a teacher because they provide valuable information for future writing teachers and researchers.

I’d like to recommend this article to our 750 reading packet. This study suggests another perspective on written feedback. Most studies in written feedback are concerned about teachers’ giving comments on students’ writings. However, this article suggests another possibility that students correct their own sentence-level errors through being a concordance user.