Visible Learning & Computers

I've just finished reading this mega meta-analysis, and condensed/extracted Hattie's thoughts as they relate specifically to what he describes as 'computer assisted instruction'.

For those of you who may be oblivious of this phenomenal study, Hattie's premise is that he's examined over 800 studies, and his book is an analysis of the analysis—a meta-analysis. John Hattie developed a way of ranking various influences in different meta-analyses according to their effect sizes. In “Visible Learning” he ranked those influences which are related to learning outcomes from very positive effects to very negative effects on student achievement. This resulted in a total of about 138 influences from positive to pointless to positively poisonous. This site provides a fantastic way to digest the lot in one graphic.

For various reasons, he likes an effect size of > d = 0.40, so that gives you a handy frame of reference.

In terms of his findings relating to computer assisted instruction, it has to be said (and I think he concedes this point in his book) that his references are not exactly recent, it was published in 2009, and based on his descriptions, I suspect many of the studies were predicated on extremely dubious TELEs (Technology Enhanced Learning Environments).

I've included his positive comments about calculators (most were critical) as I think they are equally applicable, if not more so, to spreadsheets. Oh, how I love a good spreadsheet.

P 146 Calculators 

"Ellington found that the effects were much higher when calculators were involved in the teaching process; for example, when used for composition problem solving, the effects were d = 0.72: "When compared with students who did not use calculators, students in treatment groups were able to solve more problems and make better decisions with regard to selecting methods for generating solutions"(Beddington, 2003, p 169)."

...

"Hembree and Dessart (1986) found that the pedagogical use of calculators improved students basic skills both in completing exercises and problem solving. Across all grades (and particularly above grade 5, when calculators become more prevalent) and across all ability levels, students using calculators lead to greater effects in students' basic skills in operations and particularly in problem-solving. The effect on problem-solving seem to relate to improved computation and lower cognitive workload demands. They also found that there was a better attitude towards mathematics and an especially higher self-concept of mathematics for those using calculators compare to those not using calculators. The suggestion was that this enhancement in attitude was probably because the use of calculators helped relieve students traditional dislike of problems expressed in words (by reducing the cognitive load of having to compute as well as problem solve)."

...

P 147

"Using manipulative materials and calculators helps to reduce students cognitive load and allows them to devote their attention to problem solving."



P220

Computer assisted instruction

d = 0.37 

"An analysis of the meta analyses of computers in schools indicates that computers are used effectively (a) when there is a diversity of teaching strategies; (b) when there is a pre-training in the use of computers as a teaching and learning tools; (c) when there are multiple opportunities for learning (e.g. deliberative practice, increasing time on task); (d) when a student, not teacher, is in "control" of learning; (e) when peer learning is optimised; and (f) when feedback is optimised."

P 222 Supplement don't substitute

"Over the many meta-analyses, there was an advantage for computer work to be a supplement rather than a substitute or replacement for teaching instruction."

"One of the fascinating findings is that teachers are frequent use of computers—but more for their personal and administrative use; they find it more difficult to see how computers can be related to their particular conceptions of teaching (Cuban, 2001). When many of today's teachers with students in schools, computers were not as common, and many were taught in teachers' colleges by lecturers who were even more distanced from the use of computers in their teaching and learning. For too many teachers, teaching using computer resources is not part of their "grammar of schooling". Abrami et al (2006) noted that many teachers "are still on the threshold of understanding how to design courses to maximise the potential of technology" (p 32). Hence, there needs to be some pre-training in the use of computers and the teaching and learning tool but that used to be effective. [Pedagogical technologists essential, only I'd argue training teachers to design effectively would require pre, present, and post training!]

P 224 Skill drill

"Of particular interest is the effect of drill and practice – and despite the moans by many adults, students need much drill and practice. However, it does not need to be dull and boring, but can be, and indeed should be, engaging and informative. Drill is a euphemism for practice: repeated learning of the material until it is mastered – this is the key ingredient in mastery learning, many of the more effective methods outlined in this book, and of deliberative practice. It does not have to be deadly, and a key skill for many teachers is to make deliberative practice engaging and worthwhile. Luik (2007) classified 145 attributes of drills using computers into six categories: motivating the learner, learner control, presentation of information, characteristics of questions, characteristics of replying, and feedback. The key attributes that led to the highest effect included learner control, not losing sight of the learning goal, and the immediate announcement of correctness or otherwise of the answer to the drill.

Many computer games are basically invested with high levels of drill and practice and many students can be thrilled and motivated to engage in these often repetitive tasks to attain higher levels of skill and thus make more progress through the game. Computer games include much engaging drill and practice with increasing levels of challenge that usually is mastered by overlearning or undertaking high degrees of drill and practice. So often, the evidence has shown positive effects from using computers to engage in deliberative practice, particularly for those students struggling to first learn the concept. Meta-analyses have also frequently demonstrated that drill and practice routines by computer are more effective than traditional teaching (Burns and Bozeman, 1981). Perhaps teachers should pause and wonder why their traditional teaching is less effective than many computer drill and practice programs."

P 225 Word processing & writing

"A good example of the student being in control of his or her learning relates to the use of word processors. When using these packages, students tend to write much more than when asked to write on paper, and the quality of writing is enhanced, especially for the weak writers (Bangert–Drowns, 1993). This "more" is not more of low quality, as quality of writing and length was highly positively related. Students are more likely to make revisions, write more, and make fewer errors (Goldberg, Russell, & Cook, 2003; Schramm, 1991). Torgeson and Elbourne (2002) completed a meta analysis of studies conducted between 1992 and 2002 on computers and student writing, and found that, on average, students who use computers when learning to write were not only more engaged and motivated in the writing but produced work that was of greater length and higher quality than students learning to write on paper (d = equals 0.40)."

P 236 Value ... added

"The use of resources, such as adjunct aids and computers, can add value to learning. They add a diversity of teaching strategies, provide alternative opportunities to practice and learn, and increased the nature and amount of feedback to the learners and teachers. They do, however, require learning how to optimise their uses. It is also clear that, yet again, it is the differences in the teachers that make the difference in student learning."

Hattie J (2013). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.