Posts Tagged assessment
It’s that time of year. Teachers of Year 12 around Australia are scrambling to varying degrees to prepare students for final assessments and exams, which inevitably involves a whole lotta marking.
Of course, all teachers have to grade student work. And they are engaged in doing this all year. But nothing beats the pedal-to-the-metal feeling of marking Year 12 practice tasks in a last ditch effort to refine their examination responses.
In particular, nothing beats the hellish pressure that exists in states like NSW and Victoria where the HSC and VCE exams respectively loom over teachers and students alike. And out of all these teachers and students, I argue that subjects that are writing-intensive (e.g. English and History) have it the toughest; if you have a class of 25 for Year 12 and it’s coming up to an assessment, teachers in these subjects are spending their nights and weekends correcting pages and pages and pages of long form expositions.
Which can leave your eyes (and soul) feeling kinda like this:
I was prompted to write this blog post after watching my friends Justin and Alex tweet about their marking yesterday:
I’ve taught for the HSC three times and this slavish marking routine is the only part I do not miss…having said that, the jolly task I have now of marking as a university lecturer has involved marking binges that certainly rival the pain of HSC workload.
The question is – what can we do about it?
Is there anything we can do about it?
Some ideas that I threw out into the twittersphere yesterday seem promising, but without a class to try them on I’m at a loss, not sure if they would work. The ideas I bounced around with Justin and Alex were:
- Focussing on writing just the introduction, or a body paragraph. This would make the task smaller and more focussed for students, and more manageable to mark 25-30 of them.
- Setting a paragraph writing challenge. To address Justin’s problem of the student that only writes about ‘tone’, each week set a different language feature/form for students to write a paragraph on. By the end of the term they will have a bank of paragraphs on different elements.
- Gamify the writing process. This could be done by putting students in groups, getting every student to write a paragraph (or essay), then each group submits it’s best one (as judged by the students in the group) for marking. This means you only have to mark one essay/paragraph per group, not per student. Keep a chart of which group wins each week and award them a prize at the end of the unit. Change the groups around for each new unit.
- Peer assessment. This can only be used in a limited way, as students don’t have the capacity to grade work to a Year 12 standard. However you could use the ‘medals (feedback) and missions (feedforward)’ framework that Bianca draws on to give students a direction. I think the main benefit is that they read each other’s work and discuss their strengths, not that they actually give each other a ‘grade’.
- Find an authentic audience. Partnering up with another teacher/class would provide an avenue for students to share their work with another class on a platform such as a wiki. This would give students someone to perform for besides their own teacher, which could prove motivating. The teachers could also arrange to do a marking-swap, and grade each other’s student essays…this may get you writing less comments, marking more objectively (?) and just plain old provide a change of pace as you get to read a different set of handwriting!
I really hope these ideas are useful to someone out there.
If you have any other good ideas for getting feedback to students without going through so much of the eye-bleedingly painful million-essay marking process, I would LOVE to hear them!
Thanks to Justin and Alex for inspiring this post and helping me brainstorm ideas :)
Images: Cropped screen still from True Blood, Season 5; Screen shot of conversation on Twitter.com
Postscript: If you liked this post, you may also like the post Matt Esterman wrote today, ‘The home stretch for Year 12′. Looks like we all have Year 12 on the brain this weekend!
Last year I was interviewed by Melissa Wilson, a Journalism student from the University of Newcastle, about my views on the Higher School Certificate (HSC).
Melissa was in contact again recently, and this prompted me to ask if I could reproduce the interview here on my blog. She kindly obliged, and so here it is!
I was careful to read back over my answers, to make sure I still felt the same way about these issues. I do. I wish I could say that things were vastly different up here in Queensland. They’re not. When a Queenslander tells you there is no external exam for Year 12 in their state, they’re misleading you at best. Here is a bit of information from the QSA website about the Queensland Core Skills (QCS) test:
Preparing for the test
The Common Curriculum Elements are generic skills that students work with across their subjects; therefore the real preparation for the test goes on all the time and in every subject. The QSA also makes available a variety of test preparation resources, including Retrospectives and past testpapers (see QCS Test publications and Retrospectives and MC response sheets). Most schools provide some focused preparation for the test.
Hmm, sounds familiar.
But, I digress…
Here are my responses to the interview by Melissa Wilson:
Interview answers for Melissa Wilson (University of Newcastle, 2011)
Interviewee: Kelli McGraw (Lecturer, QUT)
- How do you feel about how the HSC is structured in 2011?
When I think about the HSC structure in 2011, the main things that leap to mind are the fact that studying English is mandatory, that half of the student assessment is based on a timetabled external examinations, and that a no more than of 30% of your school assessment is supposed to be ‘exam-type’. I think it’s really important for English to remain compulsory right up to the end of school, but I’d like to see more room for students to choose electives within the course, not just different levels i.e. Standard or Advanced English. At the moment I think the HSC is still structured in a way that is too rigid for students to feel like they have a lot of choice over their learning.
- Many people say that the HSC is focused on teaching students a whole lot of information that isn’t exactly relevant to them later in life – but instead they just regurgitate it in an exam and then discard it – how do you feel about that statement?
Personally, I can think of countless things that I learned in my HSC year (1998). In those days the emphasis on exams was just as great, but I am often surprised by the things I remember from senior high school and have found a lot of what I learned to be very relevant in life. Having to finish ‘major works’ for Visual Art and Drama also taught me valuable lessons about project management and self-directed learning, which I didn’t get from participating in written exams, so in that sense I guess I was lucky to be an ‘art-sy’ student.
I think the real problem with exams is not that students have to cram ‘irrelevant’ information – I think that all learning can be made relevant, depending on what you choose to do in life. The problem I find is that the examination system has too much of an effect on what happens inside the classroom. The constant pressure to cover content is a strain on students and teachers, and even though school-based assessment is supposed to involve deep learning and reflection, many schools I know of set far more than 30% of their assessments in an exam style in order to condition students in preparation for the external exam. So I think there is a ‘hidden curriculum’ in the HSC, which can dilute learning experiences based on the official subject syllabuses.
- And from this, what would you personally change about the HSC?
I think that the only assessment that students should have to do under exam conditions is the Trial. If more student work was assessed through project work, or using collaborative group tasks, or using portfolios, I think that students would feel more connected to the learning, and be motivated to achieve. Even though the HSC now uses criteria-based assessment, students are acutely aware that the HSC places them in competition with one another as in-class assessment ranks still play a role in determining a student’s final subject results, and the year culminates for most students in receiving a ranked national placement through the UAI. With only about 30% of students moving from school to university after Year 12, it seems like we compromise a lot of educational values for the sake of a privileged minority.
- How do you feel about the pressure and emotional stress that students endure throughout their HSC?
When I think about the stories that students have told me over the years – about how they feel inadequate, or like a failure in the face of HSC assessment tasks – it makes me really upset. I have seen a lot of students in Year 12 lose a lot of weight, with girls in particular showing signs of early and advanced eating disorders. Senior school is also a time when increased numbers of students pick up casual and part-time employment, in many cases out of a necessity to contribute to household finances. I think the HSC creates an environment where students are given too many adult responsibilities without being given the corresponding rights.
While schools play a vital role in developing students’ resilience and capacity for work, the emotional stress endured during the HSC year is too much, in my opinion. I read a study awhile back where Year 11 and 12 students reported symptoms of anxiety, depression and stress that fell outside the ‘normal’ range. We know that when this happens, students stop focussing on ‘mastering’ the material, instead focussing on performance; they stop believing in themselves, stop seeing the learning as a worthwhile goal, and switch to performance-oriented goals. Some of my own school friends took years to recover from the emotional damage of the HSC year, especially those whose final results didn’t meet expectations.
Hi folks – it’s been a bit quiet here on the blog, I know.
All I can say is … omg MARKING!
I have always had an interest in assessment, but this semester has made really clear to me how dire the situation is with our current practices.
I don’t want to ‘buy out’ my marking (i.e.pay someone else to do it for me) but I feel like I am wasting so much of my time at the grindstone, like a machine, writing the same lines over and over in delightful pink pen in the margins of my students’ work.
“Check the APA style guide for rules about how to format this”
“Formal essays require shorter paragraphs than this”
“Avoid rhetorical questions – make strong statements instead”
“Use your introduction to tell me what your main points will actually be, not to explain the structure of your work”
“Don’t use a quote as a sentence on it’s own – introduce it i.e. ‘Sawyer (year) explains that…’”
“You have not included reference to any unit readings in this rationale”
I worry about RSI. I worry about carpal tunnel! Marking more tasks electronically next semester will hopefully fix the hand ache, but what about the mind ache??
I’m not alone – every teacher reading this knows what I mean.
What are we going to do about it?
Teaching at university can be tricky, mostly due to the emphasis on summative assessment.
Since starting this position in 2010 I have been attempting to infuse the unit I coordinate with greater amounts of project-based learning. However, in a context where students have little time or incentive to engage with classwork that isn’t formally assessed, it has been hard to reward things like student project work.
After three semesters of teaching English Curriculum Studies 1 I decided that a radically new assignment was in order.
Students used to do:
- Assignment 1 – Personal teaching philosophy statement and resource analysis
- Assignment 2 – Report on video lessons and learner needs observed
- Assignment 3 – Junior secondary English lesson plans
All of these assessment pieces were completed individually – no collaboration was required and no public audience was utilised.
From this semester onward, students now do:
- Assignment 1 – Personal teaching philosophy statement and resource analysis (same as before)
- Assignment 2 – Junior secondary English lesson plans (now completed in small groups of 2 or 3)
- Assignment 3 – A range of CHALLENGE TASKS published in a portfolio <– SCHMICK NEW TASK!
The New Task:
Many of the key ideas about inquiry-based and cooperative learning that I am working with can be found in a book extract provided by Edutopia: Teaching for Meaningful Learning by Brigid Barron & Linda Darling-Hammond.
Here is a brief extract – some words about project-based learning:
“Project-based learning involves completing complex tasks that typically result in a realistic product, event, or presentation to an audience. Thomas (2000) identifies five key components of effective project-based learning. It is: central to the curriculum, organized around driving questions that lead students to encounter central concepts or principles, focused on a constructive investigation that involves inquiry and knowledge building, student-driven (students are responsible for designing and managing their work), and authentic, focusing on problems that occur in the real world and that people care about.” (Barron & Darling-Hammond, 2008, p. 3; my emphasis)
What I’ve done in my new task is to create a poetry ‘project’ as one of 10 ‘challenges’ that students need to complete.
After trialling a poetry project last semester, I know that students see value in, and engage with this kind of learning. But, at the end of the day, students felt let down because the work they put into their projects didn’t ‘count’ towards their final grade.
Once I started messing around with a new assignment that gave them credit for their project work, it was too hard not to design a whole suite of ‘challenges’ that they could choose to take up! So, that’s what I’ve done – students decide what grade they want to get, and complete the number of challenges needed to obtain it.
‘Challenge-based learning‘ as a term has not gained as much traction as ‘project-based learning’, but I think there is something to be said for the difference in terminology. In my teaching context, students are completing a ‘project’, but there is a minimum standard they have to reach to be able to ‘pass’ the assessment. Also, there is less focus on a ‘driving question’ than a PBL task would have – more of an emphasis on the products needing to be made. Hence my use of the term ‘challenge’ in the overall task.
OK, the easiest way to show you the assignment is to share copies of my assignment sheets:
A matrix of challenge tasks is provided for students to choose from in assignment 3.
Students will receive a grade for Assignment 3 based on the number of challenges completed:
- 4 CHALLENGES COMPLETED = PASS
- 6 CHALLENGES COMPLETED = CREDIT
- 8 CHALLENGES COMPLETED = DISTINCTION
- 10 CHALLENGES COMPLETED = HIGH DISTINCTION!
Note the peer assessment component of this task. This is something I am especially proud of, for a number of reasons! Not only am I hoping that this will result in a more sustainable marking practice for me (I will be checking/validating the peer marking, but no re-doing it), but it is also a strategy for getting the students to learn how to share their work and act as ‘critical friends’. I also think that having anopther preservice teacher assess your work in this context can be seen as providing an ‘authentic audience’ for student work.
The student portfolios for this task are due next Friday, so I’ve yet to see how this new assessment plays out in real life.
One idea I have bubbling away about the teaching methods chosen is that ‘project-based’ learning can perhaps be broken down further as being either ‘inquiry-driven’ or ‘challenge-driven’ (and maybe even a third category, ‘play-driven’). But that’s a hierarchy that I’m still thinking through…
There is a lot going on here, I realise. But I’d seriously LOVE to hear feedback from my critical friends, including any students that end up reading this post :)
If you have any questions to ask, shoot them at me too! Obviously I’m quite proud of what I’ve constructed here, but in a few weeks it will be time to reflect again on how to improve for semester 2, so as they say…bring it!
This semester I have been engaing in the final cycle of my teaching and learning action research project – part of what I do here at QUT as an ‘Early Career Academic’.
‘Constructing a community of practice in English Curriculum Studies 1 – online and offline’
Action research cycle:
- Planning and fact-finding: 2010, semester 2
- Phase 1 action: 2011, semester 1
- Phase 2 action: 2011, semester 2
- Phase 3 action: 2012, semester 1
- Report findings: 2012, semester 2
The buzz term for how to ‘do’ curriculum planning here at uni is constructive alignment. Anyone else having to use this term?
Basically, constructive alignment is what you do when you make sure your assessment tasks match your learning objectives, and that your lesson materials feed into this productively. (OK, so I slipped the word ‘productively’ in just there…can you tell I’m living in Queensland? Productive pedagogies, anyone?)
So, the first two phases of my action research have been all about getting the assessments to work for me and my unit, English Curriculum Studies 1. I inherited a bunch of learning objectives when I took on coordination of this unit, but in the end I found that the assessment tasks weren’t engaging students in the ways I knew could happen. In the ways I was sure could happen, anyway. All of the assessment pieces have now been modified or replaced (not allowed to change the learning objectives) and things are aligning much more constructively…
The last piece in the puzzle that I was really hoping to nut out in this third cycle is the establishment of threshold concepts for this unit.
A ‘threshold concept’ is the kind of concept that, once learned, cannot be unlearned. Once we grasp a piece of threshold knowledge, we pass over a barrier into new territory, where everything is seen anew with different eyes.
In the (bazillion) Powerpoint presentations I sat through last year as a new academic, I picked up the importance of using a few well-chosen threshold concepts to drive a unit of work. For teachers like me that prefer to use project-based and inquiry-based learning approaches, having a set of threshold concepts in mind that you want students to ‘get’ by the end of the experience looks to be an excellent anchor for lesson planning. Although these concepts are related to the official learning objectives of the unit, they do serve a different kind of function…and I really want to settle on what mine are!
Until this week I was still struggling to come up with suitable concepts.
But now, I struggle NO MORE!
I have been working on a summary video for students to watch at the half-way point in semester, while I am away at a conference. In the video I want to recap the main points learned from weeks 1-5 of the unit. The process of trying to identify what the ‘big ideas’ were amongst all of the super important stuff we learned wasn’t easy. But the process of having to present the ideas to my students (not just to my academic review panel at the end of this year…!) has really helped.
Which I guess just goes to show that even teachers need an authentic audience for their work.
Trying to keep the video short (under 5 minutes) also forced my hand – left to my own devices, I’m sure I could find plenty of threshold concepts, but you only need a few. The wording of what I’ve chosen isn’t quite right yet, but these are the six big points I have chosen:
- Your personal teacher identity is unique and reflects your personal experience, but will inevitably draw on many established philosophies and practices.
- In ‘English’ we study: semiotics, text and context.
- Language codes and conventions are socially constructed.
- Verbal/linguistic language is just one semiotic ‘code’; we also learn/teach audio, visual, spatial and gestural language.
- Literacy involves more than code breaking – we also make meaning, use texts functionally, and critique texts.
- Multiliteracies pedagogies are currently favoured in English curriculum theory.
I suspect this is still too many for 6 weeks, but there you go. We’ll see. Once I’ve finished the video I’ll post it up here on the blog. I still have to add the narration, but most of the images are in. I’m using Movie Maker and Audacity as my tools of the trade…I hope the students have time to watch the bloody thing! But even if they don’t, I’m glad I went through this process and am happy that I’ve found some threshold concepts to settle on, for now. And, with any luck, a shiny new resource at the end I can be proud of. Fingers crossed!
- Post to celebrate completion of my PhD: CHECK.
- Post with an update on my upcoming conference papers: CHECK.
So…where to next?
As fate had it, this decision was made for me, with the arrival of a piece of student writing in my inbox.
The author of the piece is a recently graduated HSC student, one whom I had the pleasure of teaching year 8 English, and coaching for debating :) This is him counting down the days until the end of his exams:
I invite you to read his work (below), which he has given me permission to reproduce (along with his picture) in this post. Oriniginally published as a Facebook post on October 28th, it is a re-writing of Martin Luther King Jr.’s famous I Have a Dream speech, which has been adapted to make a satirical commentary on the HSC. It comes with a mild language warning (c’mon; it’s satire!), and is a brilliant example of a ‘textual intervention’.
I’m very proud to feature it here as my first ‘guest post’!:
I Have A Dream that the HSC Will End
By B. Wylie
I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our state.
Two score years ago, an a*shole bureaucrat, in whose symbolic shadow we stand today, created the Higher School Certificate. This momentous decree came as a great source of pain and suffering to millions of NSW students who were about to be seared in the flames of withering injustice. It came as a sorrowful dusk which signalled the beginning of their long night of academic captivity.
But fourty four years later, the student still is not free. Fourty four years later, the life of the student is still sadly crippled by the manacles of standardised testing and the chains of rankings. Fourty four years later, the student lives on a lonely island of studying in the midst of a vast ocean of facebook updates. Fourty four years later, the student is still languished in the rooms of NSW high schools and finds himself an exile in his own class. And so we’ve come here today to dramatise a shameful condition.
In a sense we’ve come to this facebook note to cash a check. Read the rest of this entry »
So many things to blog about at the moment…transmedia and transliteracy, the Gonski review of school funding…but in the thick of Semester 2 teaching I find myself inexorably drawn back to curriculum studies.
And goddess, please bless Bianca for coming through with a new blog post about Project Based Learning (PBL) to stimulate my thinking this week!
I have been trying to work out how to formally incorporate PBL into the structure of my unit English Curriculum Studies 1. This week I think I have a solution, which I’ll outline below. But first, to answer Bianca’s question: when I proposed this structure in a comment on her blog she asked:
Did you design the assessments or the pedagogy first?
And that question, RIGHT THERE, is our chicken and egg, am I right?
Because, as Bianca rightly points out, school teachers find it very challenging to engage in “inherent ‘assessment for learning’ within the rigid ‘assessment of learning’ framework already in place”. So, while it might seem logical that your pedagogy will determine your assessment, the ‘reality’ of teaching and learning puts this possibility beyond reach for most.
For some schools their ‘rigid assessment of learning framework’ is tied to NAPLAN exams, for others it is focussed more on Year 12 exit credentials. And in schools that claim not to be driven by external assessments, rigid assessment frameworks can still be constructed by Heads of Department (or others) who seek to place multiple additional constraints on teachers’ planning (e.g. “you MUST have a half yearly exam!”, “every Year 9 class must write an essay in term 1″)
The curriculum places constraints on assessment and pedagogy too, and I could start talking about the Australian Curriculum here. Instead I’ll show you what I built for the university semester context, and try to answer Bianca’s question from there.
Here is the draft outline for my unit in 2012:
- Weeks 1-4 focus: Inquiry based learning (assessment = critical/reflective essay) assessment as learning
- Weeks 5-7 focus: Project based learning (assessment = project + review of pedagogy used in class project) assessment for learning
- Weeks 8-9 focus: Challenge based learning (assessment = make lesson plans for English) assessment of learning
I can safely say that for this unit, I started with the assessment. Literally, I have adopted an existing unit with existing assessment pieces that take at least 6 months to get formally changed. So, while I have been tweaking each assessment piece each semester, I’ve been teaching it for 18 months now and a full overhaul of the structure is now needed to fully incorporate PBL and other constructivist approaches.
Beyond that initial point of departure though, I have oscillated between a pedagogy focus and an assessment focus each time I plan and change something in the unit.
I would say my major points of development around pedagogy and assessment were:
- Reviewing the balance of assessment FOR learning and OF learning in the existing unit. In the university context it is only possible to mandate summative assessment…so I had to reconsider my approach to build a learning environment where the learning process was valued.
- Reviewing the first summative assessment, which was a critical essay, gave me the idea to make the relevance or ‘connectedness’ of the opening weeks of the unit more apparent. Students now do a range of inquiry-based activities to help them engage in the scholarly material, motivated by the need to interrogate their own perspective.
- Activities planned for the first few weeks of the unit were redesigned around a new assessment that focussed on the students personal teaching philosophy. This increased the potential of the assessment to be FOR learning, I thought.
- Teaching the new opening to the unit was really affirming, but showed up the weaknesses in the pedagogy of weeks 5-7. A PBL approach was therefore introduced to ‘liven up’ this part of the unit. This coincides with the time in semester when students begin having heaps of assignments due, and I felt they needed a pedagogical experience that was less ‘intense’, and enjoyable enough to get them through the ‘hump weeks’!
- The PBL appraoch worked really well, but the students put a lot of work in that wasn’t rewarded in assignment grades. So I am now redesigning assignment 2 to include ‘project participation’ criteria so students can get their work on this counted in their final grade.
- aaand…MOST recently: because the final assessment of creating alesson plans really has proven a ‘challenge’, I’m going to use this to explore Challenge based learning. I see this as being the same as Project based learning, but where the outcome does not have to be presentation to an audience. Instead, the project outcome must ‘meet the challenge’. Think Mythbusters :)
You can see how thinking about assessment and pedagogy are totally bound together – thinking about one always raises questions for the other. Or, it should!
I’m still searching for material that can explain the realtionship between Inquiry, Project and Challenge based learning. I’ve tried to use them here in a complementary way, but tbh it’s been tough to find sources that relate the approaches to one another. I started off this process thinking they were slightly interchangable. Now I can see that each one is informed by a respect for ‘learning by doing’, but has its own unique flavour. But are these three the only three? Do they sit in a hierarchy of some kind? Are there other ‘Something-B-Ls’ out there that I don’t know about??
If you do, please add a comment! (I hope this helps someone out there!)
In NSW yesterday Year 12 school leavers got their HSC results back. And again, we reflect.
so-and-so got x amount of Band 6s this year…should I teach more like them?
my kids didn’t go as well as they had hoped…did I fail them?
there were some great successes at our school…what pressures will this bring next year?
The dizzying heights…the devastating lows.
I’m sure this post / these tweets should have some ‘IMHO’s peppered through them, but stuff it – the HSC is an evil device.
I’m so proud of every HSC student who got through the year, and was beyond excited for my ex-students who got the results they sought (I always will be). Motivation, goals, mastery, achievement, I believe in them all. But the HSC provides them too sparingly, for students and their teachers.
And now it’s time for recovery. again.
Congratulations one and all – bring on 2011…