Posts Tagged assessment
One of my students followed up this investigation with the following juicy question:
Essential fluencies seem to structure skills within select criterion, however I am curious as to whether PBL uses these as guides (depending on the student’s PBL objective) or whether students are meant to meet all of these at different stages of their PBL (to achieve a final product)?
If this is a flexible criteria, would using a feedback grid be the most effective way of communicating the development of an idea (as it focusses less on curriculum goals, more on constructive advice)?
I decided to post my answer to part of this question here on the blog:
You’ve asked a good question about skills and standards. My understanding of PBL (and other inquiry-based models) is that assessing skills is just as important as assessing content knowledge.
There are two (opposing) axioms that relate to this:
- ‘What gets measured gets done’.
- ‘Not everything that matters can be measured; not everything that can be measured matters’.
At the moment I’m inclined to agree with the PBL movers and shakers – that developing ‘soft skills’ should be seen as a vital curriculum goal, just as important as the acquisition of discipline knowledge and technical skills. The argument here is that if we don’t find a way of measuring/assessing soft skills then teachers will continue to sideline them. Because ‘what gets measured gets done’.
The BIE crowd have developed a range of assessment rubrics for the four skills that they identify as most important to PBL specifically: creativity and innovation, presentation/communication, collaboration, and critical thinking. You can find them here:
Of course, the opposing view is that such assessment rubrics lead people to forget the second axiom ‘not everything that matters can be measured’. I know sometimes I’ve watched presentations for example that are awesome, but their awesomeness can’t be explained using the BIE assessment rubric. It’s like all rubrics actually need a criteria labelled “X factor!” for when a piece of work or project does something amazing that we didn’t plan to (or cannot) measure. And sometimes by focussing students so explicitly on assessment rubrics, they can get obsessed with how to ‘game’ the criteria to reach the highest standard, rather than taking risks in their learning to work toward a big-picture goal.
As there is no ‘Ultimate God of PBL’, we are free to use whatever framework we want to think about “soft skills”. We can take up the Essential Fluencies, we can take up the skills foregrounded by BIE, we can use the 4Cs proposed by p21.org, or we can use the General Capabilities from the Australian Curriculum.
But ultimately I’d argue that yes, whatever framework you choose, you should find a way of explaining to students the standards you are looking for on a range of criteria, for the particular project they’re working on. Assessment rubric sheets should be designed to make the criteria and expected standards transparent to the learner, and to aid the feed-forward process throughout a project as well as the feed-back process at the end of a project.
I know I haven’t answered all of the parts of this student’s juicy question, and we’ll be talking more about it in class. It may generate another blog post. In the meantime…
- How would you answer this student’s question?
- Do you agree that providing assessment rubrics for soft skills is useful for learning in PBL (or otherwise)?
It’s that time of year. Teachers of Year 12 around Australia are scrambling to varying degrees to prepare students for final assessments and exams, which inevitably involves a whole lotta marking.
Of course, all teachers have to grade student work. And they are engaged in doing this all year. But nothing beats the pedal-to-the-metal feeling of marking Year 12 practice tasks in a last ditch effort to refine their examination responses.
In particular, nothing beats the hellish pressure that exists in states like NSW and Victoria where the HSC and VCE exams respectively loom over teachers and students alike. And out of all these teachers and students, I argue that subjects that are writing-intensive (e.g. English and History) have it the toughest; if you have a class of 25 for Year 12 and it’s coming up to an assessment, teachers in these subjects are spending their nights and weekends correcting pages and pages and pages of long form expositions.
Which can leave your eyes (and soul) feeling kinda like this:
I was prompted to write this blog post after watching my friends Justin and Alex tweet about their marking yesterday:
I’ve taught for the HSC three times and this slavish marking routine is the only part I do not miss…having said that, the jolly task I have now of marking as a university lecturer has involved marking binges that certainly rival the pain of HSC workload.
The question is – what can we do about it?
Is there anything we can do about it?
Some ideas that I threw out into the twittersphere yesterday seem promising, but without a class to try them on I’m at a loss, not sure if they would work. The ideas I bounced around with Justin and Alex were:
- Focussing on writing just the introduction, or a body paragraph. This would make the task smaller and more focussed for students, and more manageable to mark 25-30 of them.
- Setting a paragraph writing challenge. To address Justin’s problem of the student that only writes about ‘tone’, each week set a different language feature/form for students to write a paragraph on. By the end of the term they will have a bank of paragraphs on different elements.
- Gamify the writing process. This could be done by putting students in groups, getting every student to write a paragraph (or essay), then each group submits it’s best one (as judged by the students in the group) for marking. This means you only have to mark one essay/paragraph per group, not per student. Keep a chart of which group wins each week and award them a prize at the end of the unit. Change the groups around for each new unit.
- Peer assessment. This can only be used in a limited way, as students don’t have the capacity to grade work to a Year 12 standard. However you could use the ‘medals (feedback) and missions (feedforward)’ framework that Bianca draws on to give students a direction. I think the main benefit is that they read each other’s work and discuss their strengths, not that they actually give each other a ‘grade’.
- Find an authentic audience. Partnering up with another teacher/class would provide an avenue for students to share their work with another class on a platform such as a wiki. This would give students someone to perform for besides their own teacher, which could prove motivating. The teachers could also arrange to do a marking-swap, and grade each other’s student essays…this may get you writing less comments, marking more objectively (?) and just plain old provide a change of pace as you get to read a different set of handwriting!
I really hope these ideas are useful to someone out there.
If you have any other good ideas for getting feedback to students without going through so much of the eye-bleedingly painful million-essay marking process, I would LOVE to hear them!
Thanks to Justin and Alex for inspiring this post and helping me brainstorm ideas🙂
Images: Cropped screen still from True Blood, Season 5; Screen shot of conversation on Twitter.com
Postscript: If you liked this post, you may also like the post Matt Esterman wrote today, ‘The home stretch for Year 12’. Looks like we all have Year 12 on the brain this weekend!
Last year I was interviewed by Melissa Wilson, a Journalism student from the University of Newcastle, about my views on the Higher School Certificate (HSC).
Melissa was in contact again recently, and this prompted me to ask if I could reproduce the interview here on my blog. She kindly obliged, and so here it is!
I was careful to read back over my answers, to make sure I still felt the same way about these issues. I do. I wish I could say that things were vastly different up here in Queensland. They’re not. When a Queenslander tells you there is no external exam for Year 12 in their state, they’re misleading you at best. Here is a bit of information from the QSA website about the Queensland Core Skills (QCS) test:
Preparing for the test
The Common Curriculum Elements are generic skills that students work with across their subjects; therefore the real preparation for the test goes on all the time and in every subject. The QSA also makes available a variety of test preparation resources, including Retrospectives and past testpapers (see QCS Test publications and Retrospectives and MC response sheets). Most schools provide some focused preparation for the test.
Hmm, sounds familiar.
But, I digress…
Here are my responses to the interview by Melissa Wilson:
Interview answers for Melissa Wilson (University of Newcastle, 2011)
Interviewee: Kelli McGraw (Lecturer, QUT)
- How do you feel about how the HSC is structured in 2011?
When I think about the HSC structure in 2011, the main things that leap to mind are the fact that studying English is mandatory, that half of the student assessment is based on a timetabled external examinations, and that a no more than of 30% of your school assessment is supposed to be ‘exam-type’. I think it’s really important for English to remain compulsory right up to the end of school, but I’d like to see more room for students to choose electives within the course, not just different levels i.e. Standard or Advanced English. At the moment I think the HSC is still structured in a way that is too rigid for students to feel like they have a lot of choice over their learning.
- Many people say that the HSC is focused on teaching students a whole lot of information that isn’t exactly relevant to them later in life – but instead they just regurgitate it in an exam and then discard it – how do you feel about that statement?
Personally, I can think of countless things that I learned in my HSC year (1998). In those days the emphasis on exams was just as great, but I am often surprised by the things I remember from senior high school and have found a lot of what I learned to be very relevant in life. Having to finish ‘major works’ for Visual Art and Drama also taught me valuable lessons about project management and self-directed learning, which I didn’t get from participating in written exams, so in that sense I guess I was lucky to be an ‘art-sy’ student.
I think the real problem with exams is not that students have to cram ‘irrelevant’ information – I think that all learning can be made relevant, depending on what you choose to do in life. The problem I find is that the examination system has too much of an effect on what happens inside the classroom. The constant pressure to cover content is a strain on students and teachers, and even though school-based assessment is supposed to involve deep learning and reflection, many schools I know of set far more than 30% of their assessments in an exam style in order to condition students in preparation for the external exam. So I think there is a ‘hidden curriculum’ in the HSC, which can dilute learning experiences based on the official subject syllabuses.
- And from this, what would you personally change about the HSC?
I think that the only assessment that students should have to do under exam conditions is the Trial. If more student work was assessed through project work, or using collaborative group tasks, or using portfolios, I think that students would feel more connected to the learning, and be motivated to achieve. Even though the HSC now uses criteria-based assessment, students are acutely aware that the HSC places them in competition with one another as in-class assessment ranks still play a role in determining a student’s final subject results, and the year culminates for most students in receiving a ranked national placement through the UAI. With only about 30% of students moving from school to university after Year 12, it seems like we compromise a lot of educational values for the sake of a privileged minority.
- How do you feel about the pressure and emotional stress that students endure throughout their HSC?
When I think about the stories that students have told me over the years – about how they feel inadequate, or like a failure in the face of HSC assessment tasks – it makes me really upset. I have seen a lot of students in Year 12 lose a lot of weight, with girls in particular showing signs of early and advanced eating disorders. Senior school is also a time when increased numbers of students pick up casual and part-time employment, in many cases out of a necessity to contribute to household finances. I think the HSC creates an environment where students are given too many adult responsibilities without being given the corresponding rights.
While schools play a vital role in developing students’ resilience and capacity for work, the emotional stress endured during the HSC year is too much, in my opinion. I read a study awhile back where Year 11 and 12 students reported symptoms of anxiety, depression and stress that fell outside the ‘normal’ range. We know that when this happens, students stop focussing on ‘mastering’ the material, instead focussing on performance; they stop believing in themselves, stop seeing the learning as a worthwhile goal, and switch to performance-oriented goals. Some of my own school friends took years to recover from the emotional damage of the HSC year, especially those whose final results didn’t meet expectations.
Hi folks – it’s been a bit quiet here on the blog, I know.
All I can say is … omg MARKING!
I have always had an interest in assessment, but this semester has made really clear to me how dire the situation is with our current practices.
I don’t want to ‘buy out’ my marking (i.e.pay someone else to do it for me) but I feel like I am wasting so much of my time at the grindstone, like a machine, writing the same lines over and over in delightful pink pen in the margins of my students’ work.
“Check the APA style guide for rules about how to format this”
“Formal essays require shorter paragraphs than this”
“Avoid rhetorical questions – make strong statements instead”
“Use your introduction to tell me what your main points will actually be, not to explain the structure of your work”
“Don’t use a quote as a sentence on it’s own – introduce it i.e. ‘Sawyer (year) explains that…'”
“You have not included reference to any unit readings in this rationale”
I worry about RSI. I worry about carpal tunnel! Marking more tasks electronically next semester will hopefully fix the hand ache, but what about the mind ache??
I’m not alone – every teacher reading this knows what I mean.
What are we going to do about it?
Teaching at university can be tricky, mostly due to the emphasis on summative assessment.
Since starting this position in 2010 I have been attempting to infuse the unit I coordinate with greater amounts of project-based learning. However, in a context where students have little time or incentive to engage with classwork that isn’t formally assessed, it has been hard to reward things like student project work.
After three semesters of teaching English Curriculum Studies 1 I decided that a radically new assignment was in order.
Students used to do:
- Assignment 1 – Personal teaching philosophy statement and resource analysis
- Assignment 2 – Report on video lessons and learner needs observed
- Assignment 3 – Junior secondary English lesson plans
All of these assessment pieces were completed individually – no collaboration was required and no public audience was utilised.
From this semester onward, students now do:
- Assignment 1 – Personal teaching philosophy statement and resource analysis (same as before)
- Assignment 2 – Junior secondary English lesson plans (now completed in small groups of 2 or 3)
- Assignment 3 – A range of CHALLENGE TASKS published in a portfolio <– SCHMICK NEW TASK!
The New Task:
Many of the key ideas about inquiry-based and cooperative learning that I am working with can be found in a book extract provided by Edutopia: Teaching for Meaningful Learning by Brigid Barron & Linda Darling-Hammond.
Here is a brief extract – some words about project-based learning:
“Project-based learning involves completing complex tasks that typically result in a realistic product, event, or presentation to an audience. Thomas (2000) identifies five key components of effective project-based learning. It is: central to the curriculum, organized around driving questions that lead students to encounter central concepts or principles, focused on a constructive investigation that involves inquiry and knowledge building, student-driven (students are responsible for designing and managing their work), and authentic, focusing on problems that occur in the real world and that people care about.” (Barron & Darling-Hammond, 2008, p. 3; my emphasis)
What I’ve done in my new task is to create a poetry ‘project’ as one of 10 ‘challenges’ that students need to complete.
After trialling a poetry project last semester, I know that students see value in, and engage with this kind of learning. But, at the end of the day, students felt let down because the work they put into their projects didn’t ‘count’ towards their final grade.
Once I started messing around with a new assignment that gave them credit for their project work, it was too hard not to design a whole suite of ‘challenges’ that they could choose to take up! So, that’s what I’ve done – students decide what grade they want to get, and complete the number of challenges needed to obtain it.
‘Challenge-based learning‘ as a term has not gained as much traction as ‘project-based learning’, but I think there is something to be said for the difference in terminology. In my teaching context, students are completing a ‘project’, but there is a minimum standard they have to reach to be able to ‘pass’ the assessment. Also, there is less focus on a ‘driving question’ than a PBL task would have – more of an emphasis on the products needing to be made. Hence my use of the term ‘challenge’ in the overall task.
OK, the easiest way to show you the assignment is to share copies of my assignment sheets:
A matrix of challenge tasks is provided for students to choose from in assignment 3.
Students will receive a grade for Assignment 3 based on the number of challenges completed:
- 4 CHALLENGES COMPLETED = PASS
- 6 CHALLENGES COMPLETED = CREDIT
- 8 CHALLENGES COMPLETED = DISTINCTION
- 10 CHALLENGES COMPLETED = HIGH DISTINCTION!
Note the peer assessment component of this task. This is something I am especially proud of, for a number of reasons! Not only am I hoping that this will result in a more sustainable marking practice for me (I will be checking/validating the peer marking, but no re-doing it), but it is also a strategy for getting the students to learn how to share their work and act as ‘critical friends’. I also think that having anopther preservice teacher assess your work in this context can be seen as providing an ‘authentic audience’ for student work.
The student portfolios for this task are due next Friday, so I’ve yet to see how this new assessment plays out in real life.
One idea I have bubbling away about the teaching methods chosen is that ‘project-based’ learning can perhaps be broken down further as being either ‘inquiry-driven’ or ‘challenge-driven’ (and maybe even a third category, ‘play-driven’). But that’s a hierarchy that I’m still thinking through…
There is a lot going on here, I realise. But I’d seriously LOVE to hear feedback from my critical friends, including any students that end up reading this post🙂
If you have any questions to ask, shoot them at me too! Obviously I’m quite proud of what I’ve constructed here, but in a few weeks it will be time to reflect again on how to improve for semester 2, so as they say…bring it!
This semester I have been engaing in the final cycle of my teaching and learning action research project – part of what I do here at QUT as an ‘Early Career Academic’.
‘Constructing a community of practice in English Curriculum Studies 1 – online and offline’
Action research cycle:
- Planning and fact-finding: 2010, semester 2
- Phase 1 action: 2011, semester 1
- Phase 2 action: 2011, semester 2
- Phase 3 action: 2012, semester 1
- Report findings: 2012, semester 2
The buzz term for how to ‘do’ curriculum planning here at uni is constructive alignment. Anyone else having to use this term?
Basically, constructive alignment is what you do when you make sure your assessment tasks match your learning objectives, and that your lesson materials feed into this productively. (OK, so I slipped the word ‘productively’ in just there…can you tell I’m living in Queensland? Productive pedagogies, anyone?)
So, the first two phases of my action research have been all about getting the assessments to work for me and my unit, English Curriculum Studies 1. I inherited a bunch of learning objectives when I took on coordination of this unit, but in the end I found that the assessment tasks weren’t engaging students in the ways I knew could happen. In the ways I was sure could happen, anyway. All of the assessment pieces have now been modified or replaced (not allowed to change the learning objectives) and things are aligning much more constructively…
The last piece in the puzzle that I was really hoping to nut out in this third cycle is the establishment of threshold concepts for this unit.
A ‘threshold concept’ is the kind of concept that, once learned, cannot be unlearned. Once we grasp a piece of threshold knowledge, we pass over a barrier into new territory, where everything is seen anew with different eyes.
In the (bazillion) Powerpoint presentations I sat through last year as a new academic, I picked up the importance of using a few well-chosen threshold concepts to drive a unit of work. For teachers like me that prefer to use project-based and inquiry-based learning approaches, having a set of threshold concepts in mind that you want students to ‘get’ by the end of the experience looks to be an excellent anchor for lesson planning. Although these concepts are related to the official learning objectives of the unit, they do serve a different kind of function…and I really want to settle on what mine are!
Until this week I was still struggling to come up with suitable concepts.
But now, I struggle NO MORE!
I have been working on a summary video for students to watch at the half-way point in semester, while I am away at a conference. In the video I want to recap the main points learned from weeks 1-5 of the unit. The process of trying to identify what the ‘big ideas’ were amongst all of the super important stuff we learned wasn’t easy. But the process of having to present the ideas to my students (not just to my academic review panel at the end of this year…!) has really helped.
Which I guess just goes to show that even teachers need an authentic audience for their work.
Trying to keep the video short (under 5 minutes) also forced my hand – left to my own devices, I’m sure I could find plenty of threshold concepts, but you only need a few. The wording of what I’ve chosen isn’t quite right yet, but these are the six big points I have chosen:
- Your personal teacher identity is unique and reflects your personal experience, but will inevitably draw on many established philosophies and practices.
- In ‘English’ we study: semiotics, text and context.
- Language codes and conventions are socially constructed.
- Verbal/linguistic language is just one semiotic ‘code’; we also learn/teach audio, visual, spatial and gestural language.
- Literacy involves more than code breaking – we also make meaning, use texts functionally, and critique texts.
- Multiliteracies pedagogies are currently favoured in English curriculum theory.
I suspect this is still too many for 6 weeks, but there you go. We’ll see. Once I’ve finished the video I’ll post it up here on the blog. I still have to add the narration, but most of the images are in. I’m using Movie Maker and Audacity as my tools of the trade…I hope the students have time to watch the bloody thing! But even if they don’t, I’m glad I went through this process and am happy that I’ve found some threshold concepts to settle on, for now. And, with any luck, a shiny new resource at the end I can be proud of. Fingers crossed!