Posts Tagged assessment

Slaves to the Grade

It’s that time of year. Teachers of Year 12 around Australia are scrambling to varying degrees to prepare students for final assessments and exams, which inevitably involves a whole lotta marking.

Of course, all teachers have to grade student work. And they are engaged in doing this all year. But nothing beats the pedal-to-the-metal feeling of marking Year 12 practice tasks in a last ditch effort to refine their examination responses.

In particular, nothing beats the hellish pressure that exists in states like NSW and Victoria where the HSC and VCE exams respectively loom over teachers and students alike. And out of all these teachers and students, I argue that subjects that are writing-intensive (e.g. English and History) have it the toughest; if you have a class of 25 for Year 12 and it’s coming up to an assessment, teachers in these subjects are spending their nights and weekends correcting pages and pages and pages of long form expositions.

Which can leave your eyes (and soul) feeling kinda like this:

bill crying blood

I was prompted to write this blog post after watching my friends Justin and Alex tweet about their marking yesterday:

twitter convo 24.08.2013 HSC marking

I’ve taught for the HSC three times and this slavish marking routine is the only part I do not miss…having said that, the jolly task I have now of marking as a university lecturer has involved marking binges that certainly rival the pain of HSC workload.

The question is – what can we do about it?

Is there anything we can do about it?

Some ideas that I threw out into the twittersphere yesterday seem promising, but without a class to try them on I’m at a loss, not sure if they would work. The ideas I bounced around with Justin and Alex were:

  1. Focussing on writing just the introduction, or a body paragraph. This would make the task smaller and more focussed for students, and more manageable to mark 25-30 of them.
  2. Setting a paragraph writing challenge. To address Justin’s problem of the student that only writes about ‘tone’, each week set a different language feature/form for students to write a paragraph on. By the end of the term they will have a bank of paragraphs on different elements.
  3. Gamify the writing process. This could be done by putting students in groups, getting every student to write a paragraph (or essay), then each group submits it’s best one (as judged by the students in the group) for marking. This means you only have to mark one essay/paragraph per group, not per student. Keep a chart of which group wins each week and award them a prize at the end of the unit. Change the groups around for each new unit.
  4. Peer assessment. This can only be used in a limited way, as students don’t have the capacity to grade work to a Year 12 standard. However you could use the ‘medals (feedback) and missions (feedforward)’ framework that Bianca draws on to give students a direction. I think the main benefit is that they read each other’s work and discuss their strengths, not that they actually give each other a ‘grade’.
  5. Find an authentic audience. Partnering up with another teacher/class would provide an avenue for students to share their work with another class on a platform such as a wiki. This would give students someone to perform for besides their own teacher, which could prove motivating. The teachers could also arrange to do a marking-swap, and grade each other’s student essays…this may get you writing less comments, marking more objectively (?) and just plain old provide a change of pace as you get to read a different set of handwriting!

I really hope these ideas are useful to someone out there.

If you have any other good ideas for getting feedback to students without going through so much of the eye-bleedingly painful million-essay marking process, I would LOVE to hear them!

Thanks to Justin and Alex for inspiring this post and helping me brainstorm ideas :)

 

Images: Cropped screen still from True Blood, Season 5; Screen shot of conversation on Twitter.com

Postscript: If you liked this post, you may also like the post Matt Esterman wrote today, ‘The home stretch for Year 12′. Looks like we all have Year 12 on the brain this weekend!

, ,

8 Comments

A ‘rhizomatic’ take on Semester one so far

Last week I had the good fortune to hear Professor Diana Masny speak about her Deleuzian approach to researching multiliteracies theory (which she referred to as ‘MLT’). Masny is from Ottowa, Canada, and is an adjunct prof at QUT.

In this presentation I was returned to the idea of ‘the rhizome’, something that had interested me when I encountered the work of Deleuze and Guattari. The idea behind looking at things rhizomatically is that we can stop focussing on binary oppositions, or organising concepts into ordered taxonomies and such. Instead, rhizomatic analysis involves looking at things and ideas spread/propagate…and at where new possibilities ‘shoot off’ out of of what already exists.

A rhizome in plant form

A rhizome in plant form

This talk by Masny was interesting for a number of reasons to do with designing research methodology, as well as considering MLT from new angles. One thing that inspired me was the way that her presentation was organised around ‘entry points’ to her own topic as a rhizomatic collection of findings. This is in contrast to a presentation that tries to summarise ‘key findings’ or ‘ways forward’. Seeing as I most often use my blog to reflect on ‘findings’ and ‘planning’, I thought it might make a nice change to adopt Masny’s (after Deleuze’s) approach of exploring the ‘entry points’ into my practice so far this semester…

ENTRY POINT: Attendance

At QUT we have a policy that attendance is not to be counted in any way toward assessment, and that students choosing to catch up on their study from home are to be supported in that choice. I have heard some lecturers complain about this – they think students would learn better if they turned up to all the classes, and wish the university would enforce this. Most of us, however, respect the purpose of this arrangement, which is to provide flexible study options for the grown-up human beings that are our ‘students’, and cater for a range of learning styles. Personally I find it very motivating, as it forces me to think about HOW I can make my lessons “worth coming to”!

I’m really happy with the attendance rate in my classes at the moment. Out of the 110 students I have studying on campus, almost 100% turned up in Week one, and the students that were away mostly emailed in their apologies. In Week 2, attendance in tutorials and the lecture was down to about 85%, which is to be expected. What I am eager to see is that 85% attendance rate maintained for the rest of the 9-week semester, rather than drop of over time to 20-50%, as other lecturers often report. I’m pleased to say that in the past few years here, I haven’t noticed the same kind of drop of, and I like to think this reflects the usefulness of my classes.

ENTRY POINT: Engagement

As always it has been a slow start on Twitter…but as always, there are several students ‘coming around’ to the tool already and engaging with informal peer tutoring as well. Once again, I am glad I chose to persevere with introducing students to an unfamiliar (and for many of them, unloved) social media tool.

I had a really great out-of-context engagement moment as well last week, on Pinterest. I use Pinterest among other things to collect useful resources for English teachers, and one day I saw a collage about English teaching and ‘re-pinned it’ to my board. I thought (and commented) ‘wow…this is just like an activity I do in class!’. Then I realised that I was following one of my students already, and that it was her! Funniest bit was though, she had been following me too without realising who I was, or making any connection to out uni lives. Good times!

There has been a growth in socia media profiles and ‘chats’ that I can now connect my students to, and the most important of these is the #ozengchat that takes place on Twitter on Tuesday nights. Feeling like they are engaging with ‘real teachers’ seems to be helping with motivation in the class, but at the moment that’s just my anecdotal take on the situation.

ENTRY POINT: Assessment

In my class students undertake THREE assessment tasks:

  1. Personal essay on teaching philosophy and resource analysis (individual, 30%)
  2. Lessons plans for a junior English class (in pairs, 40%)
  3. Portfolio of completed learning ‘challenge tasks’ (individual, 30%)

What I like about what I have achieved with this set of assessments is that there is a balance of individual and group work, that there is a variety of tasks, and that no task is worth more that 40%.

At this point I’ll put myself out there to say I am disappointed to see how many uni coordinators choose to use just TWO assessment piece in their own classes. This is not good practice imo! Having less assignments does mean a smaller marking load for the lecturer, and less due dates for the student, but at what cost?

I really do believe that students in uni should not have assessments that are worth 50% or over, as this is too high-stakes to promote good learning. To do this, you must have more than two assessments for a unit in a semester.

FINAL WORDS: The CLB018 ‘assemblage’

In the theory of Deleuze and Guattari, the context of my CLB018 class provides an assemblage of bodies and things that can produce any number of effects. I hoe to keep reporting throughout the semester on the effects (and affects) of our assemblage!

In the meantime, any comments on these POINTS OF ENTRY are most welcome.

, , , , , ,

3 Comments

Evaluating the HSC: An interview

Last year I was interviewed by Melissa Wilson, a Journalism student from the University of Newcastle, about my views on the Higher School Certificate (HSC).

Melissa was in contact again recently, and this prompted me to ask if I could reproduce the interview here on my blog. She kindly obliged, and so here it is!

I was careful to read back over my answers, to make sure I still felt the same way about these issues. I do. I wish I could say that things were vastly different up here in Queensland. They’re not. When a Queenslander tells you there is no external exam for Year 12 in their state, they’re misleading you at best. Here is a bit of information from the QSA website about the Queensland Core Skills (QCS) test:

Preparing for the test

The Common Curriculum Elements are generic skills that students work with across their subjects; therefore the real preparation for the test goes on all the time and in every subject. The QSA also makes available a variety of test preparation resources, including Retrospectives and past testpapers (see QCS Test publications and Retrospectives and MC response sheets). Most schools provide some focused preparation for the test.

Hmm, sounds familiar.

But, I digress…

Here are my responses to the interview by Melissa Wilson:

Interview answers for Melissa Wilson (University of Newcastle, 2011)
Interviewee: Kelli McGraw (Lecturer, QUT)

  • How do you feel about how the HSC is structured in 2011?

When I think about the HSC structure in 2011, the main things that leap to mind are the fact that studying English is mandatory, that half of the student assessment is based on a timetabled external examinations, and that a no more than of 30% of your school assessment is supposed to be ‘exam-type’. I think it’s really important for English to remain compulsory right up to the end of school, but I’d like to see more room for students to choose electives within the course, not just different levels i.e. Standard or Advanced English. At the moment I think the HSC is still structured in a way that is too rigid for students to feel like they have a lot of choice over their learning.

  • Many people say that the HSC is focused on teaching students a whole lot of information that isn’t exactly relevant to them later in life – but instead they just regurgitate it in an exam and then discard it – how do you feel about that statement?

Personally, I can think of countless things that I learned in my HSC year (1998). In those days the emphasis on exams was just as great, but I am often surprised by the things I remember from senior high school and have found a lot of what I learned to be very relevant in life. Having to finish ‘major works’ for Visual Art and Drama also taught me valuable lessons about project management and self-directed learning, which I didn’t get from participating in written exams, so in that sense I guess I was lucky to be an ‘art-sy’ student.
I think the real problem with exams is not that students have to cram ‘irrelevant’ information – I think that all learning can be made relevant, depending on what you choose to do in life. The problem I find is that the examination system has too much of an effect on what happens inside the classroom. The constant pressure to cover content is a strain on students and teachers, and even though school-based assessment is supposed to involve deep learning and reflection, many schools I know of set far more than 30% of their assessments in an exam style in order to condition students in preparation for the external exam. So I think there is a ‘hidden curriculum’ in the HSC, which can dilute learning experiences based on the official subject syllabuses.

  • And from this, what would you personally change about the HSC? 

I think that the only assessment that students should have to do under exam conditions is the Trial. If more student work was assessed through project work, or using collaborative group tasks, or using portfolios, I think that students would feel more connected to the learning, and be motivated to achieve. Even though the HSC now uses criteria-based assessment, students are acutely aware that the HSC places them in competition with one another as in-class assessment ranks still play a role in determining a student’s final subject results, and the year culminates for most students in receiving a ranked national placement through the UAI. With only about 30% of students moving from school to university after Year 12, it seems like we compromise a lot of educational values for the sake of a privileged minority.

  • How do you feel about the pressure and emotional stress that students endure throughout their HSC?

When I think about the stories that students have told me over the years – about how they feel inadequate, or like a failure in the face of HSC assessment tasks – it makes me really upset. I have seen a lot of students in Year 12 lose a lot of weight, with girls in particular showing signs of early and advanced eating disorders. Senior school is also a time when increased numbers of students pick up casual and part-time employment, in many cases out of a necessity to contribute to household finances. I think the HSC creates an environment where students are given too many adult responsibilities without being given the corresponding rights.

While schools play a vital role in developing students’ resilience and capacity for work, the emotional stress endured during the HSC year is too much, in my opinion. I read a study awhile back where Year 11 and 12 students reported symptoms of anxiety, depression and stress that fell outside the ‘normal’ range. We know that when this happens, students stop focussing on ‘mastering’ the material, instead focussing on performance; they stop believing in themselves, stop seeing the learning as a worthwhile goal, and switch to performance-oriented goals. Some of my own school friends took years to recover from the emotional damage of the HSC year, especially those whose final results didn’t meet expectations.

Good riddance to my English notes.

Good riddance to my English notes. Flickr image by Jean-Rene Vauzelle CC-BY-2.0

, , , , ,

3 Comments

Coming up for air

Hi folks – it’s been a bit quiet here on the blog, I know.
All I can say is … omg MARKING!

I have always had an interest in assessment, but this semester has made really clear to me how dire the situation is with our current practices.

I don’t want to ‘buy out’ my marking (i.e.pay someone else to do it for me) but I feel like I am wasting so much of my time at the grindstone, like a machine, writing the same lines over and over in delightful pink pen in the margins of my students’ work.

“Check the APA style guide for rules about how to format this”
“Formal essays require shorter paragraphs than this”
“Avoid rhetorical questions – make strong statements instead”
“Use your introduction to tell me what your main points will actually be, not to explain the structure of your work”
“Don’t use a quote as a sentence on it’s own – introduce it i.e. ‘Sawyer (year) explains that…'”
“You have not included reference to any unit readings in this rationale”

I worry about RSI. I worry about carpal tunnel! Marking more tasks electronically next semester will hopefully fix the hand ache, but what about the mind ache??

I’m not alone – every teacher reading this knows what I mean.

What are we going to do about it?

Portland Oregon & VA exam 46 – CC-BY-2.0 Flickr image by Parker Knight

,

1 Comment

My schmick new assessment design!

Teaching at university can be tricky, mostly due to the emphasis on summative assessment.

Since starting this position in 2010 I have been attempting to infuse the unit I coordinate with greater amounts of project-based learning. However, in a context where students have little time or incentive to engage with classwork that isn’t formally assessed, it has been hard to reward things like student project work.

After three semesters of teaching English Curriculum Studies 1 I decided that a radically new assignment was in order. 

Background:

Students used to do:

  • Assignment 1 – Personal teaching philosophy statement and resource analysis
  • Assignment 2 – Report on video lessons and learner needs observed
  • Assignment 3 – Junior secondary English lesson plans

All of these assessment pieces were completed individually – no collaboration was required and no public audience was utilised.

From this semester onward, students now do:

  • Assignment 1 – Personal teaching philosophy statement and resource analysis (same as before)
  • Assignment 2 – Junior secondary English lesson plans (now completed in small groups of 2 or 3)
  • Assignment 3 – A range of CHALLENGE TASKS published in a portfolio <– SCHMICK NEW TASK!

The New Task:

Many of the key ideas about inquiry-based and cooperative learning that I am working with can be found in a book extract provided by Edutopia: Teaching for Meaningful Learning by Brigid Barron & Linda Darling-Hammond.

Here is a brief extract – some words about project-based learning:

“Project-based learning involves completing complex tasks that typically result in a realistic product, event, or presentation to an audience. Thomas (2000) identifies five key components of effective project-based learning. It is: central to the curriculum, organized around driving questions that lead students to encounter central concepts or principles, focused on a constructive investigation that involves inquiry and knowledge building, student-driven (students are responsible for designing and managing their work), and authentic, focusing on problems that occur in the real world and that people care about.” (Barron & Darling-Hammond, 2008, p. 3; my emphasis)

What I’ve done in my new task is to create a poetry ‘project’ as one of 10 ‘challenges’ that students need to complete.

After trialling a poetry project last semester, I know that students see value in, and engage with this kind of learning.  But, at the end of the day, students felt let down because the work they put into their projects didn’t ‘count’ towards their final grade.

Once I started messing around with a new assignment that gave them credit for their project work, it was too hard not to design a whole suite of ‘challenges’ that they could choose to take up! So, that’s what I’ve done – students decide what grade they want to get, and complete the number of challenges needed to obtain it.

Challenge-based learning‘ as a term has not gained as much traction as ‘project-based learning’, but I think there is something to be said for the difference in terminology. In my teaching context, students are completing a ‘project’, but there is a minimum standard they have to reach to be able to ‘pass’ the assessment. Also, there is less focus on a ‘driving question’ than a PBL task would have – more of an emphasis on the products needing to be made. Hence my use of the term ‘challenge’ in the overall task.

The Challenges:

OK, the easiest way to show you the assignment is to share copies of my assignment sheets:

CLB018-CLP408 challenge portfolio task

A matrix of challenge tasks is provided for students to choose from in assignment 3. 

Students will receive a grade for Assignment 3 based on the number of challenges completed: 

  •  4 CHALLENGES COMPLETED = PASS
  •  6 CHALLENGES COMPLETED = CREDIT 
  •  8 CHALLENGES COMPLETED = DISTINCTION 
  •  10 CHALLENGES COMPLETED = HIGH DISTINCTION! 

CHALLENGE TASK peer assessment sheet

Note the peer assessment component of this task. This is something I am especially proud of, for a number of reasons! Not only am I hoping that this will result in a more sustainable marking practice for me (I will be checking/validating the peer marking, but no re-doing it), but it is also a strategy for getting the students to learn how to share their work and act as ‘critical friends’. I also think that having anopther preservice teacher assess your work in this context can be seen as providing an ‘authentic audience’ for student work.

Reflecting:

The student portfolios for this task are due next Friday, so I’ve yet to see how this new assessment plays out in real life.

One idea I have bubbling away about the teaching methods chosen is that ‘project-based’ learning can perhaps be broken down further as being either ‘inquiry-driven’ or ‘challenge-driven’ (and maybe even a third category, ‘play-driven’). But that’s a hierarchy that I’m still thinking through…

There is a lot going on here, I realise. But I’d seriously LOVE to hear feedback from my critical friends, including any students that end up reading this post :)

If you have any questions to ask, shoot them at me too! Obviously I’m quite proud of what I’ve constructed here, but in a few weeks it will be time to reflect again on how to improve for semester 2, so as they say…bring it!

, , , , ,

4 Comments

Finding my Threshold Concepts

This semester I have been engaing in the final cycle of my teaching and learning action research project – part of what I do here at QUT as an ‘Early Career Academic’.

‘Constructing a community of practice in English Curriculum Studies 1 – online and offline’

Action research cycle:

  • Planning and fact-finding: 2010, semester 2
  • Phase 1 action: 2011, semester 1
  • Phase 2 action: 2011, semester 2
  • Phase 3 action: 2012, semester 1
  • Report findings: 2012, semester 2

The buzz term for how to ‘do’ curriculum planning here at uni is constructive alignment. Anyone else having to use this term?

Basically, constructive alignment is what you do when you make sure your assessment tasks match your learning objectives, and that your lesson materials feed into this productively. (OK, so I slipped the word ‘productively’ in just there…can you tell I’m living in Queensland? Productive pedagogies, anyone?)

So, the first two phases of my action research have been all about getting the assessments to work for me and my unit, English Curriculum Studies 1. I inherited a bunch of learning objectives when I took on coordination of this unit, but in the end I found that the assessment tasks weren’t engaging students in the ways I knew could happen. In the ways I was sure could happen, anyway. All of the assessment pieces have now been modified or replaced (not allowed to change the learning objectives) and things are aligning much more constructively…

The last piece in the puzzle that I was really hoping to nut out in this third cycle is the establishment of threshold concepts for this unit.

A ‘threshold concept’ is the kind of concept that, once learned, cannot be unlearned.  Once we grasp a piece of threshold knowledge, we pass over a barrier into new territory, where everything is seen anew with different eyes.

In the (bazillion) Powerpoint presentations I sat through last year as a new academic, I picked up the importance of using a few well-chosen threshold concepts to drive a unit of work.  For teachers like me that prefer to use project-based and inquiry-based learning approaches, having a set of threshold concepts in mind that you want students to ‘get’ by the end of the experience looks to be an excellent anchor for lesson planning.  Although these concepts are related to the official learning objectives of the unit, they do serve a different kind of function…and I really want to settle on what mine are!

Until this week I was still struggling to come up with suitable concepts.

But now, I struggle NO MORE!

I have been working on a summary video for students to watch at the half-way point in semester, while I am away at a conference.  In the video I want to recap the main points learned from weeks 1-5 of the unit.  The process of trying to identify what the ‘big ideas’ were amongst all of the super important stuff we learned wasn’t easy.  But the process of having to present the ideas to my students (not just to my academic review panel at the end of this year…!) has really helped.

Which I guess just goes to show that even teachers need an authentic audience for their work.

Trying to keep the video short (under 5 minutes) also forced my hand – left to my own devices, I’m sure I could find plenty of threshold concepts, but you only need a few. The wording of what I’ve chosen isn’t quite right yet, but these are the six big points I have chosen:

  1. Your personal teacher identity is unique and reflects your personal experience, but will inevitably draw on many established philosophies and practices.
  2. In ‘English’ we study: semiotics, text and context.
  3. Language codes and conventions are socially constructed.
  4. Verbal/linguistic language is just one semiotic ‘code'; we also learn/teach audio, visual, spatial and gestural language.
  5. Literacy involves more than code breaking – we also make meaning, use texts functionally, and critique texts.
  6. Multiliteracies pedagogies are currently favoured in English curriculum theory.

I suspect this is still too many for 6 weeks, but there you go.  We’ll see.  Once I’ve finished the video I’ll post it up here on the blog. I still have to add the narration, but most of the images are in. I’m using Movie Maker and Audacity as my tools of the trade…I hope the students have time to watch the bloody thing! But even if they don’t, I’m glad I went through this process and am happy that I’ve found some threshold concepts to settle on, for now. And, with any luck, a shiny new resource at the end I can be proud of. Fingers crossed!

, , , , , ,

1 Comment

Guest Post: ‘I Have A Dream that the HSC Will End’

  • Post to celebrate completion of my PhD: CHECK.
  • Post with an update on my upcoming conference papers: CHECK.

So…where to next?

As fate had it, this decision was made for me, with the arrival of a piece of student writing in my inbox.

The author of the piece is a recently graduated HSC student, one whom I had the pleasure of teaching year 8 English, and coaching for debating :)  This is him counting down the days until the end of his exams:

I invite you to read his work (below), which he has given me permission to reproduce (along with his picture) in this post.  Oriniginally published as a Facebook post on October 28th, it is a re-writing of Martin Luther King Jr.’s famous I Have a Dream speech, which has been adapted to make a satirical commentary on the HSC.  It comes with a mild language warning (c’mon; it’s satire!), and is a brilliant example of a ‘textual intervention’.

I’m very proud to feature it here  as my first ‘guest post’!:

*****

I Have A Dream that the HSC Will End

By B. Wylie

I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our state.

Two score years ago, an a*shole bureaucrat, in whose symbolic shadow we stand today, created the Higher School Certificate. This momentous decree came as a great source of pain and suffering to millions of NSW students who were about to be seared in the flames of withering injustice. It came as a sorrowful dusk which signalled the beginning of their long night of academic captivity.

But fourty four years later, the student still is not free. Fourty four years later, the life of the student is still sadly crippled by the manacles of standardised testing and the chains of rankings. Fourty four years later, the student lives on a lonely island of studying in the midst of a vast ocean of facebook updates. Fourty four years later, the student is still languished in the rooms of NSW high schools and finds himself an exile in his own class. And so we’ve come here today to dramatise a shameful condition.

In a sense we’ve come to this facebook note to cash a check. Read the rest of this entry »

, , , ,

7 Comments

Follow

Get every new post delivered to your Inbox.

Join 182 other followers