Showing posts with label assessment. Show all posts
Showing posts with label assessment. Show all posts

Wednesday, 29 April 2015

Spinning a Yarn

In the 1980s, when I went to Melbourne University, the Old Arts building had only one renovated lecture theatre. In contrast to the oak panelling and uncomfortable chairs of the other spaces, the "Public Lecture Theatre" had tiered seating upholstered in the pastels of late '80s modernity. It was a place of flickering fluorescent lights and shimmering contrasts. Outside, the sandstone walls spoke of tradition and gravitas; inside, the sound-absorbing roof panels whispered invitations to the future.

Each week I sat in this shimmering space for my "Introduction to Philosophy" lecture. My lecturer was one of the best teachers I have ever had. Frustratingly I don't remember much about him, but I have vivid recollections of his teaching. Lectures would begin with him walking briskly to the lectern, removing his tweed cap and taking a piece of chalk from his pocket. The entire lecture would be structured as a conversation between "Chalky" and "Hatty". Hatty would pose a problem to which Chalky would respond - occasionally using one of the many blackboards to illustrate his point. I don't remember my teacher ever referring to notes nor do I remember him ever standing still; he paced constantly, eye's closed, holding in turn Chalky or Hatty as the dialogue progressed.
https://www.flickr.com/photos/oflinesandcolor
/2241615711/in/set-72157594251486125
 Some rights reserved

Two particular lectures have stayed deeply rooted in my memory. The first was an introduction to Plato's Theory of Forms. Plato, in responding to the philosophic challenge of how we might know the universal essence of a thing, developed one of the most famous theories in philosophy. Let me try to illustrate the problem and Plato's solution.

Hatty: This morning I drank coffee from my coffee cup. As I drank, I wondered about how I know that a cup is a cup. I know my particular coffee cup is a cup because I have drunk from it before and I recognise it as the cup I drank from yesterday. But if today my housemate accidentally broke this particular cup and bought a new one to replace it and left it on the table for me, how would I know that the new cup was a cup and not, for example, a plate? No two cups are exactly alike and the new cup might be a little shorter or taller or fatter or thinner. It would likely have slightly different characteristics to any other cup I had ever known, but I would still know it to be a cup. How could this be?

Chalky: Well Socrates would tell you that in your head you have a perfect image of "cupness" and when you see the new cup, you compare it to this image and note that it fits the image more nearly than it does that of your image of "plateness" or any other category you hold in your head. Socrates called these categories "Forms" and argued that they are deeply grounded in our minds. They are part of an ideal world which includes also categories like "Good" and "Evil".

In a very simplistic nutshell, that is Plato's Theory of Forms. Hatty would no doubt have a lot of questions to ask about the idea as have philosophers since.


Assessment - from a Platonic perspective

The Theory of Forms has something in common with the challenge we as teachers have in assessing students' writing. Like the "cup" on the table, we must decide whether a student essay is a good approximation of "essayness" or not. But like the cup bought by Hatty's housemate, every student essay will have properties that are unique; there will be combinations of words I have never seen before. So how, when I look at an essay, can I decide if it is a good approximation of the ideal?

A common answer teachers bring to this problem is the rubric. One view of a writing rubric is that it describes the characteristics of an ideal essay and acts as a template for judging individual student approximations of this ideal.

Immediately, of course, we hit a problem. Plato's forms exist only in the abstract (like the idea of a perfect circle) and a rubric that was even close to accurate would need to be so detailed as to be unworkable. Such a rubric might describe some of the elements of an ideal essay, but could never tell us how to combine these elements with originality or creativity and an essay without originality or creativity is surely not an ideal essay at all. In the end what rubrics for writing (such as those used for the GCSE) often end up saying is that an excellent essay demonstrates creativity and flare, which, as a detailed guide to teaching and assessment, isn't much use. In essence, all a rubric of this nature is telling a reader is that a good essay show's "good-essay-ness".

An alternative view

Fortunately Chalky and Hatty had more to say on the matter. Whilst the early lectures focussed on an introduction to Plato, some of the later ones address more modern responses to the same issues. A lecture about Wittgenstein's theory of Family Resemblances is vividly in my memory.

Hatty: Understanding how two objects are related is difficult. One of the problems I am finding with Plato's Theory of Forms is that it seems to require that the ideal forms are present in my head from birth and that doesn't seem to correspond with my understanding of the world. I know that I can learn new things and whilst I can agree that there seem to be some categories that might be in my head from the beginning, that doesn't seem to be the case for many of the things I think I know. Maybe the concept of a circle was always in my mind, but I don't think the concept of a Canadian was. The category "Canadian" seems to be something I have learned and I can even imagine that, had history unfolded differently, this category might never have existed. Plato certainly would not have had an category of "Canadian". 

Chalky: Indeed. The problem of how we can understand two related objects as being connected is very difficult. There is another way of understanding the problem developed by the 20th century philosopher Ludwig Wittgenstein


Imagine you have a family with 5 brothers. 

Brother one has blue eyes, long hair, a long nose and big ears. 

Brother two has blue eyes, long hair, a long nose but small ears.

Brother three has blue eyes, long hair but a short nose and small ears.

Brother four has blue eyes but short hair, a short nose and small ears.

Brother five has brown eyes, short hair, a short nose and small ears.

All five brothers belong to the same family but brothers one and five have no characteristics in common. What links them is not their individual characteristics but their relationship through shared characteristics with other members of the family. 

Hatty: Interesting. But how does this help to solve the problems of essential understandings that we found existed in Plato's theory? 

Chalky: Well what Wittgenstein is pointing out is that words and concepts don't have to have meaning in any essential way, but they can point to relationships or connections. What makes our 5 brothers members of the same family is not any particular quality they share, but the connections between them. If we think about the problem of recognising the cup that my housemate has bought, it is not any essential "cupness" that makes the new cup a cup, but some qualities which I recognise it as sharing with some other cups. 

Although this may not sound very profound there is a very important difference in recognising the problem from this perspective of resemblances or, as Wittgenstein described them on other occasions, as language games. "Cupness" becomes not an essence but a set of shared characteristics which I recognise because I have been learning this "game". The rules of "cupness" can change as my culture changes its conventions on what will be seen as a cup. The rules of "Canadianess" are certainly changing constantly both at a formal legal level and also at the more subtle level of cultural identity. 

Another metaphor that Wittgenstein used to explain the nature of concepts is that of a thread. In referring to the concept of "number" here's what he said:

Why do we call something a "number"? Well, perhaps because it has a direct relationship with several things that have hitherto been called number; and this can be said to give it an indirect relationship to other things we call the same name. And we extend our concept of number as in spinning a thread we twist fibre on fibre. And the strength of the thread does not reside in the fact that some one fibre runs through its whole length, but in the overlapping of many fibres. (source)

Hatty: So the insight we take from Wittgenstein is that words indicate relationships and we know concepts as being like a set of games, or connections or threads that we are constantly weaving and elaborating and extending. 

Chalky: Exactly.

Some implications for assessment.

As we develop Standards and Benchmarks for our curriculum at UWCSEA and prepare them for use in our teaching and assessment, how should we understand their nature and purpose?


No doubt my thinking will evolve as the yarn spins, but, for the moment, here's what Chalky has lead me to think:

Rubrics are useful because they help to organise and prioritise the fibres that we will focus on in our teaching and students will use to make the threads of their learning. But the threads themselves are beyond the scope of a rubric, and, in the end, it is the threads that matter most, not the fibres.

To put this another way: our curriculum helps us to define specific parts of an essay that are useful to learn because this learning points to characteristics of language games which help students understand the cultural territory of "the essay". But a curriculum (or a rubric) can never provide the kind of complexity needed to understand how a specific essay fits within the category "essayness". This requires the enormous complexity of a well-trained, culturally complex, essay reader - a professional English teacher. The teacher who reads the essay and notices the original organisation of words can look to see how these words fit within the conventions of the game of "essayness" and make a decision about whether this particular essay is playing the game with flair or not.

This suggest two things to me about assessment in English:

1. The role and function of rubrics is limited. Powerful, yes, in focussing in on the crafted fibres of learning that expert teachers hone to point students towards the culturally vast and unstable spaces of understandings. But limited in that they are never an end in themselves.

2. Teachers need to trust their professional knowledge when they make judgements about attainment. In the end, it is the skill of the expert reader that we need to rely on when judgements are to be made about the way a particular essay has engaged in the playful cultural spaces of our language games.


Spinning a yarn


In Australia, my home, the process of storytelling is known vernacularly as "spinning a yarn". Bushmen sit drinking tea around campfires conjuring worlds from their words. For me, one of the great mysteries of the world is how this process works. How is it that a particular combination of words can come together in the hands of a gifted storyteller to build something strong and beautiful?

What Wittgenstein suggests is that spinning a yarn is not a skill we are born with but one we learn through our rich engagement in the complexity of our culture. I thank my wonderful lecturer and Chalky and Hatty for sharing this understanding.

At the most profound level, learning is a conversation; a thread we continue to spin.













Friday, 17 October 2014

In search of the holy grail of assessment



A perfect storm of converging ideas brings this post into being.

A recent IBO program on "Affective Skills and Mindful Living" has got me thinking more about the skills students need for success post school. This reminded me of the role of Capabilities in the Australian curriculum which in turn took me to an article by Richard Bates where he writes:

The central assessment issue for educational institutions has now become that of how teachers and learners are to devise ways of testing validity claims – of testing the validity of information and knowledge claims that are new to both. This is by no means a simple issue, but contemporary circumstances force the issue to the centre of the curriculum and pedagogy of educational institutions. The open curriculum and an autonomous pedagogy require tests for truth and utility that are centred around individual and social purpose.  
(Bates: 2012, Is global citizenship possible and can international schools provide it? p.272 quoting an earlier article he wrote in 2008)

The challenge is complex, but the essence of it, as I see it, is to understand how we prepare students for a future where they can write their stories rather than ours.

One major part of what we do in education is transfer the understandings and opportunities of one generation to the next. Paraphrasing Schultz (1.) via Bates, this "neo-liberal" approach to teaching and learning focuses on the content of curricula assuming that the role of education is to equip students with the key knowledge they will use to replicate and acquire positional advantage in future societies. Once that key knowledge has been identified, benchmarked and disseminated to schools and teachers through documents like the IB curriculum the Australian Curriculum or UWCSEA's Learning
Program, teachers know what to teach, students know what to learn and examiners know what to assess. Those who learn it best can get the highest grades to take them to the most prestigious universities and be first in line to be the next generation of leaders. The best schools become those that get the best grades and provide parents and students with access to a future with economic and personal security.

The problem with an education system built on these neo-liberal lines is that it runs the risk of reproducing both the strengths and the failures of the past and of creating students who know what to think but not necessarily how - students whose education has given them knowledge about one way of understanding the world but not the skills to find other ways. This kind of education does little to address the sorts of concerns that  Mikhail Gorbachev raised in 2012 in this letter to the UWC community:

In today’s world, old threats to peace are persisting and new ones are emerging.The current economic crisis, the crisis of international relations and the threat of a new arms race testify to the fact that the twenty years after the end of the Cold War have been largely wasted instead of being used to build a more secure and just world order. The economy of many countries is in deep crisis. One of the causes of this crisis is the model that has defined global development for the past few decades, a model based on seeking super-profits and overconsumption, on social and environmental irresponsibility, making the human being merely a cog in an economic machine.

A neo-liberal curriculum does little to prepare students to transform the world and instead is more likely to solidify the processes that have created the problems in the first place.


...oOo...


My description of the IB, the Australian Curriculum and UWCSEA's Learning Program does none of the three curricula justice, of course. All three curricula have very deliberately built into them structures to teach students how to think and spaces for students to develop the autonomy they need if they are to be the creative leaders of the future.

In the IB these structures are most specifically represented by the "Approaches to Learning" (ATLs), in the Australian Curriculum by the Capabilities and in the UWCSEA Learning Program by the "Skills and Qualities." Following further Schultz's formulation of different schemata for understanding education, it is the existence of these elements of the respective curricula that might most represent the possibility of a "transformationalist" rather than a "neo-liberal" approach to education. These are the the key elements that underpin a more 'autonomous pedagogy'  which has the potential to empower the next generation to creatively engage with the world and write their own stories rather than re-writing ours.

Of these three curriculum structures, it is the Capabilities that have been around for the longest. Their earliest versions go back nearly 25 years to the 1989 "Hobart Declaration" and became more fully articulated in the 2008 "Melbourne Declaration on Educational Goals for Young Australians." The two key goals of the Melbourne Declaration are:

Goal 1: Australian schooling promotes equity and excellence 
Goal 2: All young Australians become:
– Successful learners
– Confident and creative individuals
– Active and informed citizens 

And the second goal is broken down into 24 specific descriptors. Here are three examples:

Successful Learners:
  • are creative, innovative and resourceful, and are able to solve problems in ways that draw upon a range of learning areas and disciplines

Confident and creative individuals:
  • develop personal values and  attributes such as honesty, resilience, empathy and respect for others 


Active and informed citizens:
  • act with moral and ethical integrity

These are wonderful goals and in their modern form as "Capabilities" are surely central to shaping a population ready to address the problems that Mikhail Gorbachev reminds us are so pressing.

The challenge, however, is in giving the Capabilities the space they deserve in the curriculum. For nearly 25 years leaders in business, education and politics in Australia have been saying that we need to be teaching the skills students need to be the autonomous leaders of the future. Speaking from my own experience at the chalk-face through that period of time, nothing much has changed. And I am not sure that the Capabilities, as they are being articulated through the Australian Curriculum will have much impact either.

The problem, as I see it, is in the monolithic shadow of Year 12 exams. Skills can't be assessed in isolation and if something can't be assessed, it has a hard time surviving in a school. The academic subjects with their Neo-liberal capacity to support the processes of sifting and sorting through the Year 12 exams, dominate the pages of the Australian Curriculum; it is through the lens of the subjects that the Capabilities are being articulated. The Capability of "Ethical Understanding", for example, finds it's articulation in English as:

Students develop ethical understanding as they study the issues and dilemmas present in a range of texts and explore how ethical principles affect the behaviour and judgment of characters and those involved in issues and events. Students apply the skills of reasoning, empathy and imagination, consider and make judgments about actions and motives, and speculate on how life experiences affect and influence people’s decision making and whether various positions held are reasonable.

This kind of articulation adds very little to what English teachers have always done: we look carefully at ethical ideas in texts, discuss them, and students write about them in exams. The key problem is "write about them in exams". As long as the summative objective for assessment of skills is an exam, students and teachers remain trapped within a system which, by definition, requires normative judgements and limits the capacity of students to imaginatively engage with their own stories. In the example of English, this is quite literally the case; English exams are structured around students writing about the stories of others, it is beyond the imaginative scope of an exam system to assess students who are writing their own stories.


So what is the answer? It is not, in my opinion, to do away with Year 12 exams or to try to break down the walls that hold up the Subject edifices. We need the stories of the past in our future and we need students to know how to work with them. What we also need is students skilled and confident enough to write stories of their own when these stories are needed. The credibility and reliability of Year 12 exams are a very important pillar of stability in our cultures - both in regard to their reproduction of knowledge and in the stability they provide as a mechanism for sorting and selecting. Articulation of skills like "ethical behaviour" as a Capability - or ATL or "Skill and Quality" - through subject knowledge can do no harm, but as long as we are thinking only of high-stakes exams, I can't see the skills being given the opportunity to do a whole lot of good.

The answer, to my way of thinking, is in the other places and spaces in our curricula where we can assess differently.

This puts a very clear responsibility on the k-10 curriculum. And once again, my description of the Australian Curriculum (and tarred with the same brush the IB and the UWCSEA Learning Program) does it no justice. There are no exams required in the Australian Curriculum prior to Year 11/12. The long shadow of Year 12 certainly has an influence over how teachers and students think in the lower years, but there is no policy requirement to assess with exams.

The question this begs is how should students be assessed K - 10? What does an assessment system that 'tests for truth and utility' in forms 'centred around individual and social purpose' look like? If my analysis is correct, then clearly not like an exam where 'truth and utility' are centred around the power structures of a familiar elite and the normalising tendencies of existing knowledge.

This is too big a question to explore in any detail in this post but it's one I need to return to; my purpose in this post was not to provide a comprehensive answer but to ask what I hope is a compelling question. That said, here are a few preliminary thoughts towards some kinds of assessment which might better support student autonomy and skill development in line with the Capabilities/ATLs/Skills and Qualities:


  • The process of setting goals and teacher and student evaluation against those goals puts students more at the centre of their learning. 
  • Spending more time outside classrooms exploring how to use their skills in and on the world would seem like another potential opportunity.
  • I think my Outdoor Ed colleagues would be mumbling under their breath that it is about time the rest of us caught up - they've been doing this for generations.
  • Our Middle School Writing units all have built into them this question to the student:


How are you going to publish your writing

In other words, who or how will your writing influence, change, inspire, entertain, engage, amuse, enrage, provoke or prod? In engaging with students around this question, we are engaging with their skills for understanding their autonomy and social purpose. Conversations around who you want to write for and why can become conversations about being "principled" and "self-aware" as students grapple to build their skills for shaping the world. I prepare them for exams by teaching students to shape language, I prepare them for life by supporting them to learn how to shape the world.
At the moment this important question about publishing gets only a small space in my planning. What I have persuaded myself in this post is that it must get more.

  • Student self assessment of learning through structures like "portfolios" of writing seem very important. A carefully considered "student lead portfolio assessment" process where students take the lead in demonstrating learning to parents has many of the elements that might support student autonomy and skill development - particularly if the Capabilities/ATLs/Skills and Qualities can be a structuring element.
  • The IBO, in their wisdom, have a range of structures that develop student autonomy and encourage connection with authentic audiences. The Extended Essay CAS and TOK requirements in the IBDP are one mechanism as are the Exhibition in the PYP and Personal and Community projects in MYP.


I see the K - 10 curriculum as having a responsibility to balance the scales. Through necessity, subject content weighs heavy in the final years and skills get less priority in the curriculum. In the Middle School and the Primary School, we need to redress this balance and put more weight on the ability of students to demonstrate skills through autonomous and self-directed learning where reading the stories of others takes a bit less of the focus and writing stories of their own takes more.







1. Schultz L (2007) Educating for global citizenship. The Alberta Journal of Educational Research 5(3): 248–258.



Monday, 24 September 2012

Foxed by thought


Ms P and Mr R have got me thinking about my theory of good writing. My problem is, I don’t have one.

I understand the idea that there is bad writing and I certainly know the experience of reading writing which is good, but I’ve never really been comfortable with the idea of being able to define it. I am, in this instance, a relativist, which means that I don’t believe that there is any such thing as “perfect” writing nor that there is some way of making irrefutable judgements about the quality of writing.

I’m not sure if my students should be pleased by this declaration or not. On the one hand, I don’t claim to have the knowledge to make final judgements about their writing so they can feel more comfortable to explore and experiment without fear of too harsh a judgement. On the other hand, they don’t have a teacher who feels able to point them towards perfection with any degree of confidence.

Whilst I appreciate that writing is a discipline and that there are very definitely practices that help produce the final product effectively and efficiently, I’m not sure that the real poetry of writing can be achieved solely through discipline. There is something in the best writing which is beyond words: a paradoxical special ingredient that somehow evaporates if you try to take the lid off and define it in a classroom.  The best I think I’ve ever been able to do is put it in front of my students and hope for a little osmosis.

So, below, I offer a little bit of magic. This is Ted Hughes’ description of the act of writing poetry depicted, appropriately, through metaphor. My favourite line in this poem is the first line of the last stanza. A masterpiece of descriptive writing – but I can’t, exactly, tell you why.



I imagine this midnight moment's forest: 
Something else is alive 
Beside the clock's loneliness 
And this blank page where my fingers move. 

Through the window I see no star: 
Something more near 
Though deeper within darkness 
Is entering the loneliness: 

Cold, delicately as the dark snow 
A fox's nose touches twig, leaf; 
Two eyes serve a movement, that now 
And again now, and now, and now 

Sets neat prints into the snow 
Between trees, and warily a lame 
Shadow lags by stump and in hollow 
Of a body that is bold to come 

Across clearings, an eye, 
A widening deepening greenness, 
Brilliantly, concentratedly, 
Coming about its own business 

Till, with a sudden sharp hot stink of fox 
It enters the dark hole of the head. 
The window is starless still; the clock ticks, 
The page is printed.
Ted Hughes