In search of the holy grail of assessment

A perfect storm of converging ideas brings this post into being.

A recent IBO program on "Affective Skills and Mindful Living" has got me thinking more about the skills students need for success post school. This reminded me of the role of Capabilities in the Australian curriculum which in turn took me to an article by Richard Bates where he writes:

The central assessment issue for educational institutions has now become that of how teachers and learners are to devise ways of testing validity claims – of testing the validity of information and knowledge claims that are new to both. This is by no means a simple issue, but contemporary circumstances force the issue to the centre of the curriculum and pedagogy of educational institutions. The open curriculum and an autonomous pedagogy require tests for truth and utility that are centred around individual and social purpose.  

(Bates: 2012, Is global citizenship possible and can international schools provide it? p.272 quoting an earlier article he wrote in 2008)

The challenge is complex, but the essence of it, as I see it, is to understand how we prepare students for a future where they can write their stories rather than ours.

One major part of what we do in education is transfer the understandings and opportunities of one generation to the next. Paraphrasing Schultz (1.) via Bates, this "neo-liberal" approach to teaching and learning focuses on the content of curricula assuming that the role of education is to equip students with the key knowledge they will use to replicate and acquire positional advantage in future societies. Once that key knowledge has been identified, benchmarked and disseminated to schools and teachers through documents like the IB curriculum the Australian Curriculum or UWCSEA's Learning

Program, teachers know what to teach, students know what to learn and examiners know what to assess. Those who learn it best can get the highest grades to take them to the most prestigious universities and be first in line to be the next generation of leaders. The best schools become those that get the best grades and provide parents and students with access to a future with economic and personal security.

The problem with an education system built on these neo-liberal lines is that it runs the risk of reproducing both the strengths and the failures of the past and of creating students who know what to think but not necessarily how - students whose education has given them knowledge about one way of understanding the world but not the skills to find other ways. This kind of education does little to address the sorts of concerns that  Mikhail Gorbachev raised in 2012 in this letter to the UWC community:

In today’s world, old threats to peace are persisting and new ones are emerging.The current economic crisis, the crisis of international relations and the threat of a new arms race testify to the fact that the twenty years after the end of the Cold War have been largely wasted instead of being used to build a more secure and just world order. The economy of many countries is in deep crisis. One of the causes of this crisis is the model that has defined global development for the past few decades, a model based on seeking super-profits and overconsumption, on social and environmental irresponsibility, making the human being merely a cog in an economic machine.

A neo-liberal curriculum does little to prepare students to transform the world and instead is more likely to solidify the processes that have created the problems in the first place.

...oOo...

My description of the IB, the Australian Curriculum and UWCSEA's Learning Program does none of the three curricula justice, of course. All three curricula have very deliberately built into them structures to teach students how to think and spaces for students to develop the autonomy they need if they are to be the creative leaders of the future.

In the IB these structures are most specifically represented by the "Approaches to Learning" (ATLs), in the Australian Curriculum by the Capabilities and in the UWCSEA Learning Program by the "Skills and Qualities." Following further Schultz's formulation of different schemata for understanding education, it is the existence of these elements of the respective curricula that might most represent the possibility of a "transformationalist" rather than a "neo-liberal" approach to education. These are the the key elements that underpin a more 'autonomous pedagogy'  which has the potential to empower the next generation to creatively engage with the world and write their own stories rather than re-writing ours.

Of these three curriculum structures, it is the Capabilities that have been around for the longest. Their earliest versions go back nearly 25 years to the 1989 "Hobart Declaration" and became more fully articulated in the 2008 "Melbourne Declaration on Educational Goals for Young Australians." The two key goals of the Melbourne Declaration are:

Goal 1: Australian schooling promotes equity and excellence 

Goal 2: All young Australians become:

– Successful learners
– Confident and creative individuals
– Active and informed citizens 

And the second goal is broken down into 24 specific descriptors. Here are three examples:

Successful Learners:

• are creative, innovative and resourceful, and are able to solve problems in ways that draw upon a range of learning areas and disciplines

Confident and creative individuals:

• develop personal values and  attributes such as honesty, resilience, empathy and respect for others 

Active and informed citizens:

• act with moral and ethical integrity

These are wonderful goals and in their modern form as "Capabilities" are surely central to shaping a population ready to address the problems that Mikhail Gorbachev reminds us are so pressing.

The challenge, however, is in giving the Capabilities the space they deserve in the curriculum. For nearly 25 years leaders in business, education and politics in Australia have been saying that we need to be teaching the skills students need to be the autonomous leaders of the future. Speaking from my own experience at the chalk-face through that period of time, nothing much has changed. And I am not sure that the Capabilities, as they are being articulated through the Australian Curriculum will have much impact either.

The problem, as I see it, is in the monolithic shadow of Year 12 exams. Skills can't be assessed in isolation and if something can't be assessed, it has a hard time surviving in a school. The academic subjects with their Neo-liberal capacity to support the processes of sifting and sorting through the Year 12 exams, dominate the pages of the Australian Curriculum; it is through the lens of the subjects that the Capabilities are being articulated. The Capability of "Ethical Understanding", for example, finds it's articulation in English as:

Students develop ethical understanding as they study the issues and dilemmas present in a range of texts and explore how ethical principles affect the behaviour and judgment of characters and those involved in issues and events. Students apply the skills of reasoning, empathy and imagination, consider and make judgments about actions and motives, and speculate on how life experiences affect and influence people’s decision making and whether various positions held are reasonable.

This kind of articulation adds very little to what English teachers have always done: we look carefully at ethical ideas in texts, discuss them, and students write about them in exams. The key problem is "write about them in exams". As long as the summative objective for assessment of skills is an exam, students and teachers remain trapped within a system which, by definition, requires normative judgements and limits the capacity of students to imaginatively engage with their own stories. In the example of English, this is quite literally the case; English exams are structured around students writing about the stories of others, it is beyond the imaginative scope of an exam system to assess students who are writing their own stories.


So what is the answer? It is not, in my opinion, to do away with Year 12 exams or to try to break down the walls that hold up the Subject edifices. We need the stories of the past in our future and we need students to know how to work with them. What we also need is students skilled and confident enough to write stories of their own when these stories are needed. The credibility and reliability of Year 12 exams are a very important pillar of stability in our cultures - both in regard to their reproduction of knowledge and in the stability they provide as a mechanism for sorting and selecting. Articulation of skills like "ethical behaviour" as a Capability - or ATL or "Skill and Quality" - through subject knowledge can do no harm, but as long as we are thinking only of high-stakes exams, I can't see the skills being given the opportunity to do a whole lot of good.

The answer, to my way of thinking, is in the other places and spaces in our curricula where we can assess differently.

This puts a very clear responsibility on the k-10 curriculum. And once again, my description of the Australian Curriculum (and tarred with the same brush the IB and the UWCSEA Learning Program) does it no justice. There are no exams required in the Australian Curriculum prior to Year 11/12. The long shadow of Year 12 certainly has an influence over how teachers and students think in the lower years, but there is no policy requirement to assess with exams.

The question this begs is how should students be assessed K - 10? What does an assessment system that 'tests for truth and utility' in forms 'centred around individual and social purpose' look like? If my analysis is correct, then clearly not like an exam where 'truth and utility' are centred around the power structures of a familiar elite and the normalising tendencies of existing knowledge.

This is too big a question to explore in any detail in this post but it's one I need to return to; my purpose in this post was not to provide a comprehensive answer but to ask what I hope is a compelling question. That said, here are a few preliminary thoughts towards some kinds of assessment which might better support student autonomy and skill development in line with the Capabilities/ATLs/Skills and Qualities:

• The process of setting goals and teacher and student evaluation against those goals puts students more at the centre of their learning. 

• Spending more time outside classrooms exploring how to use their skills in and on the world would seem like another potential opportunity.

• I think my Outdoor Ed colleagues would be mumbling under their breath that it is about time the rest of us caught up - they've been doing this for generations.

• Our Middle School Writing units all have built into them this question to the student:

How are you going to publish your writing? 

In other words, who or how will your writing influence, change, inspire, entertain, engage, amuse, enrage, provoke or prod? In engaging with students around this question, we are engaging with their skills for understanding their autonomy and social purpose. Conversations around who you want to write for and why can become conversations about being "principled" and "self-aware" as students grapple to build their skills for shaping the world. I prepare them for exams by teaching students to shape language, I prepare them for life by supporting them to learn how to shape the world.

At the moment this important question about publishing gets only a small space in my planning. What I have persuaded myself in this post is that it must get more.

• Student self assessment of learning through structures like "portfolios" of writing seem very important. A carefully considered "student lead portfolio assessment" process where students take the lead in demonstrating learning to parents has many of the elements that might support student autonomy and skill development - particularly if the Capabilities/ATLs/Skills and Qualities can be a structuring element.

• The IBO, in their wisdom, have a range of structures that develop student autonomy and encourage connection with authentic audiences. The Extended Essay CAS and TOK requirements in the IBDP are one mechanism as are the Exhibition in the PYP and Personal and Community projects in MYP.

I see the K - 10 curriculum as having a responsibility to balance the scales. Through necessity, subject content weighs heavy in the final years and skills get less priority in the curriculum. In the Middle School and the Primary School, we need to redress this balance and put more weight on the ability of students to demonstrate skills through autonomous and self-directed learning where reading the stories of others takes a bit less of the focus and writing stories of their own takes more.

1. Schultz L (2007) Educating for global citizenship. The Alberta Journal of Educational Research 5(3): 248–258.

Curriculum as Dialectic rather than Linear narrative.

UWCSEA is doing interesting curriculum work at the moment. In an attempt to clearly articulate our beliefs about learning and our understandings about how best to address the school's mission statement we are rewriting our curriculum.

At a purely pragmatic level, this might seem foolish. The thousands of hours spent exploring other curricula and negotiating our own understandings could be avoided were we simply to choose a well-established program from outside the school. ACARA's new Australian curriculum, for example, is based on years of academic discussion overlayed with extensive professional consultation and a rigorous validation and review process. What ACARA (or Common Core in the USA or the UK National Curriculum or any of a large number of other national or international curricula) provide to schools is a carefully considered package for teaching and learning. These curricula are valid and reliable and provide a level of credibility in the eyes of the community that allows teachers to get on with the practice of teaching knowing that they have easy answers to questions like "why do you teach that?" Answer: "because it is in the curriculum."

What I am discovering through being part of the UWCSEA Curriculum Articulation process, is that "because it's in the curriculum" is not a good enough answer. If you teach something because someone else told you to, then you are missing out on one of the most interesting, rewarding and, I would like to argue, vital pieces of the learning jigsaw.

The curriculum we are articulating at UWCSEA will not be as well-considered as the Australian Curriculum or Common Core or the UK National Curriculum. It can't be. As I sit with my colleagues exploring and discussing what learning looks like in the English curriculum (the area where I have been most focussed), I am very aware that our discussions are compromises and approximations. We borrow and build on other curricula and adapt them to our own context but we are constantly reaching points where we say "I'm not sure, this looks like the best answer at the moment and we will need to see how it works with students and make adaptations as we review the curriculum in its implementation." There are only 6 or 8 of us sitting around the table when we nut out the English curriculum and we lack the massive resources available to governments as they do similar work. But we have something very special available to us that governments do not: we are both designing and implementing what we have designed. Our answers to the question "why do you teach that?" will never be "because it's in the curriculum."

In the end what I think matters most is not so much the curriculum we are building as it is the relationship we will have to it. Building a curriculum requires a demanding re-assessment of the purpose and structure of each of its constituent disciplines. Before even this happens, it requires a demanding assessment of the very purpose and structure of teaching and this in turn requires a thoughtful reassessment of the purpose and structure of the school. In each of these processes, the relationship between the curriculum and the school community has changed. Each re-engagement with purpose and structure has breathed a little more life into our curriculum and moved it a little further from being a linear document that instructs, towards becoming a dialectic that engages.

Building Agency

"Thought bubbles"
 Some rights reserved by Michael Taggart Photography

We know that it is difficult to disentangle the relationship between structure and agency. The way the structures of society and the agency of individuals build on each other is an ongoing arena of debate in Sociology. And whilst there is significant disagreement about what the relationship between these elements might be, a common theme amongst theorists from Max Weber to Anthony Giddens is that the structures of society serve to limit the agency of some individuals more than others. Crucially, education plays a particular role in this process; the sociologist Basil Bernstein argued that the rules and "codes" of education which are implicit in curriculum and pedagogy often serve to close rather than open doors into an individual's future.

When children fail at school, drop out, repeat, they are likely to be positioned in a factual world tied to simple operations, where knowledge is impermeable. The successful have access to the general principle, and some of these – a small number who are going to produce the discourse – will become aware that the mystery of discourse is not order, but disorder, incoherence, the possibility of the unthinkable. But the long socialization into the pedagogic code can remove the danger of the unthinkable, and of alternative realities. (Bernstein, 1996, p.26)

In the context of UWCSEA's curriculum articulation, what speaks powerfully to me in this formulation is the dichotomy Bernstein describes between an understanding of knowledge as either impermeable or permeable - as a vessel in which the agency of the individual remains trapped or as a colander through which agency can filter and reform.

Too often education reform focuses only on the detail of curricula rather than recognising the important work of deliberately and strategically nurturing the agency of those who will be bringing curriculum to life in the classroom. Those of us who have been given the rare opportunity to reflect on the structure of our disciplines through the Curriculum Articulation process go into our classrooms armed with the most important of all understandings: we know what it's like to wrestle with the structures of knowledge and we can model this to our students. Critically, we know that the foundations of our knowledge are not fixed and nor are they easily categorised and labelled. Our knowledge is permeable and could be described differently. We have less certainty and more awe and this is what I think our students need if they are to take responsibility for achieving the UWCSEA Educational Goal to "shape a better world."

My students and I are privileged to have behind us the courage of a community which is prepared to take the brave and dangerous step into a world of the 'unthinkable, and of alternative realities.'

Cultural Foundations of Learning

I've just been reading a fascinating book recommended to me by our librarian: Cultural Foundations of Learning: East and West (2012) by Jin Li, Associate Professor of Education and Human Development at Brown University.


The book really stirred me up (as a good book should); below is the email I sent to my librarian. I feel very pretentious making such broad statements in response to such a well-researched and argued book, but I also feel my response is an important one. In the school where I teach, the need to account appropriately for differing cultural perspectives on learning is incredibly important, not just because we want to teach our students well, but also because the ethos of the school is grounded in a belief that we must respect diverse cultures and explore what it means to make a better world for all cultures.


Here are my thoughts:

I found the ideas and arguments fascinating and - for me as a Western reader - very useful as I learn more about teaching students from an Eastern cultural background.

The problem for me is that the book grounds its explanations entirely in ancient philosophy and has nothing to say about modern sociology. The influences on modern Western education stop, according to Jin Li, with Kant. There is no discussion of Max Weber and the idea of the Protestant Work Ethic, for example, and little discussion of the impact of the 200 years since the Industrial Revolution. The general argument about a difference between East and West in education is convincing but grounding the causes predominantly in ancient philosophy leads to the potential to miss the impact of industrialisation on education. In the West, industrialisation has been a 200 year process which has had a dramatic impact on society, family and education (which has been famously slow to catch up). In China, this same process has happened largely in one life-time. Families and culture are different in the East and West as much because of a variation in the modern history of the countries (including the repression of industrialisation in the East as a result of Western colonisation). I would contend that if you compared education in the East and West 200 years ago, the similarities would be much more striking (even given that Jin Li's differences would still be noticeable and important). 

Why I think my framing of the differences is important is that Jin Li's account suggests that cultural differences are central, axiomatic and immutable. If, as I am suggesting, the differences are as much grounded in a differing rate of acceleration through the processes of transition from agrarian to post-industrial economies, then we would expect that education practices in Eastern society are likely to change. As teachers, supporting that change isn't disrespectful to the cultural traditions of the East, it is important to preparing all our students for the world of the future.

Cheers,
Ian

Community = Communication

I’ve been reading a lot about the good and the bad of blogging this week.

Jeff Plaman shared these two articles from The Atlantic: Why American students can’t write and How Self-Expression Damaged my Students. Both articles present a general position about the dangers of a learning environment in which there is too much freedom for students and too little direct instruction from teachers.

These articles are in contrast with Jeff’s own writing about our digital identity and that of my colleagues Paula Guinto and Jabiz Raisdana. If I can grossly simplify the collective position of Jeff, Paula and Jabiz, I think it is that blogs provide a space for students to explore and develop their sense of themselves as writers and that a certain amount of “freedom” is absolutely necessary for this to occur.

Central to this discussion is the concept of “freedom”. For Peg Tyre and Robert Pondiscio, the two writers from The Atlantic, freedom seems to represent an abdication of responsibility by teachers. Pondiacio argues that giving students freedom to explore their identity as writers through the “Writers Workshop” model is to ignore a more important responsibility we have as teachers: direct instruction.

…at too many schools, it's more important for a child to unburden her 10-year-old soul writing personal essays about the day she went to the hospital, dropped an ice cream cone on a sidewalk, or shopped for new sneakers. It's more important to write a "personal response" to literature than engage with the content. This is supposed to be "authentic" writing. There is nothing inherently inauthentic about research papers and English essays.

[…]

…at present, we expend too much effort trying to get children to "live the writerly life" and "develop a lifelong love of reading."

You're not going to get to any of those laudable goals without knowledge, skills, and competence. For every kid who has had his creative spark dimmed by "paint-by-numbers" writing instruction, there are almost certainly 10 more who never developed that creative spark because they grew up believing they can't write and never learned to adequately express themselves.

Whether Pondiacio’s depiction of the “Writer’s Workshop” method is accurate or not (and my own recent training in Writers Workshop would suggest not), he nonetheless represents a concern about the failure to teach “basics” which rings many chords. It takes little time surfing the net to discover waves of disgruntled writers concerned about the loss of basic skills in the education system.

As I’ve written elsewhere, this concern seems to reflect something much more pervasive than just the teaching of writing. A general concern about a lack of disciplined teaching in schools is pervasive in the popular press despite a lack of evidence to this effect. In western countries, the massive increase in participation in post-primary education over the last 50 years has lead to a concomitant increase in literacy. The population of today is without doubt more literate than that of yesterday. What this has also meant is that instead of a small group of the educated elite defining the lingua franca, there are increasingly diverse groups contributing their ideas and their voices to the discourses of power.

It seems important to me to remember that the whole project to fix language into one definable form is not only political but also very recent. The project to “fix the English language” which Samuel Johnson began 250 years ago in the writing of his dictionary, reached its apotheosis in the creation of The Oxford English Dictionary in the mid-19^th Century, a project which was not finished until early in the 20^th Century. We think of English language and grammar as being largely fixed and unchanging but they never have been and the idea that they might be fixed is essentially a modern one. Shakespeare did not have a standard spelling, grammar or lexicon and, arguably, could not have written Hamlet if he did. A freedom to play with language is at the heart of most great writing and particularly poetry.

My argument is not that I think language should be loose or that any form of communication should be fine, but rather that it is important to understand that all decisions about which words, grammar and spelling are “right” are conventions and that these conventions should and must evolve. We do need to understand the conventions of our day, but we also need to stop and ask ourselves why we want to communicate and it is this question that I think is missing from the “back to basics” agenda.

We communicate to create communities.

Language is, at its most basic level, communicative and our identities are the consequence of this communication. What I find unsettling about articles such as the two which began this post is that they almost see writing as combative; the desired outcome is to conquer, not to communicate and successful writing is that which is “better” than others. The idea of a polished prose based in a view of potential perfection is anathema to communication because communication is a negotiated medium in which meaning cannot be static.

One’s prose is important, but far more important is the connection between interlocutors and the possibility of building and evolving a better understanding of self and others. Respectful communication is first and foremost concerned with forming connections to the interlocutor – not with evaluating the status of their prose.

Which brings me back to what I think is a good blog. First and foremost it is one which communicates. Language can facilitate communication in a range of guises. At times it works best when it is well-dressed in black tie or ball gown; at other times board-shorts and “T” shirt fit better. It is absolutely the responsibility of the teacher to bring students to an understanding of what clothing will gain them easiest entrée to which venue but a far more pressing need in any society is to teach them to look for the person beneath the veneer and to truly communicate.

What I find truly inspiring in the work of my colleagues is that they are giving students the space to find themselves and each other in their writing. Part of the mission of our school is to “make education a force to unite people, nations and cultures for peace and a sustainable future”. Such an important task must begin with communication.