On different ways of solving the very same problem

.
In this spring's bachelor's thesis course, I gave a seminar in the beginning of the term where we looked closer at two theses from last year's crop in order to analyze their strengths and weaknesses. In one of these two theses, the students in question took on the task of creating a system for other students to review eligible KTH courses, i.e. courses that they can choose to take or not to take. Part of the inspiration for their system came from "Rate my professors" and from other systems that are used at Swedish universities to evaluate courses online.

Because there are different "stakeholders" and I'm a teacher rather than a student, I have interests that are systematically different from students' interests. I'd for example like not to get negative reviews on my courses - even if (some) students don't like them. I'd also like to not encourage students to specifically and instrumentally search for "easy credits" ("this course is easy, but that one is difficult - stay away from it"). And so on.

As a consequence and from a teacher's perspective I can thus see lots of "problems" ("challenges", "complications") with the proposed system. What exactly is being graded, the course or the teacher? If the teacher is being graded, does that pose a problem in terms of integrity? Does that improve courses - on behalf of all stakeholders; university, teachers, students (i.e. is the fact that a teacher/course "doesn't work" enough information to help said teacher improve the course)? If students can rate teachers, should teachers then be allowed to rate students (rather than just their knowledge)? If it is the course that is being graded, what happens with the ratings if a course is taken oven by a new teacher, or if the old teacher changes the course? What if teachers and departments "protect" themselves against bad reviews by systematically discontinuing courses but creating "similar" (i.e. the same) courses each year (so as to dodge bad reviews)? Who exactly can read the reviews and who exactly can write them? Can everyone read and write, or only KTH students who have actually taken (passed?) a course? How then should students' identities be handled? Should the system differ between students who got a good grade (more ambitious? more credible?) and those who got a crappy grade on the course (slackers?)? I presume there is a correlation between liking a course and getting a good grade, but we also know that people who are discontent with something are more likely to answer surveys and evaluations (to "get back at") than people who are satisfied (who instead "move on"). Finally, here's a great question that the students can empathize with; what if a prospective employer looks at (all) the reviews written by a particular student and draws the conclusion that this is someone who whines too much and is not suitable to hire or even call to an interview? Would students' be aware of such possible uses (misuses) of the system and adapt to them in advance? By the way, what if the system is "hacked" and I write bad, whiny reviews and sign them with your name? And so on. The concept is interesting to explore, but also raises a lot of questions as can be seen.


But this is not a blog post about that specific proposed system. It's not about design decision and intended or realized functionality and interface issues (some of the problems raised above where actually answered in the thesis). I'm just warming up here with some of the questions that such a system raises and that I thought about when I read the proposal and then later the finished thesis (I was the examiner). However, apart from this particular (proposed) system, the underlaying question is of course valid; how can students be supported in making informed choices about what courses to read - instead of finding out that a particular course wasn't their cup of tea some weeks after the course has started ("sunk costs" and all that)?

What struck me at the seminar in the beginning of the term was that this is the exact same question that a project group tried to answer last autumn in another course of mine - but they came up with a radically different solution. I wrote a blog post about their propsed solution to the same problem - Storycast - half a year ago, but this is a second look at the system and from a slightly different angle. Short summary:

Someone (students, teacher or some other "producer") interviews a teacher and edits the interview down to a short downloadable podcast. Instead of reading a boring, bland, official and perhaps outdated description about a course, why not listen to a podcast where the teacher talks about the topic (rather than about the course, the examination, prerequisites etc)? Stuff like why I as a teacher am interested in the topic(s) to be treated in the course, where my interest come from, some anecdotes or stories about the topic or my own forays into it, my favorite books or stuff that has happened in the area lately, why I as a teacher think this topic is fascinating, and, in short, why students should take the course. In other words information/a radio program (perhaps bordering on entertainment) that can give students a feel for both the teacher, the topic and perhaps the course itself. A customized "radio show" (podcast) that could be as lively and entertaining as the interviewer and the teacher can make it (like a studio TV program). Heaven is the limit. Instead of sifting through course descriptions, why not spend an hour listening to some professors present their courses and topics and themselves?

What is interesting here though is that this is a radically different approach to solve the very same problem; how can students make informed choices about what courses to choose. To be exhaustive, I guess that I should also mention a third and a fourth way to fix the same problem:

Another way is to make sure that course descriptions are not boring and bland. But such texts are by their very nature "official", and there are conflicts of interest involved. As a teacher I want to write a text that is "open" and flexible enough for me not to have to rewrite the text every year or every time I change something in the course. Taking into account that such texts have to written far in advance, I don't want to be bound by a text that is all too specific far in advance of having done the detailed planning for the course (which of course is made right up until the last moment). The course description, by being "official", is not to be tampered with right before or during the course and I naturally prefer to be restricted as little as possible. If I change my mind about something (perhaps even during the course), I don't want a restrictive, pesky, outdated text to stand in the way. All of this encourages me as a teacher to write non-binding, non-restricting descriptions that then turn out to become boring and bland. They tend to be "fluffy" and unspecific and I imagine it must (always, or often) be hard for students to get an understanding of the inner workings of courses through these texts.

I know that writing good, informative course descriptions is an ability that teachers can improve upon, but my argument here is that there are inherent forces pulling such a text in different directions, creating "tensions" or perhaps even "contradictions" as regards to the form and the content or such texts. In short, I think course descriptions will never (or seldom) be good enough for students to make informed choices.

Yet another way to fix the same problem is to encourage students to talk to each other so that students who have taken a course can tell other, younger students about it and whether and why the liked it (or not). We help facilitate such processes by mixing students from the first three years in our program-integrating course. Hopefully every students in our program knows at least a few older students (and perhaps their friends) and can learn informally about courses they can choose later on in their education from them.

This "solution" is interesting since it is the exact opposite of creating a computer system (or working on a radio/podcast "production"). You do nothing except help people get in touch with and talk to each other - but alas, anything can come out of such a process and nothing is guaranteed. You don't know for sure that information about courses will be disseminated "effectively" - but other things can on the other hand come out of such a process - like for example friendship... And really, what could be simpler? No need for expensive and time-consuming (technical) solutions with set-up costs and maintenance costs (in time and/or money). Except of course for the maintenance costs of setting up and running a program-integrating course...


To summarize, I've written about no less than four very different ways of solving the same problem. And there is most probably also a fifth and sixth way that I haven't thought of... It is important to realize that there will always be trade-offs between these different solutions in terms of effort and effects.

It's good that I got around to writing this blog post. I can already now imagine that I might direct students to this particular blog post later - to help them break out of the self-imposed illusion that the first impulse about how to solve a particular problem is the best (or the only) way it can be solved! "To a man with a hammer, everything looks like a nail". And to a student with programming skills, every solution will tend to look like a computer program. And to the student with radio production skills, every solution will tend to look like radio program. And so on.

Do you have yet another suggestion for how to solve the problem of informing students about courses and help them make informed choices?
.