How the media (unfortunately) works

.

This week I have unfortunately gotten an(other) insight into the shallowness of how media works. I am both the advisor of (a handful) of students and the examiner of (20 or so) students in a course where they write their bachelor's theses over the course of the spring term. I was taken aback earlier this week by a KTH press release that was based on an unfinished bachelor's thesis that as of now is being written by students at our department and in this course.

It seems the two students in question were quite hot on marketing their own personal selves, jumped the gun and went public with the results of their survey of (no less than 891) Swedish iPad users. Unfortunately, their thesis is not yet finished and will not be presented for another month. I (and everyone else involved in the course) thus have no insight whatsoever into the quality of the results - although the quantity might in fact be pretty impressive. But - and it is a big "but" - that quantity is of little worth if the quality is lacking in terms of exactly how the survey was conducted, what the results are, how these results are being interpreted and what conclusions are reached based on the results and the analysis. It is entirely possible that a substantial part of this has not even been written yet. I happen to know the students in question and they are very focused and goal-oriented, so I'm sure they haven't done a bad job - but the problem is that I (and everyone else) don't really know how good a job they have done - since their job isn't finished and noone has had the chance to have a look at it yet. And that is a problem.

The students themselves seem to think that since they have received almost 900 answers to their survey, their survey are three times "better" than an earlier survey that received less than a third as many answers. But the raw number of respondents of course has little impact on the quality of the results if the survey itself is flawed in any way. In this particular case (and as far as I understand), they have reached out to iPad- and Apple-lovers' Internet discussion spaces with requests for answering their survey. The people who hang around these fora are most certainly not representative of Swedish iPad-users, and it is to be expected that the persons who furthermore actually answer such a survey are the most enthusiastic of the enthusiasts.

It is therefore highly doubtful that (for example) Swedish iPad users in general spend more than 16 hours per week using their iPad, that almost 40% have the music-related application GarageBand on their iPads and that almost half of them own a Macbook Pro computer etc. Some people have furthermore misunderstood the fact that 95% of the respondents were men and now think that 95% of Swedish iPad owners/users are men...! So the question is to what extent a survey like this (and the ways in which the results have been disseminated) reveals new information about iPad users and iPad uses, and to what extent it spreads misinformation and infobabble about the same issues. It for sure says something about how quite a few people (891 persons) use their iPads, but it is still unclear what it says about (Swedish) iPad users in general. We are all eagerly awaiting the finished thesis, where these and other answers will hopefully be revealed...

To summarize, the survey is "only" part of a bachelor's thesis (e.g. a relatively limited investigation), it has not been reviewed by anyone yet - perhaps including the students' advisor (!). It is furthermore not yet finished, but is still being presented as representing "important" research results from KTH and originating from our department. Other teachers that I have talked to at our department don't really feel comfortable about this situation and think this course of events is unfortunate. I hope this won't affect the judgement of the students' finished thesis and affect their grades negatively when that time comes around some weeks from now (I'm not the one who will examine the thesis in this specific case)...

That might be as it is, but what really troubles me deeply is that the person responsible for press contacts at KTH made the judgement call of choosing to "make news" out of this as-of-yet unfinished bachelor's thesis. It seems like a remarkable misjudgement that could easily have been handled by just postponing the press release a month, until the thesis was finished and it had been reviewed. What instead happened was that a press release was written which soon found its way to the front page of KTH's website. It was then picked up by major Swedish morning newspapers and trade press and blown out of proportion by being uncritically (of course - who would expect something else nowadays) reported by so-called journalists. Anyone with some sense would have picked up on the fact that it is after all "only" a bachelor's thesis (limited work/effort) and not even finished or reviewed at that. This makes it difficult to understand of frame the reported results beyond parroting the press release and it is a dismal witness to the poor state of our media and to the triumph of surface over depth and speed over correctness in today's media landscape and in today's society.

The absurd part though is that after expressing my misgivings to the KTH press officer, it seems he took offense and came back with a request for me to "clarify" my actions. My actions in question consisted of sending an internal mail to my colleagues expressing my doubts (same as here, see above), requesting more information and questioning the use of university funds for hyping unfinished student "research" over actually using that same money to spend more time meeting and advising our students when they write their theses - and then forwarding this mail to him as a curtesy. The press officer also received an official rebuke (complaint) from the head of our research group, which was a much more diplomatic and toned down version of what I had previously written in my mail.

Now, the press officer obviously felt no need to clarify his own culpability in this process of "manufacturing" news of questionable (or at least uninspectionable) quality and "selling" an unfinished student research project as bona fide "KTH research", and he furthermore obviously did not even understand that his actions in this specific case might be questionable or objectionable, or at least counterproductive for KTH and its "brand". The world we live in can sometimes be truly absurd. His complaint was basically that we (me and the head of our research group) were making his job more difficult by "spreading rumors" about the fact that the emperor might not have any clothes...

The technical expressions for what has happened is:

- The students were fleecing the bear before it was shot.

- The press officer was making a hen out of a feather.

- The so-called journalists were uncritically parroting a press release.

- I expressed critique over the course of events and the actions of the press officer in this specific case.

The press officer felt that I questioned the quality of his work (in general) and he tried to bully me and squeeze an excuse (a "clarification") out of me. He did not and has not addressed the more important underlying question of whether his actions were perhaps a wee bit objectionable, misguided, or premature. I rebutted his request for an "clarification" (excuse) and squarely put the blame for this (in my opinion) "mess" where I believe it properly belongs - at his footsteps. I have not heard from him since although I have received some (mild) critique for being too frank about expressing my opinions.

Comment (April 2012): Here's a great blog post about the same (or a related) topic.

.