Keywords

At the same time, as supervision practice involves aspects such as relationship building and encouraging student participation and independence through a variety of tools, assessment is also an important part of the interaction in supervision. For example, when supervisors give praise or recognition to students, they are also making an assessment of what the students have achieved or the choices they have made. The same is true when supervisors provide negative critique or communicate in various ways that something the students have done or plan to do does not meet expectations and quality requirements. In this chapter we take a closer look at the supervisor’s role as an assessor and how this too can be understood in relation to student independence, academic literacies and scaffolding.

When Feedback Becomes Assessment

As we have discussed, supervisors have multiple roles and functions in relation to students. One of these roles has to do with assessment and evaluation, which is related to how a large part of the supervisors’ work involves giving feedback on what the students have achieved or thought (cf. Lundström 2016; Reid 1994; Rienecker et al. 2019).Footnote 1 Following the definition of van de Pol et al. (2010) and van de Pol (2012), giving feedback to students, that is providing students with information about their performance, may constitute an essential part of active scaffolding work in supervision.

Just as the teacher or supervisor needs to be aware of the students’ level of knowledge and previous experience in order to be able to assess where they stand, the students need to be made aware of the extent to which their performance so far is sufficient to continue their work. This is particularly true in relation to independence, in the sense that degree project writers need some kind of indication as to whether what they have achieved is good enough, in order to be able to continue to make the choices and decisions that will allow the work to move forward. During a supervision session, this kind of assessment occurs continuously and in many different ways, which was clearly evident in our material. Sometimes it is done through short responses such as ‘good’ or ‘ok’, sometimes through repeated questions that force students to rethink what they have done or to think about their work in new ways, and in other cases through shorter or longer statements and comments with negative or positive critique on different parts of the degree project work.

In other words, supervisors’ feedback and responses include positive assessments that encourage students to continue with their ideas and perspectives—such as the praise we discussed in Chap. 6—as well as negative assessments that cause students to step back, rethink and clarify. Supervisors’ assessments can thereby be described as formative in relation to examiners’ summative assessments. To a large extent, it is a forward-looking assessment, aimed at developing students’ academic literacies and academic writing by identifying what seems promising and functional, as opposed to what seems problematic and unfruitful. But there are also summative aspects to supervisors’ assessments, for example if, at the final stage of the degree project work, they inform students that the work is not up to scratch and advise them not to submit the degree project for examination, which we return to later in the chapter.

The formal organisation of the assessment of degree projects varies between disciplines, academic programmes, universities and higher education systems. In Sweden, supervisors are not supposed to be examiners of the degree projects they have supervised. However, there is often some kind of dialogue between supervisor and examiner, although this may be more or less formalised. In our material from Russian universities it was described how there was generally a formal examination committee or commission, at least at MA level but partly also at BA level, which in some cases could include representatives of stakeholders or representatives of the future professions (Zackariasson and Magnusson 2020). This made the assessment of the final product a collective decision involving several people (Focus Group Interview 5). Rienecker et al., who have written about the Danish context, discuss supervisors and examiners assessing degree projects together and the supervisor thereby having a dual role (Rienecker et al. 2019, 175f). Thus, the relationship between supervisors and examiners varies and can be relatively complex when it comes to the assessment of degree projects and whether the assessment is summative or formative.

When we discussed supervisors’ positive assessments in the supervision interaction in Chap. 6, it mainly concerned praise and various functions that praise can have in the supervision context. As mentioned above, different types of negative assessment are also important for how students progress in their work, and in this chapter we will look more closely at how these can be communicated to students, orally during supervision sessions, but also in writing, as written comments on texts and drafts, and in e-mails.

Supervisor Versus Examiner

At the beginning of the degree project process, feedback from supervisors to students inevitably tends to focus on what students are planning to do. Comments on project drafts and ideas are given on the basis that they are something preliminary, which should maintain a certain standard, but which can also be expected to change as work progresses. When supervisors begin to have access to longer texts, feedback often concerns both what students have achieved so far and how they intend to proceed. In other words, in these early stages of degree project work, supervisors’ assessments are to a high degree formative.

In the final phase of the degree project work, when time is running out and increasingly polished versions of the degree project manuscript are being discussed, there is less room for major modifications of the project and alternative choices. The supervisors’ response therefore tends to focus on the presented texts. The assessments that are made are still largely formative, in the sense that they aim to enable students to develop and improve their texts, but they can also be summative, for example when the supervisor, having read a final manuscript, judges that it is not of a sufficiently high quality and therefore advises the student(s) not to submit the degree project. This may happen earlier in the degree project process, but then such advice is often based on what the students have not done, due to lack of time or for other reasons. For example, the supervisor may claim that there is not enough empirical material on which to base the degree project, or that the students have not progressed far enough in the writing process to make it realistic to complete the degree project within the timeframe. At the final stage, it is primarily what the students have actually done that is assessed, as it is the final manuscript that is the focus of supervision.

In the focus group interviews that we conducted as part of our research project, the participating supervisors rarely talked explicitly about making assessments in the sense of giving positive or negative critique to students during supervision discussions. What they did talk about on several occasions was their experience of colleagues examining and grading degree projects they had supervised, and how students might experience differences in assessment between supervisors and examiners. As in the following example from a focus group at a Swedish university:

I have experienced very different tendencies for failing [degree projects]. Sometimes I think it’s a bit scary. We are still quite similar, we who do the assessments. We have the same education and often the same age, but we think very differently. And this is also a problem that students often say: “You think differently depending on where you come from. No two people think the same.” For example, if you have a supervisor and an examiner, or if you have had several examiners … There is a lot of talk about this and it also undermines legitimacy.

Focus Group Interview 2

That supervisors are well aware that it can be very important for students what grade they obtain, especially when it is a question of pass or fail, was also evident in the Russian focus group material:

Supervisor L: You know, usually we see discussions in cases when we decide whether to give a grade of 5 or 4 [good or excellent] according to our grading system, or 3 or 2 [satisfactory or failed]. Then there can be some heated debates. When it comes to something in between, colleagues usually come to a consensus. But when you can fail a student by giving him a 2, and that student won’t get a diploma and would have to come back next year, then the supervisor usually tries to protect his student, while the commission may disagree on the quality of the work.

Focus Group Interview 10

From an academic literacies perspective, it is not surprising that supervisors and examiners can make different assessments of the same text, especially if they come from different disciplines, considering how academic writing, and the expectations and assessments associated with it, from this perspective is assumed to differ between different local academic contexts (e.g. Lea & Street 2000; Lillis 2001, Shanahan and Shanahan 2012). At the same time, it is an aspect of degree project work that can be difficult for both supervisors and students to relate to, as the extracts above illustrate. From the students’ perspective, it may seem that the supervisor’s assessment is, or should be, equally important, even though the examiner or an examination board is primarily responsible for the final assessment. This attitude may also be shared by supervisors, not least from the perspective that, as noted above, they may in some cases make a kind of summative assessment of the final version of the work in order to determine whether the student should be discouraged from submitting the degree project—albeit in the knowledge that the examiners will then make their own assessment.

One aspect where supervisors in our material acknowledged difficulties was in relation to expectations or requirements of student independence. They could see challenges with encouraging students to make independent decisions and choices if examiners then had a relatively narrow view of how a degree project should be written and what choices are possible or desirable:

Then there is a kind of built-in contradiction in that we talk about independence at the same time as we want them to conform to certain folds. Some examiners are very, like, narrow and think it should be this way, and if it is not, then the student has to change it. While others might be a bit broader and accept other ways of … arguing or other kinds of theories. And maybe you should have a somewhat self-critical discussion among examiners about this. That if on the one hand we’re requiring independence, then at the same time we can’t demand that they do exactly what I think (laughs) they should do.

Focus Group Interview 3

While supervisors in our material commented on the different roles of supervisors and examiners, and the potential conflicts that could arise, it is worth noting that several of the supervisors in our study were or had been degree project examiners. Thus, they had experience not only of having the degree projects they had supervised assessed and marked by others but also of assessing and marking degree projects supervised by colleagues. In many cases, therefore, they had experience of being ‘on the other side’, as in the following example from a focus group interview at a Russian university:

Supervisor J: I have witnessed some conflicts as a member of [degree project] commissions. I think that in some cases, unfortunately, a supervisor does not see the difference between evaluation and attitude. The grade given has nothing to do with the student or the supervisor. It is an assessment of a specific piece of work. But some people take it as a personal insult and show resentment because they see a lower grade given by the reviewer as a sign that they do not value the supervisor or think badly of them. So we need to set ourselves a different task psychologically.

Focus Group Interview 6

In addition to perhaps providing a better understanding of the roles of supervisors and examiners and the conditions under which they work, there may be greater opportunities for collegial discussion involving both categories if the examiners are also involved as supervisors in a particular course. If, on the other hand, there is a specific group of examiners who are not the same as the supervisors, something we also encountered in the course of the project, there is a certain risk that there will be a gap between these two categories unless arenas for joint conversation and collegial discussion are created. One way of creating such arenas is to include dialogue between supervisors and examiners as a formalised element in the organisation of the degree project course. That this can be fruitful was described in one of the focus group interviews at a Swedish university where this had recently been done:

I feel that if you look at the course evaluations since we have introduced this [formalised dialogue], it appears much less in the course evaluations that: “Yes, that was really strange - how the examiner comes up with something completely different.” Because that was quite common… so that’s decreased significantly. But it still occurs. Because we have different traditions. We have different views. /…/ And we have different ways of … If you have a bad degree project, how do you save it? Then I suggest one way and you suggest another way and you suggest a third way, so to speak, and then the students may experience it as a conflict.

Focus Group Interview 4

Considering these potential issues, it may be helpful as a supervisor to actively discuss the role of supervisor versus examiner with supervised students, to raise their awareness of how these roles and categories relate to each other, and to give students a basic insight into the differences between formative and summative assessment and what this might mean for the supervision interaction. In addition, it can be valuable to reflect on how feedback and critique is given to students in relation to how student independence might be encouraged. This discussion is developed in the remainder of this chapter.

Critique as Formative Assessment

As we discussed at the beginning of this chapter, formative assessment in supervision can take many forms and is often described as giving critique. In Chap. 6 we discussed positive critique in the form of praise, and here we develop the discussion of negative critique and how this also has the potential to be a scaffolding tool that can help students to progress in their work on the degree project. In this context, negative critique means that supervisors make judgements about students’ written drafts or reasoning and descriptions of what they have done or plan to do. These assessments therefore concern things that, according to the supervisors, are missing or need to be developed.

How such negative critique is formulated may depend on whether it is given orally in a supervision session, in writing in commented drafts, or by e-mail. Critique that is formulated in writing by the supervisor, such as text comments on the students’ various drafts, tends in our collected material to be relatively short and specific, in contrast to negative feedback that is given orally in the interaction.Footnote 2 This can be seen in the following examples from an annotated student text draft from our research material:

  • What paradigm is this? And what was it like before?

  • It’s very old research, is it really still relevant? A lot has happened with preschool since then.

  • Shouldn’t this go under ‘definitions of learning’?

  • Don’t forget to include the word ‘groups of children’ when you write age-homogeneous and age-integrated. Otherwise it is unclear WHAT is homogeneous and integrated.

  • Loose. It becomes fragmentary here when the different parts are not clearly connected - leading from one to the other. It is not clear what kind of approach you are talking about.

Supervisor F, examples from annotated draft text

These examples include both critical questions, recommendations to do or say something in a certain way and statements indicating that the student has chosen a less fruitful approach. All of these forms of criticism were common in the comments that supervisors made on different drafts in our material.

When supervisors provided feedback via email, they commented on specific wording and details as well as more general aspects of the texts or the degree project work. Not infrequently, their response to e-mails was a direct reply to students’ questions about whether they were doing or thinking the right things, and sometimes the e-mail comments were linked to longer or shorter drafts of the text, which the supervisors also commented on and returned to the students. Negative criticism could be quite direct and straightforward, but it was often mixed with praise or encouragement, as in the following examples:

Excellent work, it’s starting to look good. Note that I am attaching a document with ‘track changes’ where I have corrected bits here and there. Think of my corrections as an indication of what you need to look at throughout the document. I have edited your abstract linguistically. Think of it as a help along the way, the abstract can still be worked on.

There are still an incredible number of spelling mistakes, small errors and inconsistencies in the document. You need to go through your brackets: Sometimes you write (Author, year, p. ), sometimes (Author year, p.), sometimes (Author: year:). There should be a long hyphen – between years and other numbers that are from – to. The headings are better, but still inconsistent. /…/ Watch your spaces and edit where they are doubled or missing.

I think the title is the best so far. See my rewritten suggestion.

There can’t be too much of your own voice in the analysis.

Go for it - all the way to the top.Footnote 3

Supervisor B, e-mail

In this case, the students specifically asked for feedback on their proposal for the title of the degree project and on how they should think about how visible they can or should be as authors in the text. However, the main focus of the response given is on the small details, which the supervisor had also noted in the annotated draft text.

Just like the conversations in the supervision sessions, the email responses can often be given according to the sandwich method (Henley and DiGennaro Reed 2015) that we discussed in Chap. 6, meaning that the response begins and ends with positive comments and encouragement, while the criticism is placed in the middle:

Thanks for the exciting read!

First of all, let’s just say that you have an interesting take on an exciting subject and you write very well - so that bodes well! It’s just a matter of not losing momentum and energy in the work – it’s a matter of fighting to make the points (and what’s good can always be even better)! I’m looking forward to reading more of the text analysis itself! The sample definitely left me wanting more!

But I have one overall point. I think you have a remarkably good understanding of many of the advanced theories you refer to, and you have convinced me of the good point of using them. But you have a challenge in using your excellent writing to be pedagogical to your readers and to explain many things that are taken for granted in this version of the text without being so to those who will read your text. Tell your readers (= your classmates)! Illustrate with more examples what the point of this or that is (complicated concepts, for example). /…/

Then you will probably be able to become more purposeful and economical in your presentation, thereby leading your readers on a straighter (but perhaps still somewhat wordier) path to the actual investigation. A win-win situation for your work!

I look forward with excitement to the next progress report! Signal when you think it’s time to meet and talk!

For now: go, go, go!

Supervisor H, e-mail

In both the above examples, the supervisors encourage student activity, thus emphasising that this is something they expect from the students. They address general aspects or recurring problems that students are encouraged to work on throughout the document. In the last example, the supervisor also makes it clear that it is the student who is expected to take the initiative for the next contact. From a scaffolding perspective, this can be seen as an example of how supervisors can gradually transfer responsibility for the degree project process to students over the course of the semester (cf. van de Pol 2012; van de Pol et al. 2010).

The tendency to give students more responsibility as the degree project work progresses was evident in several of the email conversations documented in our material. For example, it was usually the supervisors who invited the students to the first supervision meeting, often giving them specific instructions about the type of text they were expected to submit. By the end of the degree project period, however, the responsibility for keeping in contact or for the type of text submitted was placed more on the students, with the supervisors taking a more detached role. Since, as we discussed in Chap. 3, taking responsibility is a fundamental understanding of what student independence can mean in relation to degree project writing, this can be understood to mean that supervisors expected greater independence as the degree project work progressed. At the same time, this was based on the assumption that students would continue to make progress and take the supervisors’ comments on board. Otherwise, the response in the final phase of the degree project period may again become more directive, which we also have examples of in the material (see also Zackariasson 2019).

While supervisors’ comments via email were often a response to questions or requests for feedback from students, in the recorded supervision material it was usually the supervisors who initiated critical topics of conversation and pointed out problematic aspects of the drafts. In some conversations, however, the supervisors’ criticism emerged partly in response to the students’ questions, as in this example:

Supervisor A::

Then you ask: “What do you think an article should contain to be relevant?” How do you answer that?

Student C::

Mm, yes, maybe we should elaborate on that?

Supervisor A::

Yes.

Student C::

What we mean …

Supervisor A::

Yes.

Student C::

… with relevant.

Student D::

Yes.

Supervisor A::

Yes.

Student C: :

Do you think we’re asking too broad questions?

Supervisor A::

Yes.

Supervisor A, Recording 5, Pair supervision

In this example the supervisor reads out an interview question that the students have formulated in their interview guide, which the supervisor thinks is difficult to answer. The supervisor’s initial criticism is implicitly formulated as a question, but then the students ask a counter-question, “maybe we should elaborate on that?”, to which the supervisor answers “yes”, as well as to the question the students ask later, “do you think we’re asking too broad questions?”. These two “yeses” confirm the students’ self-formulated criticism.

In the following example it is the supervisor who initiates a topic relating to something they were critical of in the students’ work, again introduced in the form of a question: “Media logic, is it really a theory that fits?”

Supervisor A::

So I’m thinking about media logic, is it really a theory that fits?

Student D::

Hm, yes, I guess that’s mediatisation.

Student C::

Mm.

Student D::

I actually have a bit of a hard time telling the two apart sometimes, but …

Supervisor A::

Mm.

Student C::

Those are the two that we’ve been thinking about anyway.

Supervisor A::

Um … media logic explains why the content of the media looks the way it does.

Student C::

Mm.

Student D::

Yes.

Supervisor A::

Yes, but you’re not going to examine the content.

Student C::

No.

Supervisor A::

You’re going to look at how people perceive the content.

Student C::

Yes.

Supervisor A::

That’s kind of the next step, isn’t it?

Student D::

But can you have that as a kind of background then?

Supervisor A, Recording 2, Pair supervision

The supervisor’s initial question contained negative criticism in the form of a challenging and problematising question (cf. Chap. 6), which led to a discussion in which the students and supervisor worked through the issue together. In this way the students were actively involved in finding out what the problem was and how to deal with it.

In the following example, the supervisor criticised the students for not linking their results to theories and previous research, and for not having a section with a discussion and conclusion.

Supervisor A::

Yes, that’s what I’m missing so to speak, and that’s what you have to do. Because then when you come to the conclusion and the discussion …

Student C::

Mm.

Student D::

Mm.

Supervisor A::

There, what you’re going to do there is link your results to theories and previous research.

Student C::

Yeah.

Student D::

Yeah.

Student C::

Exactly.

Supervisor A::

Because the thematic analysis is not conclusion and discussion, but it is …

Student D::

No.

Supervisor A::

It’s part of …

Student C::

Yeah, right, of the analysis.

Student D::

Mm.

Supervisor A::

Are you with me?

Supervisor A, Recording 5, Pair supervision

In the extract, expressions of lack and absence as well as negations are used to express the criticism. The supervisor points out what is missing and underlines that the thematic analysis is not the same as a discussion and conclusion.

Negative criticism can also be expressed through value judgements that are perceived as negative. For example, supervisors may suggest that students’ research questions or aims are too vague and general or too broad. Opinions of this kind make it clear that it is often desirable and positive to make the degree project and the text more specific, limited and clear. However, the opposite can also be true, as in the following example where the supervisor described the students’ description of the concept of media logic as ‘a little too narrow’.

Supervisor A::

You also write about media logic, there I think … You describe it a bit too narrowly. Like, media logic contains many different mechanisms, such as how it is about … Media logic is about how the working methods and working conditions of the media shape the content, so to speak.

Supervisor A, Recording 4, Pair supervision

Negations and value judgements are examples of explicitly formulated negative critique. There are also more implicit forms of negative feedback, not least the challenging and problematising questions, which by their very form can encourage students to think in new ways and change perspectives (see Chap. 6). These types of questions are often perceived by students as negative criticism, even though it is implied, as in the following example:

Supervisor A::

That’s why I think this question that you have: “What things do journalists tell in their own words and what things is the person allowed to say?” That question … What does it say? What do you think? Why do you include this question?

Student C::

Yeah.

Student D::

I don’t really know. Like, it was one of those things that we thought was good in the beginning but now it might feel a bit superfluous, when we have done the analysis.

Supervisor A::

Then you simply remove it.

Student C::

Well, we’ll remove it.

Supervisor A, Recording 5, Pair supervision

The students’ reactions to the supervisor’s questions indicate that they perceive them as criticism. When, as a result of the students’ responses, the supervisor suggests that the research question be removed, the students agree. These types of challenging and problematising questions are quite common among all supervisors in our material and are the most common form of criticism. Using such questions to communicate criticism can offer more opportunities for students to explain and develop their thinking than explicitly and unambiguously stated negative criticism. However, the challenging and problematising questions are often combined with other expressions of negative criticism, although often the question comes first and other more explicit forms of criticism come second, as in the following example, where the supervisor asked the two supervised students several critical and challenging questions, followed by recommendations and criticism:

Supervisor C::

What would you say … What is your main research question at the moment? Or rather … Now you write four different questions as well as some other questions in these two paragraphs that are there earlier [in the text]. So it’s all connected, but it’s a bit … The emphasis is sort of on different things.

Student J::

Last time we probably just changed the questions and not the purpose itself.

Supervisor C::

Yes, but what were your thoughts on the questions then? Or how … Why are they in the order they are and what is the most central thing you want to get answers to?

Student K::

Well, it’s really the comparison between the national tests and these different teaching materials. And that’s why we’ve moved this question up because it was further down last time, right?

Supervisor C::

Yes.

Student J::

Right, or we … well, like a little bit the order in which it can be done. First of all, you might have to find out what kind of … What kind of writing discourses they contain before you can compare them. But maybe you could still put the questions in a different order.

Supervisor C::

Well, it would be good if they were in the order that the most important or central thing that you want to get at is at the top and then it becomes, like, more precise or more detailed or whatever. Because now it’s … At first when I read this, both the aim and the questions, then it was a bit unclear to me what you think is most important.

Supervisor C, Recording 3, Pair supervision

By using questions to provide critical feedback, the supervisor gives students the opportunity to develop their thinking and thus invites them to be active in the scaffolding work that can be said to take place during the supervision conversation (cf. van de Pol 2012; van de Pol et al. 2010). The students also respond by explaining their thinking and choices. This then becomes the basis for the supervisor’s recommendation at the end of the exchange and the explanation of what would be the problem with the way the students have chosen to do it now—that it becomes unclear to the reader what is most important. At the same time as the supervisor’s questions are mainly developing (see Chap. 6), it is clear to the students that they also contain criticism, as can be seen in Student J’s comment that it might be possible to do things differently to the way they have done them now.

However, implicit criticism in the form of questions can also become a tool that is unclear or misunderstood by students. The following dialogue is an example of when it appeared as not entirely clear to the student what the supervisor was trying to get at by asking the questions. Instead of developing and explaining the thoughts and choices, the student reacted by expressing confusion and uncertainty:

Supervisor B::

How would you access the conceptions through your surveys, would you say?

Student G::

How I … eh, to see which working method they … which method the students find most motivating. Those are the research questions, yes.

Supervisor B::

So … mm.

Student G::

I don’t think I really understand. (laughs)

Supervisor B::

How did you set up these surveys?

Student G::

They answer from 1 to 5, like on a scale, from number 1 to 5 that they answer.

Supervisor B::

I think my question is this: How do you avoid, through the surveys, that you don’t get a yes or no result? I’ve asked this before. Like, how do you get to this interesting question, “factors in the classroom environment that lead to increased motivation”?

Student G::

Mm … I don’t know. I don’t have a good answer. /…/

Supervisor B::

Mm. Do you have any of these [survey] questions in the back of your head?

Student G::

Oh God, I am at a complete standstill. And I haven’t … it’s on the other computer …

Supervisor B, Recording 5, Individual supervision

This quote thus illustrates the difficulties that can arise when using questions to convey negative critique to students, such that it may be unclear to them what the supervisors are actually trying to say. The student’s reactions in the quote indicate that he/she perceives that the supervisor is being critical, but not what that criticism concerns specifically, which makes it difficult for the student to actively contribute to the ongoing scaffolding work. An alternative approach for the supervisor might have been to start by pointing out that there were problems with the way the student had designed the survey, explaining why it was problematic, and then asking the student to explain the choices made in terms of collecting material and what possible alternatives there might be.

If the intention is to increase student activity and facilitate greater student independence, such an approach may seem counterintuitive, as it may be perceived as more directive, making the supervision characterised more by teaching and less by partnership (cf. Dysthe 2002). At the same time, there are situations and stages in the supervision process when it may be both necessary and desirable for the supervisor to step in and provide a clear framework for the students in order to keep the work moving in the right direction. In the focus group interviews we conducted, it also emerged that clear frameworks and directiveness may in some cases promote student independence and participation in the degree project work. In other words, in situations like the one above, students may find it easier to develop their arguments and thoughts if it is clearer to them what the problems are, even if this means that the supervisor takes a more directive role in the discussion.

In Chap. 6 we wrote about how supervisors often hedged and moderated their positive assessments and praise in the recorded supervision sessions (see also Magnusson 2020). This can be understood in relation to the different roles of supervisors and examiners, that we discussed above, in the sense that overly positive assessments from supervisors could lead to students building up unrealistic expectations of the examiner’s forthcoming assessments. Similarly, when it came to negative assessment or critique, supervisors in our material often used hedging and downplaying strategies. In the examples we have from our material, this seems to be related, for instance, to ensuring that the students did not experience the criticism as face-threatening (cf. Brown 1987). As in the following extract, where the supervisor tried to soften the criticism not only by how it is phrased but also by laughing and commenting on the students’ diligent note-taking:

Student J::

But then we want to take the opportunity to compare once we have the results from the different teaching materials. It feels like it could be relevant too, but it does become two different things sort of … Like there will be two different investigations.

Supervisor C::

Well, that’s not really the case. You are doing a survey but you can draw several kinds of conclusions. You will get answers, or you already have a little bit, to both questions. But … We talked about something similar last time we met, and it’s good, as you say now, to have this “alignment” as the most central thing. And then it should come first in the aim. Then digitisation should not come first.

(Silence, except for someone typing on a keyboard)

Supervisor C::

(chuckles) You’re taking so many notes! (laughs) But if we then look at the text itself or, say, in a little more detail. Because then I understand that this formulation of the aim is an older one, and here you have already rewritten the research questions. But when you then formulate the aim … Now you are constructing it in a way that is quite difficult for the reader to follow. /…/ So it’s much better to turn it around.

Student J::

Absolutely.

Supervisor C::

So that the reader first knows “What is the purpose? And then what methods you use to answer the purpose.

Supervisor C, Recording 3, Pair supervision

To sum up, there are many ways to give negative critique: implicitly and cautiously, or explicitly and in a more challenging way. Negative critique highlights gaps and the need for change, and we would argue that it is an important part of the scaffolding that students need to move forward in their work. However, it can also make students feel incompetent and vulnerable. In order for the negative critique to be constructive and formative, supervisors need to be aware of how it can be presented in the specific situation. This has to do with contingency—being able to adapt the supervision to the students’ needs and conditions and choosing the scaffolding strategies accordingly (cf. van de Pol and Elbers 2013; van de Pol et al. 2015).

Assessing Independence

In the introductory parts of this book, it was discussed that independence is an aspect that is often addressed in the intended learning outcomes of the syllabus or in the grading criteria for the degree project. Where this is the case, student independence also needs to be assessed. Lecturers and seminar leaders, supervisors and examiners need to know and be in agreement about what is meant by independence in a particular course, and where and how this independence is to be found. The matrix introduced in Chap. 3 presented seven ways of understanding student independence that recurred in our focus group material:

  1. 1.

    Taking initiatives

  2. 2.

    Demonstrating originality, creativity and enthusiasm

  3. 3.

    Relating to sources and context

  4. 4.

    Arguing, motivating and making choices

  5. 5.

    Taking responsibility

  6. 6.

    Demonstrating critical thinking and reflection

  7. 7.

    Ability to generalise and synthesise

These different ways of understanding independence can serve as a basis for a collegial discussion, and by extension a common position, when it comes to how independence should be understood and therefore assessed. In the same matrix, the different contexts identified by the focus group participants as relevant in relation to independence were also presented:

  • during the process of writing a degree project

  • in the final text submitted for assessment

  • during the defence of the degree project

Thus, it is in these contexts that the supervisors and examiners who participated in our focus groups believed that independence could be identified. However, it is not obvious how these different perceptions of independence can be found and evaluated in the different contexts. The different ways of understanding independence provide a starting point, but each of these conceptions or understandings of independence needs to be operationalised and broken down into concrete application in the respective context in order to serve as a basis for assessment. In the last part of the chapter we will discuss an example of what such operationalisation and application might look like. First, we would like to take a small detour to consider how artificial intelligence (AI) might influence the assessment of student independence, and what issues this might raise in the context of degree projects and supervision practice.

AI and Assessment in the Context of Independence

Every supervisor and faculty needs to consider AI and the opportunities and risks that new LLM models,Footnote 4 such as ChatGPT, offer in relation to student independence and the writing of degree projects. There are two perspectives that become particularly important in this context: the supervisor’s role in assessing student independence and the AI writing tools that students are encouraged to use and those that become problematic for students to use in relation to independence. The role of the supervisors is to support students in writing a degree project, which includes support in the socialisation into an academic environment and the development of academic thinking (Brodersen 2009). For a long time, it has been possible for students to purchase a thesis from a ghostwriter or to have a degree project written for them by someone close to them. Today, however, the situation is quite different, as students can use AI tools to help them at all stages of the writing process: generating ideas, structuring ideas, formulating texts and editing texts according to different levels of style and various specified criteria.

One way of dealing with potential ghostwriting is to give the process a greater role and visibility in degree project courses. For example, course coordinators can require students to submit drafts in different rounds and to work on them based on different types of feedback. Giving more importance to the process also means giving more importance to the supervisor as someone who has insight into the process and can assess its independence. The supervisor then becomes not only someone who supports but also someone who controls and evaluates the students’ work and the process, which can have both advantages and disadvantages. The advantages include that as the process becomes more important than the product, supervisors and examiners are encouraged to collaborate in the assessment, making the assessment more collegial. The disadvantages include that the supervisors’ role becomes more complex and partly conflicting when support and control are pitted against each other in even more ways.

With AI tools, the supervisor’s task becomes even more complex and the supervisor’s controlling role is further strengthened. Here, the supervisor needs to guide the students on which AI tools are acceptable to use and which are not, in relation to the expectations and requirements of independence. It becomes increasingly difficult to define what kind of independence is desirable and necessary in the writing of the degree project:

  • Is it important for students to come up with the topic themselves or can AI tools be used to generate content and ideas?

  • Is it important for students to write the text themselves or is it acceptable to dictate the text and have an AI tool or a classmate rewrite it?

  • Is it important that students are able to spell for themselves or is it acceptable to use spell checkers?

  • Is it important for students to summarise and compare text content (empirical evidence, previous research) themselves or can AI tools be used for this?

  • Is it important that students are able to write in a second language by themselves (e.g. English if their first language is not English)?

  • Is it important that all analysis is done by the students or can digital and AI tools be used to identify patterns, categories and relationships?

These are just some of the questions that course coordinators, supervisors and examiners need to discuss and agree upon, and which supervisors then need to address in their supervision practice. As indicated above, such questions may concern, in different ways, more or less all the main understandings of student independence that we identified in our focus group material, such as taking initiatives, taking responsibility, the ability to synthesise and generalise and so on. In other words, the role and significance of AI is one of the aspects of operationalising the different understandings of student independence in the degree project context, in order to provide a basis for assessing them. In the following, we will discuss other such aspects in relation to the understanding of student independence as the ability to relate to sources and context (cf Chap. 3).

An Example of Assessing Independence

Independence understood as students’ ability to relate to sources and context is something that can be manifested both in the supervision interaction and in the written texts, and we will in the following look at both of these aspects. What constitutes sources and context here, includes, e.g. books and academic literature that the students have read, things that informants have said in interviews and that the students refer to or describe, as well as what the supervisors have said and recommended in supervision sessions.

Relating to Sources and Context in Conversations

To relate to something generally means to make an evaluation, and in this case it implies that a student makes an evaluation in relation to a source or context of some kind. In our material, students related to sources by evaluating them in a variety of ways: as good, fun and interesting, as useful and clear, or as difficult and so on (cf. Magnusson 2021). The ways of relating to sources varied considerably in complexity and specification, and in our material we could see two main levels of how students could do this:

  • Relating to and evaluating something as good or interesting.

  • Relating to and evaluating something as good or interesting and justifying this in relation to the degree project work.

Relating to a source by simply evaluating something as good or interesting has a low degree of complexity and specification and thus usually belongs to the first level. The same applies when students merely confirm what the supervisors say or recommend, without more developed reasoning or motivation, which was quite common in our recorded supervision sessions. As in the example below, where the supervisor suggested a reading strategy or workflow for the student—to read a particular researcher’s work and choose a method based on what was written there:

Supervisor E::

So read [scholar], but as quickly as possible, so that you jump down to the test and try to see: What do I choose?

Student N::

Right, I want to read through all the texts and so on.

Supervisor E::

Yeah, sure.

Student N::

That sounds good.

Supervisor E, Recording 3, Individual supervision

It is perhaps not very surprising that positive affirmations, in this case ‘that sounds good’, are by far the most common student response to what supervisors describe or recommend in our material, as it can be perceived as face-threatening in the interaction to show a lack of appreciation or scepticism towards what supervisors are recommending and suggesting (cf. Brown 1987). In this example, the positive affirmation is not substantiated and the student does not show how he/she relates to what the supervisor brings up more specifically. Consequently, it can be considered as an evaluation of relatively low complexity and specification, corresponding to the first level of relating to sources and context. In another example, a student described and evaluated what an informant had said in a recorded interview:

Student Q::

That was very interesting. I have to say that. Because I got quite a lot of material then.

Supervisor F, Recording 4, Individual supervision

The student here evaluates what the informant has said in the interview as ‘interesting’, but there is no justification or further specification of what is interesting about what the informant has described in the interview or how it can be related to the study the student is doing. Therefore, in terms of complexity and specification, this example could also be said to correspond to the first level of relating to sources and context.

In an example of more developed reasoning, the student evaluated and related to theoretical sources, in this case a theoretical work on language theory that the student had read:

Student S::

It’s kind of outside of the degree project itself when it comes to grammar and stuff like that, but in a way the same entry is super useful … fundamental sort of … kind of the same thing … And that feels great.

Supervisor H, Recording 3, Individual supervision

The theoretical source is considered by the student to be ‘super useful’ in the degree project work, which is linked to the fact that it is ‘fundamental’. The student also describes that it is “kind of the same thing”, which in this case can be understood as the student perceiving that there are parallels between this theoretical source and other theoretical sources that the student has read before. That is, it is possible to link these sources and perspectives, one as a general starting point and the others as more specific in relation to the focus of the degree project. Evaluating aspects in relation to each other in this way, and relating them to one’s own study and how they will be used there, can be seen as an example of relating to sources and context in a more advanced way.

In another, more complex example, the same student relates to different theoretical perspectives, comparing them and positioning them in relation to the degree project:

Student S::

Uh, and then … The way I interpreted it, social semiotics and multimodal discourse analysis, as I understand it, is also equivalent to the multimodal perspective, and it’s the same perspective as [scholar], who has worked in both of them. And then, I’ve only read a little bit, but then I saw [scholar]’s functional grammar as a starting point for both, as well as critical linguistics and social semiotics. /…/ And then critical linguistics and social semiotics with discourse analysis as a kind of subheading to that, sort of, and then there’ll be a bit of repetition. /…/ Because then it’s a bit like this functional grammar, that it’s like a bit more theoretical.

Supervisor H, Recording 3, Individual supervision

The evaluative formulations used in the student’s discussion of different theoretical perspectives and how they are relevant to each other and to the student’s own study have to do with structure and hierarchy. It’s about what should be a starting point, what should be superior and subordinate, what should come first and what should come later in the text. Word choices such as ‘starting point’, ‘after that’ and ‘subheading’ clarify how the student evaluates and relates to the different perspectives and how they might relate to each other in the forthcoming degree project. There are also phrases that compare the different theoretical perspectives, such as ‘equivalent to’ and ‘same perspective’. These phrases can also be seen as evaluative, as they judge parts in relation to each other, what should come first and be most important in relation to other things, and which parts are equivalent and comparable. All this together means that the extract can be seen as an example of the second level of how students can relate to sources and context, in terms of complexity and specification.

Below is another example that can be seen as corresponding to the second level of complexity, where the student in question has come into contact with the notion of hierarchy of needs and is considering whether it might be a relevant concept or model in a study of which age groups teachers choose to work with:

Student R::

Well, previous research. Then there was somebody who had brought up some sort of hierarchy of needs, which I initially thought sounded very far-fetched. I associate it with marketing, I think it was, like, far away from here. But then when I thought about it, it’s a bit more and more like it makes sense. Because it’s a bit like this … It might explain why you choose to do something. How is it that you … So I’ll see if I can get hold of any [literature], so maybe I’ll have a look at that too.

Supervisor G, Recording 1, Individual supervision

In this example, the student described a development of thought by explaining how he/she first perceived the theory in question as ‘far-fetched’, but then as ‘making sense’. Both evaluations were justified in relation to the investigation the student was working on. The ‘far-fetched’ was justified by the student’s description of associating the theory with marketing, which was far from what the student’s study was about, while the ‘making sense’ was motivated by the statement ‘it can explain why you choose to do something’, which was closely related to what the student was investigating in the degree project.

If the supervision interaction is seen as an important setting for assessing student independence, which, as mentioned above, may be even more the case as a result of the increasing use of AI in the writing process, then, based on these examples, assessment criteria could be formulated so that on one level students are expected to refer to sources and context, but on another level they are also expected to be able to justify this explicitly. In a supervision discussion, the supervisor can then support the student in this, for example, by asking developing questions (see Chap. 6) that elicit motivation and specification: How do you think this will be relevant to your study? When you say it’s interesting, what do you mean by that? How can this be useful for your analysis?

Relating to the Supervisor in Text

In order to see how students actually absorb and relate to the supervisors’ demands, recommendations and questions in the supervision interaction, it is necessary to see how the students’ text drafts are rewritten and developed. From this perspective, it becomes relevant to assess or evaluate the student’s remediation. Remediation in a supervision context concerns how what is said or discussed in the supervision session is (re)used as text, another medium. Looking at how students process their texts in relation to the supervisors’ recommendations and suggestions reveals how students relate to them, i.e. how they are remediated in the students’ texts (cf. Magnusson and Sveen 2014).Footnote 5 The supervisors’ recommendations and suggestions can be remediated by the students in different ways, for example as imitations or appropriations (e.g. Wertsch 1998). Imitation in this context involves students reproducing and repeating what the supervisors recommend, without showing that the understanding or knowledge has been fully integrated and without clearly relating to what the supervisors recommended more than by repeating it. If students have integrated the learning or knowledge and clearly relate it to the supervisor’s recommendations, appropriation has taken place instead.

Based on these concepts, two different levels or degrees of independence can be distinguished:

  • Following the supervisor’s recommendations and writing exactly what the supervisor suggests (imitation).

  • Following the supervisor’s recommendations in several contexts and/or using them to find one’s own solutions and formulations (appropriation).

In other words, these levels or degrees of independence could serve as a basis for supervisors to assess student independence, for example, in terms of their ability to relate to sources and context. Imitation is a necessary strategy in learning and should not be seen as undesirable or problematic in itself (e.g. Wertsch 1998). Rather, imitation may indicate that the student has not progressed so far in the learning process if the goal is student independence, understood as relating to sources and context. In a previous article by Magnusson and Sveen (2014), we have examined these different forms of remediation in a number of supervision sessions and in the text drafts written after the current supervision sessions. The supervisors’ advice in the form of recommendations and suggestions has thus been followed up in the students’ drafts, where it is possible to see if and how the students have remediated what the supervisors have said and suggested.

In the first example, taken from this article, the supervisor quotes and comments on a choice of words made by the student in the draft text, a choice of words that is taken out of its original context and could therefore be misunderstood:

Supervisor G::

“That traditional methods of science such as measuring data and experiments do not provide relevant information” … ehh and then you start with [scholar] and how she talks about traditional scientific methods. And then one is talking about positivism and natural science and so on, right? But here it’s taken out of context and then you wonder: “What do you mean, ‘traditional methods of science’?” At this point the social sciences are pretty traditional too.

Student R::

Hmm.

Supervisor G, Recording 1, Individual supervision

In line with the definition by van de Pol et al. (van de Pol et al. 2010; van de Pol 2012), the supervisor could here be said to scaffold the student by explaining why the choice of words in question is problematic and by explaining it from a reader’s perspective: “and then you ask yourself: ‘What do you mean – traditional scientific methods?’” In the draft text that the student wrote after this supervision session, it was clear that the student had taken on board the point and made a modification in the text, changing ‘traditional’ to ‘natural sciences’:

When, as in this case, the purpose is to find out people’s experiences and goals with their actions, the hermetic (sic) theorists argue that traditional methods of science natural science investigation methods, such as measuring data and experiments, do not provide relevant information (reference).

This can be described as imitation, as the student followed the supervisor’s advice directly. The student was made aware of the risk of misunderstanding and given suggestions on what to write instead: “and then one talks about positivism and science”, advise which was followed without further contributions or additions. This is thus a first, important and necessary step on the path to greater independence, where the student takes on board what the supervisors say at a basic level.

In another example from the same supervision session, the student relates to the supervisor’s suggestions in a different way in the subsequent revision of the draft text. The supervisor’s comments again concern a choice of words that might be misunderstood by readers. The advice offered by the supervisor moves from the general to the specific—from talking about formulations at a general level to giving specific examples from the student’s text—and includes scaffolding in the form of modelling (cf. van de Pol et al. 2010; van de Pol 2012), by suggestions of what a potential formulation might be:

Supervisor G::

And then there’s another one of those wording things on the next page, under ‘qualitative interviews’. At the beginning of the second paragraph there, you write: “By qualitative interviews one means interviews in which the interviewee makes up the answers” and there I would have used a more elegant phrasing, for example that they formulate the answers freely. Otherwise it sounds like they’re making something up, and I don’t think that’s what you mean.

Supervisor G, Recording 1, Individual supervision

In the draft submitted after this supervision, the student took on board the supervisor’s point of view and modified the sentence, for instance, by changing ‘makes up’ to ‘decides’ and ‘determined’ to ‘defined’:

Qualitative interviews refer to interviews in which the interviewee makes up decides the answers, unlike quantitative interviews where the researcher has determined defined specific alternatives (reference).

This can be seen as appropriation rather than imitation, as the student makes choices based on what the supervisor recommends, finding their own way of relating to and reformulating what the supervisor has commented on. The student did this even though, again, the supervisor had provided suggestions on how to write in order to avoid misunderstandings.

In another example from a later supervision session with the same student, the student’s actions could also be seen as appropriation rather than imitation, as the student did not receive support in the form of specifically formulated suggestions but in the form of a broader and more general recommendation. The student was encouraged to think about what should be seen as more or less important to include in the text, given the space available in the degree project. The supervisor suggested, among other things, that the student should not write so much about hermeneutics.

Supervisor G::

/…/ Then it would be good to put this topic in a scientific context and, like, that you want to interpret and understand and so on. And you don’t do that so much, so extensively now. But I still wrote [in my comments] that you could still think about how much you need to discuss hermeneutics specifically and that the most important thing is, like, how you have done things and your reflections on it.

Student R::

Yeah, it was also like I wrote everything I could think of in this first version and then I included everything I could put together, kind of. So I’m pretty much on the idea that I’m going to cut that particular part.

Supervisor G::

That sounds good.

Supervisor G, Recording 2, Individual supervision

In the draft that the student wrote directly after this supervision meeting, nothing had changed, but in the final version of the degree project it seemed that the supervisor’s recommendation had been taken into account, since the student had removed a large part concerning hermeneutics (see marked text).

This study is qualitative and based on a hermeneutic ideal of science. Hermeneutics is a scientific tradition concerned with the interpretation of meaning and is mainly used in studies of people whose actions hermeneuticians believe cannot be explained by measurable data (reference). The purpose of this study is to understand people’s perceptions of the teaching profession and their goals in choosing teacher education in order to understand how they ended up where they are today. When researchers try to understand people’s actions in this way, hermeneuticians believe that it is necessary to make interpretations of the collected material (reference). One of the most important theorists of hermeneutics, Hans Georg Gadamer, points out that interpretations are always influenced by the person who makes them, so that the knowledge it leads to is never universal. The results we arrive at, he argues, depend on the prejudices we have that form the framework for what is possible for us to understand (reference).

The hermeneutic research tradition, which states that research on people should be conducted by searching for and interpreting meaning in people’s expressions (reference), formed the basis for this study. The method used was semi-structured qualitative interviews, which were analysed using content analysis.

When, as in this case, the purpose is to find out people’s experiences and goals with their actions, hermeneutic theorists argue that traditional scientific methods such as measuring data and experiments do not provide relevant information (reference). In hermeneutics, one is instead interested in collecting different types of meaning content, such as stories or observations of actions (reference). When analysing meaning content, we are not interested in how many times the interviewees use certain sounds or words, but we are interested in what they say, the meaning they express. Therefore, in hermeneutic studies it is irrelevant to try to analyse the informants’ narratives through measurements (reference).

The different levels at which students can relate to the advice or recommendations of their supervisors is thus reflected in these examples by the way in which they demonstrate in their written texts that they have understood and processed the recommendations and advice of their supervisors. Do students simply copy what the supervisors have said, or do they relate to and modify what the supervisors have said and formulate their own proposals and solutions? This, we believe, can be another tool to distinguish between a more or less independent way of relating to the supervisor’s comments and advice.

To sum up, in these sections we have identified two different ways of accessing how students can relate to different sources and contexts, two ways in which this particular aspect of student independence can be operationalised, differentiated and broken down into levels or degrees:

  1. 1.

    • Evaluating and thus positioning oneself in relation to a source.

    • Evaluating and positioning oneself in relation to a source and justifying this in relation to one’s own study.

  2. 2.

    • Following the supervisor’s recommendation and writing precisely what the supervisor suggests.

    • Following the supervisor’s recommendation by finding your own solutions and formulations.

These two types of differentiation of how students can relate to sources and the supervision context in which they and the supervisors are involved could, we would argue, be useful to supervisors as a basis for assessing independence. There are, of course, a number of other ways of identifying, differentiating and assessing different aspects of student independence, and it is up to each supervisor and each department to decide which are particularly relevant and desirable in their own activities and how best to access them. With the rapid development of AI, there will certainly be a need for further ways of accessing independence, as AI tools such as ChatGPT provide not only summaries of sources but also comparisons and positioning in relation to those sources.

Discussing and operationalising not only student independence but also other concepts and formulations that are included in grading criteria and assessment matrices, as a supervisor collective within an academic programme, degree project course or discipline, can be a fruitful way of trying to articulate the implicit norms and beliefs that exist around academic writing in different local academic contexts (cf. Lea and Street 2000; Lillis 2001). In this way, supervisors may become more aware of how to address the fact that assessment too is an essential part of supervision practice, as well as their role as an assessor in this context, and how this role relates to, for example, the role of the examiner. This kind of collegial discussion thus has the potential to contribute to the development of the individual supervisor’s practice in this respect as well.

In the chapters of this book, we have discussed several aspects of degree project supervision and supervision practice, ranging from the relationship between supervisor and student and the emotional aspects of the supervision process to specific tools to scaffold students in their degree project work and encourage independence. As we have now also discussed different ways of understanding the supervisor’s role as an assessor and given examples of how one particular aspect of independence—relating to sources and context—could potentially be assessed, all that remains is to tie the knot. We do this by presenting in the concluding chapter some of our main findings and some perspectives and approaches that we hope supervisors can take away from this book.