Evaluation guidelines for LeMill
Organize a teacher training. The evaluation process of LeMill should start with a short teacher training that will help teachers to get started. The teacher training is needed because LeMill does not have a critical mass of content yet.
Give clear task for teachers. In Estonia teachers were asked to use CALIBRATE tools in four lessons in a month. Teachers had to create learning resource in LeMill for two lessons in a month. After the lessons teachers had to post a short summary of the lesson to their group forum in LeMill. The summary contained:
- the topic of the lesson;
- CALIBRATE content used in the lesson;
- other learning resources and ICT tools used in the lesson;
- problems that occured with LeMill and the Portal;
- self-evaluation of the lesson. What went well? What went wrong?
Organize a focus group interview. At the end of evaluation period you should organize a focus group interview with 6...8 teachers who were active users of LeMill. The focus group interview should have 5...6 main themes. The latest list of suggested themes is available at: http://lemill.org/trac/wiki/FocusGroupInterviewThemes
Translate the themes and questions to your language and customize these according to your local context. If possible you should contact LeMill developers (tarmo.toikkanen@…) and ask if there are any special questions for teachers from your country.
For the focus group interview you need:
- A room that encourages discussion (round table, comfortable sofas, etc.). Computer lab is not a good place for focus group interviews.
- 2 researchers: one guiding the discussion and another one taking the notes.
- A voice recorder.
In the beginning of the interview ask if participants agree with the audio recording. If they do not agree you should only take notes manually.
The themes and questions should be used to guide the discussion. Ask direct questions only when the answers do not come out from the discussion. Do not give the list of themes and questions to the teachers. The focus group interview should take about 90 minutes.
After the focus group you should use the audio recording to write down the list most important comments that teachers made. Translate the main comments English and send to LeMill developers (tarmo.toikkanen@…).