eLExp Assignment 3: e-Learning Course Criteria … (Optional Component)
Developing a criteria for the evaluation of the BBC Genre-Buster: Children’s Fiction online course in a reading program context
Monday 12th November 2007
INTRODUCTION
Many primary schools run reading programs in which students from higher grades or secondary schools, or parents, volunteer their time to help develop reading skills of young children. Based on this approach and thinking about new ways to help children, particularly those who are struggling, improve their reading abilities I thought that writing a short story specifically created for each child would be a great way to do this. The logic behind my idea was to have the children involved in the process so that they will actually be reading something that is of interest to them. So rather than selecting a book and learning to read it, volunteers would interact with the child to get ideas for a story that they will develop over a short period and then once compiled have the child read it. Thus, if this were introduced into school reading programs, volunteers would need a basic understanding of how to write a children’s story so that they can maximize the effectiveness of the stories they create. And so having volunteers participate in a short online course seemed a very good option.
Now the purpose of the evaluation criteria is to gauge the effectiveness of the BBC children’s writing course, ‘Genre-buster: Children’s Fiction’, in helping to create the stories and then to measure the effectiveness of the stories in improving the reading ability of young children. Having participants evaluate the course will show whether or not it is a worthwhile tool for the proposed alteration to the reading program and thus indicate whether it should remain or be discarded for future use.
KEY ISSUES THAT WILL BE ADDRESSED BY THE CRITERIA
- Administration and Support materials – how supportive were they?
- Presentation of the course – technology, interactivity, feedback, content, navigation (user friendliness)
- How much the course contributed to making the reading program effective
- How well it suited them – because they are volunteers, how much time did they give up, is it worthwhile?
- Improvements/Comments about the course – do the participants recommend it to be used in future?
- Comparing the proposed reading program with the original reading program
DEVELOPMENT OF THE CRITERIA
In a paper by Davies et al. (2007), it was identified that questions asked in university surveys were either questions about the lecturer and the subject or questions about the student and their learning. The criteria developed is set out in the same manner – the questions either relate to the course or to the learner. It is not necessarily divided into two separate sections rather is intertwined to show a relationship between the learning experiences of the participant to specific aspects of the BBC course.
Sonwalkar (2001) identified that ‘objective criteria for a summative evaluation should be applied in five major areas, including:
(1) content factors,
(2) learning factors,
(3) delivery support factors,
(4) usability factors, and
(5) technological factors.’
These areas are all generally covered in the criteria. Sonwalkar’s article addressed why these factors were important which has assisted in evaluating the BBC course as it raises the level of expectation of the how the course should be delivered. The example given in the article of a summative evaluation instrument for rating online courses provided a good basis for which to build the criteria, however it was referred to, to simply gain a better understanding of the areas I should be concerned with when evaluating the BBC course and as my criteria also addresses the learning experience in relation to the context I wanted to put more emphasis on that and attempt to gain more qualitative rather than quantitative answers from the criteria.
In an evaluation questionnaire developed by Reeves and Hedberg (2003) questions were divided into six main areas – course content, course design, course instructor, course environment, course results and self-paced delivery. It concluded with questions about how the learner could improve the course and what they found most and least valuable from the course. The way in which the final questions were set out is particularly helpful in gaining quality information from learners about their experiences as before openly asking what could be done to improve the course, Reeves and Hedberg set out a series of suggestions such as ‘provide better information before course, clarify the course objectives and so on’ that the learner can check off. This initiates a more critical approach to be taken on board by the learner as it gets them thinking about ways to improve the course. In a similar fashion, the questions about what the learner found most and least valuable also gets the learner to think about what they have taken away from the course. Particularly for institutions looking for feedback on their courses, this can help in identifying components of the course that are relevant as well the methods that are being used to make them successful.
Owston’s paper (1999) didn’t necessarily assist the development of the criteria however it discussed an example of a long running project called Writers in Electronic Residence (WEIR) which is a ‘web-based conferencing system to link writing and language students to Canadian authors, teachers and each other for the exchange and discussion of original work’. This was of particular interest as it can be used to compare how the BBC course differs to the approach of the WEIR project. Generally, WEIR involves ‘the participation of up to 120 classes in any given year from all areas of the country and including students ranging from the junior elementary to the senior high school levels’(Owston, 1999). ‘Students “post” drafts of creative works into forums which the assigned author reads and then responds to, posting back comments and suggestions for revision to the student (which others accessing the conference are free to read). Students also read and respond to work posted by other schools. WIER strongly encourages participating teachers to require every student who submits a composition to provide a written response to two other student works’(ibid).
Compared to the BBC writing course this project is a lot more dependent on collaboration which I believe would provide better outcomes as writing is a process that must be critiqued by outsiders so that your biased opinion does not restrain you from writing a quality piece. However, considering the context that I have described above, implementing a project like WEIR is much more time and resource consuming and so many would disregard it as an option. But in my personal opinion, having an online network is a valuable tool because you are gaining insight and criticism from others around you which opens you up to more ideas.
THE EVALUATION CRITERIA TOOL
The evaluation criteria tool is provided below as a separate Word document link. The text in orange in this document indicates my individual evaluation of the course considering the context.
REFERENCES
Davies. M, Hirschberg. J, Lye. J and Johnston. C (2007). An Analysis of the Questions on University Teaching Surveys and the Universities that Use Them: The Australian Experience, Viewed 8 November 2007 <http://www.economics.unimelb.edu.au/SITE/research/workingpapers/wp07/994.pdf>
Owston. R.D (1999) Strategies for Evaluating Web-based Learning, Viewed 8 November 2007 <http://www.edu.yorku.ca/~rowston/aera99.html>
Reeves. T.C and Hedberg. J.G (2003) Interactive Learning Systems Evaluation: Evaluation Questionnaire, Viewed 8 November 2007 <http://it.coe.uga.edu/%7Etreeves/edit8350/QUES.html>
Sonwalkar. N (2001) A New Methodology for Evaluation: The Pedagogical Rating of Online Courses, Viewed 8 November 2007 <http://campustechnology.com/articles/38946_3/>
