Di’s E-learning Experience

Module 3 – Additional work – option 2

Develop a criteria for evaluating the effectiveness of the course, and based on individual experience, and make recommendations for improvement.

Introduction

Because of my recent participation in the online BBC Spanish program (Spanish Steps) and as I am at present studying Spanish here at the University of Technology, Sydney,  I have decided to develop an evaluation criteria tool for an online Spanish program.  As online learning develops the focus of learning is moving from a teacher directed environment to learner-centred programs. I have decided to develop this criteria evaluation tool with the focus on issues from the learner’s point of view.  It will be designed to allow students participating in the program to provide feed back on the different elements of the Program. This feed back will be valuable resource to administration, program designers and teachers for reassessment and adjustment. 

Over the past decade the number of online language programs has increased rapidly.  Some of these courses are only offered online (e.g. BBC Languages) and others are being developed by schools and universities as part on a blended learning program. These courses combine elements of face-to-face learning with a variety of online components (Phipps & Merisotis, 2000).   The development of these courses seems to be in early development stages and many institutions have recognised the need to develop a set of criteria or benchmarks to evaluate the quality and effectiveness of the courses or as stated by Chao, Saj and Tessier (2006, p. 6)  “The criteria serves as the benchmark for effective course design”.

An important part of the strategy when developing an e-learning program is to put into place a means to evaluate the program and provide feedback that can be used to review and readjust the program to ensure its success.  To carry out this evaluation a set of criteria that looks at such issues as “technology, content, administration and support, communication and financial analysis” (De Vries 2005, p. 2) needs to be developed and defined.

Where do you begin when you look at evaluating an online course? Canning-Wilson (2000) advises to start with the “the simplest aspect, the course layout” and develop your criteria from there.

Using references which include Wright’s “Criteria for Evaluating the Quality on Online Courses, NACOL National Standards of Quality for Online Courses, Mayberry’s (2005) article on Kirkpatrick’s Evaluations and my recent experience with the BBC Spanish Steps language program and Spanish language and Culture at the University of Technology, Sydney, I have developed a list of factors that I think should be addressed in a set of criteria.  These are:

1.        Administration and General Information

• At the beginning of a program learners need to be provided with a clear outline of what the course involves, its goals and the expected learning outcomes.
• Provide  learners with list of required texts and online resources,
• Access to any special software needed for the course
• Provide learners with guidelines for online communication or “netiquette“.
• At the commencement of the program the learners should also be provided with contact numbers or emails for their facilitator and technical assistance and the times when they are contactable.  (Wright).

2.        Course Design and Structure

• A clear outline of the syllabus, goals and objectives
• Assessment and assignments aligned with institute, state and national guidelines.
• The content is directly related to stated course objectives
• The course is divided into lessons within units.
• Course content is current, relevant and linked sequentially in a logical manner.
• Links to other sources of information provided.
• Does the design take into account the participants different learning styles and provide alternative choices for engaging in the course activities.
• The structure of the course allows the leaner to learn new skills and transfer this knowledge into their everyday environment.

3.      Technical  

• Learners are provided with assistance to bring their online skills up to the standard required for the course.
• Facilitators are provided technical support and training to bring them up to date with the technology used. (Phipps & Merisotis 2000, p. 12).
• Hardware, web browser and software requirements are specified.
• The courses uses appropriate tools and software
• Learners and facilitators are provided with technical assistance when technical difficulties arise
• The reliability of the technology delivery system is as failsafe as possible.

4.      Learning Resources

• Learning resources are accessible and appropriate for the subject matter
• A variety of learning resources are provided to allow for the learners different learning styles and abilities.
• The resources are current and relate to the course content.
• Learners are provided with a reference list that outlines web links, books, journals etc.

5.     Language

• The instructions  are written in a clear and direct style.
• The directions are clear and easy to follow.
• The tone of the writing is engaging, supportive and encouraging.
• The sentences are short and precise with the use of correct grammar.
• The writing reflects multicultural behaviour and contains no bias relating to race, sexual preference, age, gender or culture.
• Learners are provided with constructive feedback on assessment tasks.

6.    Learning Experience

• Consideration  has be given to learners who may have visual or auditory difficulties.
• Facilitators have been provided with training to update their skills for teaching online programs. (Finkelstein, 2006, pp.65-67).
• Learners have been given guidance in the development of their study skills and knowledge management (Watkins, 2005, p. 5).
• The learners have been provided with the opportunity and resources to extend their learning.
• A variety of different learning activities are provided that promotes interactivity. e.g. collaborative work, online forums, blogs, online conferencing.

7.        Evaluation and Assessment

• Evaluation strategies are in line with the learning goals and objectives and the relationship between the learning goals and assessment is clear to the learners.
• The course structure includes adequate means to assess the learner’s progress, (i.e. ongoing assessment to assess learners as they work their way through the course).
• Learners are provided with prompt and constructive feedback on assessment tasks. (Finkelstein, 2006, pp.23-25).
• Learners are provided with guideline for submission of assignments and detailed instructions are provided for all assessment tasks.
• The numbers of assessment tasks are reasonable.
• The consequences of plagiarism are explained to learners.
• Learners are able to check and evaluate their progress.

The seven areas  or list of criteria that I have discussed here can be used as a systemic approach to evaluate the quality of an online course.  They are all interconnected components that must be considered and reviewed regularly to ensure the success of an online program.  Using these criteria I have developed a Criteria Evaluation Tool, which is designed to evaluate the criteria from the perspective of the learner in an online Spanish Course.

References

Canning-Wilson, C. 2000, “E-learning with the e-teacher; considerations for On-line Course Design”, Eltnewsletter, Article 42, viewed on 2/11/2007, http://www.eltnewsletter.com/back/December2000/art422000.htm.

Chao, T, Saj, T and Tessier, F. 2006,  “Establishing a quality Review for Online Courses, a formal review of online courses measures their quality in key areas and reveals changes needed for improvement, if any”, Educause Quarterly, no 3, viewed on 10/10/2007, http://www.educause.edu/ir/library/pdf/eqm0635.pdf

De Vries, J. 2005, “E-learning Strategy: a Framework for Success”, Learning Circuits, viewed on 30/08/2007, http://www.learningcircuits.org/2005/aug2005/devries.htm

Finkelstein, J., 2006, Learning in Real Time, Synchronous Teaching and Learning Online, Jossey-Bass, San Francisco.

Flemming, N., VARK, A Guide to Learning Styles, viewed on 5/10/2007, http://www.vark-learn.com/english/index.asp

Massachusetts Department of Education, 2003, “Massachusetts Recommended Criteria for distance learning Courses”, viewed on 4/11/2007 on http://www.doe.mass.edu/edtech/news03/distance_learning.pdf

Mayberry, E. 2005, “Kirkpatrick’s Level 3: Improving the Evaluation of E-learning”, Learning Circuits, May, viewed on 2/ll/2007, http://www.learningcircuits.org/2005/may2005/mayberry.htm

NACOL National Standards for Online Courses, viewed 6/11/2007 on http://www.nacol.org/nationalstandards/NACOL%20Standards%20Quality%20Online%20Courses%202007.pdf.

Palloff, R.M., and Pratt, K. 2003, The Virtual Student: A profile and guide to working with online learners,  Jossey Bass, San Francisco.

Phipps, R. and Merisotis, J. 2000, Quality on the line: Benchmarks for success in Internet-based distance education.  Washington, DC: The Institute for Higher Education Policy.  Viewed 4/11/2007, http://www.ihep.com/Pubs/PDF/Quality.pdf

Smith, T.C. 2005, Fifty-One Competencies for Online Instruction, The Journal of Educators Online, vol. 2, no. 2.  viewed 25/10/2007, http://www.thejeo.com/Ted%20Smith%20Final.pdf

Spanish Steps, BBC Languages, viewed on 5/10/2007, http://www.bbc.co.uk/languages/spanish/lj/

Flemming, N., VARK, A Guide to Learning Styles, viewed on 5/10/2007, http://www.vark-learn.com/english/page.asp?p=helpsheets

Watkins, R. 2005, “Preparing e-learners for online success”, Learning Circuits, viewed 19/10/2007, http://www.learningcircuits.org/2005/sep2005/watkins.htm

Wright, C., Criteria for Evaluating the Quality of Online Courses, Instructional Media Design, viewed  24/10/2007, http://www.imd.macewan.ca/imd/content.php?contentid=36

Trackback URI | Comments RSS