Nutrient Management Regulations and Protocols Online Course Evaluation Project Summary

The information contained in this document is derived from the Nutrient Management Act, 2002 and O. Reg 267/03 as amended. Every effort has been made to make it as accurate as possible, but is not authoritative. Please refer to the e-Laws website or the official volumes printed by Publications Ontario for the authoritative text of the act.

Table of Contents

1. Background
2. Objectives
3. Methodology
4. Key Findings
5. Next Steps

Background

In early 2009, OMAFRA conducted an evaluation of the Nutrient Management Regulation and Protocols online course. This course, which went live in February 2008, was developed to help producers and future consultants develop the knowledge and skill needed to prepare nutrient management strategies and plans.

As the online course was approaching its one year anniversary, OMAFRA felt it was important to assess the effectiveness and appropriateness of the program for the target audience and to make recommendations for the future direction of eLearning at OMAFRA.

Objectives

The evaluation was conducted to determine perceived knowledge transfer and to measure alignment with the business goals. Existing data and new data were collected and evaluated in order to answer six key questions. The questions are:

1. Are participants satisfied with the program? Does it meet their personal objectives for taking the course?

2. How is the online program being used by participants? What features are most valued and used most?

3. Does the program effectively teach the knowledge and skills specified in the learning objectives? If not, why not?

4. Do the knowledge and skills taught in the course change their nutrient management practices on the farm?

5. Have the business and management objectives of the online course been achieved? If not, why not?

6. How does the effectiveness and appropriateness of the online course compare with the classroom version of the program?

Methodology

In order to eliminate the possibility of bias in responses and data analysis, the evaluation was conducted by University of Guelph Ridgetown Campus and Gram Consulting.

Review and analysis of existing data:

• Online participant survey completed by 100 per cent of participants immediately upon completion of the course
• End of course online final quiz measuring the acquisition of knowledge and skills based on the learning objectives of the program
• End of course satisfaction surveys completed by participants in the classroom version of the program
• An evaluation report of the course by the Canadian Society of Training and Development

An online questionnaire was made available to every registered course participant: completed, incomplete, and expired. In total, the questionnaire was made available to 80 participants.

The online questionnaire in this study had a response rate of approximately 52 per cent, much higher than the typical response rate of 15 per cent generally used for online questionnaires.

Key Findings

The results of the study demonstrate that the online course successfully meets both OMAFRA business needs and participant learning needs. Although the program frustrated some users with dial up connection, taken as a whole, the results of the study confirm that eLearning is a highly appropriate strategy for OMAFRA, farmers, and consultants. The key findings include:

1. Participants are very satisfied with their learning experience and the convenience features offered by the program:
   
   
   • 86 per cent of participants would recommend the online course to other farmers or consultant
   • 79 per cent of participants were either satisfied or very satisfied with the support they received during completion of the program
     
     
2. Participants are learning the targeted skills in equal measure with the classroom learning participants:
   
   
   • 94 per cent of participants agreed or strongly agreed that the program "improved my understanding of nutrient management"
   • Self assessed learning gain is 68 per cent with classroom learning and 70.1 per cent with online learning. This similarity across the two versions suggests the programs are equally effective at producing the desired knowledge, skills, and confidence in participants.
     
     
3. The course has improved the reach of OMAFRA training and support to previously underserved areas of the province:
   
   
• 83 per cent of participants agreed or strongly agreed that the course was available for them when they needed it.
  
  
4. Positive participant reaction has been maintained over time:
   
   
   • Immediately upon completion of the course, 81 per cent of participants reported that they were satisfied or very satisfied with the program
   • In the follow-up survey, 83 per cent indicated they were satisfied or very satisfied with the program

Next Steps

OMAFRA will continue to offer the course in eLearning format as an alternative to the classroom format.

Results of the evaluation showed that there was room for improvement in the online course and recommendations were provided as to how to make this happen. These recommendations will be considered in the development of future eLearning courses.

• Add a social and collaborative element to the course
• Add scheduled events at defined points in the course
• Add program facilitators or coaches
• Include required group work
• Marketing and promotion

Having online courses available via CD/DVD would enhance the experience for users on dial-up connection.