Web-based Learning Systems and Assessment

Patrick Wegner, California State University Fullerton

Web-based systems have features that constitute a new learning environment for all involved: students, instructors, administrators and parents. The system is designed for use both in the classroom as well as at home. For the student it can provide continuous access to instructional materials and to learning and assessment tools. For the teacher it can provide automated assessment and real-time reports that yield information needed to refine and improve instruction. The system can individualize the learning environment at virtually any instructional level, by institution, by class or by student. It provides an immediate record of the learning status and progress of the students. Information needed to optimize the pace of instruction, to iterate topics and modify materials is readily available. At the institution level, it can provide greater instructional coherence among instructors. Beyond a specific school, it can provide greater coherence among institutions. The system has great potential to improve consistency in the instructional process and produce greater efficacy and efficiency in the learning process.

In this section we discuss the components of a Web-based system and some pertinent features. The elements of the system are shown in the schematic diagram.

The system is organized on client-server principles. The server side of the system contains three components, a database of learning materials, a database of assessment units that provides individualized automated assessment at any time and a database for learning analysis that collects and analyzes student performance data and automatically generates reports for the students and the instructor. Students and faculty enter the system using standard browsers anytime or anyplace they can connect to the Internet. For use in the classroom, standard PCs that have a good connection to the Internet are needed. Institutions can provide this access during class while outside of class, students can connect directly from home, dormitory or open computer laboratories. Thereby students extend their learning opportunities by accessing at any time exactly the same learning materials that they are using in the classroom. Note that this type of system addresses many of the issues that have hindered the adoption and use of instructional technology. Institutions don't have to purchase special software nor do they need staff to maintain the software or to set-up and manage servers. A single server can manage instruction for a number of courses at several different institutions.

The elements of the database of learning materials are shown in the schematic diagram.

Each of the categories listed is a database that contains learning materials that can be scheduled in a course. In some cases, for example, animations, tutorials and individualized assessment, these are sophisticated units developed with specialized software and simply stored in the database. In other cases such as explorations, examples and exposition, these are materials that multiple instructors can prepare and directly store in the database using browser-based systems. The instructional material development systems allow a group of faculty to create, edit and review simple instructional materials directly within a database. Once approved these materials can be made available for scheduling in any course using the learning system.

As constituted the system allows great customization. Each course is an independent entity that is accessible only to the students that are enrolled and the teacher. An instructor has full control over scheduling, choosing the instructional materials and the assessments to which the students have access. Via a browser, the instructor selects the assignments, specifies due dates and schedules the instructional materials. The instructor also specifies the grading parameters that are used for automatic grading and scoring.

The system provides a variety of automatic reports for both students and instructor that track student performance via the learning analysis database. Learning data are collected each time a student uses an assessment unit. Each use provides a unique (individualized) assessment. The learning analysis database stores these data and, using standard database query techniques, individual and class reports are automatically generated. Reports are automatically revised whenever any student does new work. The teacher sees the reports for any individual student as well as several reports of overall class performance. Students see only their own records. They are provided with a report indicating their progress compared to the assignments. Having these functions reside on an external server is useful and efficient. If a new report is created, or if new learning materials are added, or if an assessment unit is upgraded, the changes need be made only once on the server and then, they are immediately available to all who are using the system.

The assessment units are a key component of the system. These units provide an individualized assessment each time they are used. The units are graded based on parameters specified by the instructor. Students receive immediate feedback and use the units to determine their learning status and become self-activated learners. Information to improve and refine learning can be obtained from the assessment data that is collected both at large scale and high resolution.

The assessment units can be designed to address critical learning issues in science. These learning issues can be probed quantitatively and reproducibly in great detail for large numbers of students. The schematic shows the common elements of the assessment system.

Each unit is organized around a coherent unit of curriculum. Each unit is self-paced and provides individualized learning and assessment. Each assessment unit contains a content database that organizes the information and data pertinent to the curriculum in readily accessible form. The content database that organizes the information and data pertinent to the curriculum of the unit is structured to support the unit's question templates. Each unit is comprised of several question templates that contain variables that support the continuous generation of unique question sets. Once designed a question template always probes the same underlying content and learning issues. Each time a template is used it generates a unique question because the values of the variables in a template are refreshed from the content database. The learning analysis database captures the values of the variables, the calculated answer, the student answer and the time to answer the question ever time the template is used. These data are captured for every student in every class that is using the unit; hence, the scale is very large. For properly designed templates the value of the variables yields very detailed data regarding the state of a student's learning on a specific scientific concept or skill.

About 60 assessment units are available for chemistry instruction now. These units focus on the important conceptual and quantitative skills and processes that a student is expected to master for a specific unit of chemistry curriculum. The skills include the development and extension of quantitative proportional reasoning, molecular level visualization and analytical reasoning. Further the units concentrate on the different ways chemistry is represented textually, mathematically and graphically and requires the student to integrate these representations.

Let us examine a case involving a fundamental question in chemistry learning. Chemists represent a chemical substance in a number of different ways. Learning involves recognizing and integrating the different representations. As shown in the diagram we name a substance, establish its formula and determine its atom connectivity as a start to structure.

Visualizing the three dimensional structure of the substance represented by the name and formula is a critical component chemical learning. Experienced chemistry students make these connections reasonably well but beginning students do not, especially atom connectivity and structure. Our assessment units focus on establishing the relationships between the different representations. The central issue here is how to develop and assess molecular level visualization. Textbook representations are limited since they are immutable; they cannot change and regenerate at will. Computer-based molecular level displays that can be continuously renewed and displayed provide new opportunities for teaching, learning and assessing visualization. Figure I shows an example from such a unit. The unit displays covalent molecules in the solid liquid and gas state. Ionic substances are displayed as regular ionic solids with discrete ions. The unit contains a database of about 250 substances. In the example shown the liquid state of a molecular substance is represented as dense but disordered. The question displayed relates atom connectivity and formula. Requesting the name would have related atom connectivity and name. Note that other important molecular level information, although not the specific learning focus of this unit, also is conveyed. The three fluorine atoms are terminal atoms, connected to the central phosphorus atom but not to each other. The structure is three-dimensional (trigonal pyramidal) not flat.

Understanding the meaning of a chemical reaction at the molecular level is another important concept critical for learning chemistry. Figure II shows how this can be treated. The product side of this representation was blank until the question was answered. The template is addressing the concept of the balanced equation, the skill to use the coefficients of the equation and ultimately, the conservation of matter law. The rearrangement of atoms that defines a chemical reaction is clearly illustrated. A database of about 100 molecules and 125 chemical equations provide the individualization for this unit. Again additional information is presented. The carbon dioxide molecules are linear and the methane molecules tetrahedral. A common misconception among beginning students is that a phase change involves atom reorganization i.e.is a chemical reaction. If a phase change were represented in a parallel fashion to the reaction graphic, students would see that atoms had not rearranged thus highlighting the difference between a phase change and a chemical reaction.

To illustrate how systems of this type can probe mathematical and conceptual issues consider the concept of density and its application. This concept and its applications are taught and used at many levels. Review is warranted and useful even at the college level. Mastery of the concept requires the development and application of both qualitative and quantitative proportional reasoning skills. A well-designed assessment unit can probe the understanding of density in great detail. Consider the two question templates shown below. The bold items in brackets are variables that change with every use of the template. Individualization is provided by these variables and a database of about 90 solids and liquids with varying densities.

Template A: A sample of solid metal has a mass of {32.1} g. The volume of the sample was determined to be {13.6} mL. Calculate the density (g/mL) of the metal.

Template B: The total mass of a cup containing a sample of solid metal is {77. 4} g. The empty cup had a mass of {45.3} g. A measuring cylinder contained {15.3} mL of water. When the metal sample was placed in the cylinder the volume measured {28.9} mL. Calculate the volume that {45.7} g of the metal would occupy.

Template A evolves into template B via intermediate templates that involve transitions to at least six different learning levels that include the knowledge and application of concepts such as weighing by difference, liquid displacement, mathematical manipulation of the density definition and using data from two different sources. The status of a student's understanding of these concepts can be tracked in exquisite detail by examining the intermediate templates.

Qualitative understanding can be tracked using a set of different templates. A template that displays four clearly identical figures (equal volume) such as cubes or cylinders and also supplies the density of the substance in each figure would be presented. The student would be asked to rank order the mass of the figures or select the highest or lowest mass. The combination of qualitative and quantitative templates would provide a comprehensive assessment of how well the density concept is understood and applied. The results of such studies could be used to improve instructional methods in areas that were shown to be most problematic for students.

The titanium Web site (http://titanium.fullerton.edu/mcweb/) is a working example of a Web-based learning system. These systems such are not theoretical constructs. The titanium system is robust, relatively easy to use and very scalable. Over 5000 students at more than 25 different institutions encompassing four-year universities and colleges, two-year colleges and high schools have used the system this past year as a graded component of regularly scheduled chemistry instruction. At the University of Iowa, the University of California, Irvine (UCI) and UCLA, the learning system has been effective in very large classes enrolling between 350 and 650 students. At UCI, 1595 students used the system in a recent term, which corresponds to about 50-60 individual classes at the high school level. UCI students indicated that they most often used the program from home (36%), their dorm rooms (34%) and open campus computer labs (30%).

Web-based learning systems are just beginning to evolve. As they become more developed and sophisticated, they will become an important tool at every level of education. The ability to immediately identify a students learning status as well as the ability to rapidly gather data to improve and refine instruction is compelling as is the ability to have greater consistency in instruction. Ultimately these systems will inform both learning and teaching.