Answer To: Page 1 of 5 ITECH XXXXXXXXXXSoftware Testing Assignment 1 Individual Research on Software Testing...
Deepti answered on May 06 2021
Applying the ISO 9126 model to the evaluation of an e-learning system
184
Applying the ISO 9126 model to the evaluation of an e-
learning system
Bee Bee Chua and Laurel Evelyn Dyson
Faculty of Information Technology
University of Technology, Sydney, AUSTRALIA
Despite the widespread use of e-learning systems and the considerable investment in purchasing
or developing them in house, there is no consensus on a standard framework for evaluating
system quality. This paper proposes the ISO 9126 Quality Model as a useful tool for evaluating
such systems, particularly for teachers and educational administrators. The authors demonstrate
the validity of the model in a case study in which they apply it to a commonly available e-
learning system and show how it can be used to detect design flaws. It is proposed that the
metric would be applicable to other e-learning systems and could be used as the basis for a
comparison to inform purchase decisions.
Keywords: e-learning, ISO, ISO 9126, Blackboard, online learning
Introduction
Most universities and colleges use e-learning systems to support face to face learning in the classroom or
to implement distance learning programmes. The growth of e-learning systems has increased greatly in
recent years thanks to the demand by students for more flexible learning options and economic pressures
on educational institutions, who see technology as a cost saving measure. Yet, there has been
considerable criticism of the quality of the systems currently being used. Problems include low
performance, poor usability, and poor customisability, which make it difficult to serve the specific needs
of different learners. Furthermore, online education has often been criticised as not supporting learner
centred education but replicating traditional face to face instruction (Vrasidas 2004).
Despite the widespread use of e-learning systems and the considerable investment in purchasing or
developing them in house, there is no consensus on devising a standard framework for evaluating system
quality in this area. The lack of an agreed e-learning system quality model is in stark contrast to the
extensive work on software quality assurance in general (Crosby 1979; Garvin 1984; Juran 1988; Norman
& Pfleeger 2002).
This paper proposes the ISO 9126 Quality Model (ISO 1991) as a useful tool for evaluating such systems.
The ISO 9126 model was developed by the International Organization for Standardisation (ISO) and is
one of a large group of internationally recognised standards applicable across a wide range of
applications. To date, ISO 9126 has not been applied extensively to the e-learning environment.
Nevertheless, the authors believe that it has potential to provide a useful evaluation tool: this belief
derives from the many years of industry experience that one of the researchers has had in software quality
assurance. Perspectives from this domain could provide insights relevant to e-learning educators. In this
paper we propose that the ISO 9126 model could be used as the basis for a comparison of e-learning
systems to inform decisions regarding review of existing systems and the purchase of new ones.
First of all, the paper examines the e-learning system literature and evaluates some of the software quality
tools and frameworks that have been proposed. Secondly, we introduce the ISO 9126 Quality Model as a
basis for evaluating e-learning tools and explain the characteristics and sub-characteristics of the model.
The main objective of our paper was to demonstrate how the model can be used to evaluate an e-learning
system. With this in mind, we chose a commonly used system, Blackboard, as a basis for our research
and adopted a case study approach. We applied the model to the system in the context of an Information
Technology subject in an undergraduate programme. In this paper, we summarise the results of the
evaluation of the system: generally, our results show the model is a good framework for assessing e-
learning systems, although we do identify several possible refinements to the model. Finally, we analyse
the implications of using the ISO 9126 Quality Model to evaluate and improve e-learning systems.
Chua & Dyson
185
E-learning system quality
Research into evaluating e-learning systems comes from two directions: the educationalists and the
software developers. Many educators have shown significant interest in the pedagogical evaluation of e-
learning systems, that is, in course design issues and how to promote good learning (Laurillard 1993;
Reeves 1992). Although these fundamental issues of course design are vital, these studies do not assist
educators in evaluating the quality of the system as such, and therefore do not incorporate frameworks to
support decision making regarding review of existing systems and the purchase of new ones.
There is also a vast body of literature relating to various technical frameworks for software developers
who wish to improve the quality of the e-learning systems they are developing. A systematic approach is
the IEEE Learning Technology Standard Committee (LTSC) reference model, IEEE P1484.1 LTSA. This
model has five layers, which focus on reusability and portability, and compares different e-learning
systems by numerical rating scales for various factors, e.g., assessment, administration, curriculum
development, etc. (O’Droma, M. S., Ganchev, I. & McDonnell, F. 2003). The Sharable Content Object
Reference Model (SCORM) is another widely known framework. It supports content compatibility, that
is the portability of content from one e-learning system to another and the re-usability of learning objects
by extensive cataloguing using metadata (Bohl, Schelhase, Sengler & Winand 2002). The Instructional
Management Systems (IMS) project is another approach to defining technical specifications in order to
promote interoperability between e-learning systems (IMS Global Learning Consortium). These standards
focus on technical aspects of e-learning systems and neglect the Human Computer Interaction (HCI)
component, that is, how the user will interact with the system. More importantly, they are too complicated
for the average educator or educational administrator to understand and apply when choosing an e-
learning system. They are specially designed for technical trained system developers.
The few studies that have been undertaken for educators and people working in educational institutions
who need to evaluate e-learning systems are often inadequate. This is due to the lack of systematic tools
or approaches. For example, Roberts (2002) gains a good overview of Blackboard using surveys, focus
groups and interviews, but the results are too general and do not provide detailed analysis of features such
as usability. In another study, the Learning and Teaching Technology Group (LTTG) undertook a
comparison of Blackboard and WebCT. Their main approach was an evaluation based on the number of
times students accessed different tools in the system, e.g. discussion board, group areas and others.
Nevertheless, these quantitative counts are not meaningful without details of the subject design, for
example how the group area activities were incorporated into the learning environment. The rest of their
paper offers a miscellaneous group of features for evaluation, e.g. data integration, pricing, hardware or
software platforms and ease of access. However, there is no system or justification for their choice of
features and many common usability criteria are omitted. A third approach we examined was “20
Questions”, which Driscoll and Dennehy (2002) propose putting to suppliers of the system. They resolve
the adoption of an e-learning system into two factors, organisational and technical, although only a few of
their questions deal with organisational issues and the main emphasis is on the technical issues, e.g. back
end integration and the partitioning of the system. Student interaction with the system receives very little
attention in their approach. Likewise, Parisotto (2003) focuses broadly on high level issues in evaluating
e-learning systems. He considers three organisational perspectives (academic, administrative and IT
support) but fails to discuss the operational levels, that is, the system in use.
The ISO 9126 model
The International Organization for Standardisation (ISO) was founded in 1946 in order to facilitate
international trade,...