Will Reader and Nick Hammond
ABSTRACT
Although hypertext can be a useful way of delivering learning materials it is problematic in that there is no requirement to actively manipulate knowledge as part of the interaction. The mindtools, or concept mapping, approach requires learners to construct graphical representations of the information that they cover forcing them to engage more actively in the information as a consequence. Advocates of concept mapping argue that by encouraging learners to represent their knowledge using a node-link formalism, learners are forced into activities that aid the organisation and integration of knowledge, and that the map itself can serve to communicate the learner's knowledge more effectively than text. This paper reports the results of a study designed to test the effectiveness of a concept mapping tool in aiding student learning from a hypertext system. It was found that use of the concept mapping tool enhanced the scores on a post-test when compared to standard note taking. Qualitative analyses of the process of network construction, and of the maps produced argue that there may be a case for offering learners more support in the networks that they construct, to explicitly encourage structuring activities and it is hoped integration, and more encouragement to revise networks so to enhance their use in communication.
There are however a number of problems with hypertext that are specifically relevant to its use as an educational medium. It is well known that in order to promote effective learning learners must actively engage in the information that they are reading, processing the material in an elaborate manner [1,2]; however hypertext is basically a passive medium and there is no requirement that learners do this. One of the methods that has been adopted by developers of hypertext CAL systems has been to provide the learners with batteries of questions, usually in the form of a quiz, that encourage the learner test themselves and thus promote more effective learning [3]. Such approaches fall short of the ideal because the nature of hypertext means that the questions may not address the information that the learner wishes to learn, resulting in redundancy. This problem also applies to printed texts, which are also by nature passive entities, but in the case of hypertext it is compounded by the instability of the discourse; because learners can follow many different paths through the same information, hypertext will often not have the coherence provided by paper-based texts. As Charney points out [4], hypertext cannot rival the ability of print to support rational, deductive, goal-directed discourse. This lack of coherence means that learners may often fail to obtain an overview of the way that the information fits together to form a whole [3]. It therefore seems that learners may need support whilst learning from hypertext, first to encourage them to be more active processors of the material, and secondly to help them to form more coherent representations of the information. An approach such as that offered by mindtools or concept mapping tools may offer help in both these areas.
There are a number of claims as to why concept mapping may be effective in aiding learning which can be divided into claims about the process of concept map construction and claims about the product, that is the completed map.
Claims about the process of map construction tend to view concept mapping as a problem-solving exercise [6], the various restrictions built in to the tool forcing or encouraging the learners to engage in these activities more effectively than if they were not using the tool. Although concept mapping tools vary as to the degree to which they support various learning activities, by and large most of them encourage the learner to organise and structure their knowledge. Organising is where the learner places a loose structure on the material, clustering concepts that are similar, and perhaps specifying what these concepts have in common. Structuring is where learners make explicit the relationships between concepts, to provide a coherent, integrated network. It is proposed that the explicit activities of organising and structuring can, in turn, have implications for the organisation and integration of new knowledge into schemata.
Other claims indicate that the map itself can have educational benefits. Kozma [7] argues that since the network is at the level of the overview it may act as a more efficient aid to memory than normal text-based notes, freeing up short term memory and allowing students to devote more effort to performing meaningful activities. Other theorists such as Novak and Gowin [5] emphasise the role that concept mapping can play in communicating the learner's knowledge. Ideas may be communicated to an instructor, to other learners, aiding collaborative learning or to the same learner over time. It is argued that because concept maps require more explicit representation of knowledge than does text, ideas can be communicated more readily then text. This is one of the reasons why formalisms such as systemic grammars and semantic networks have been used as intermediate or mediating representations in knowledge engineering [9], [10].
Constructing concept maps therefore not only encourages the learner to engage with the material more actively, it also requires them to assemble overviews of the material covered, both of which are claimed to be useful activities for use in learning from hypertext.
_
Figure 1: the concept mapping tool used in this experiment, containing a well structured network.
A hypertext system was also developed to be used as a learning resource, the system was a HyperCard stack containing approximately 50 cards on the topic of the Callanish stone circle. This was chosen as a suitable domain both because it would be novel to most subjects, thereby reducing the effects of prior knowledge to a minimum, and also because the information contained a large number of different types of relationships, concerned with the structure of the circle, its function, the people who built it and so forth. The system contained most of the standard hypertext access facilities with the exception of a map as it was thought that learners may merely copy the map when constructing a concept map.
In order to measure learning, a test was developed which consisted of a series of questions that related to information relevant to the study goal. There were 16 questions, with a maximum score of 44. The questions were of the short answer format, and required different types of information to answer them, some were factual, whilst some were more relational in content (see results section).
One possible explanation for the substantial difference between conditions is that for some reason subjects in the concept mapping condition spent more time reading the material in the hypertext system than did those in the note tool condition and thus retained information more effectively, relevant data are shown in table 1. Table 1 shows that whilst concept mapping subjects spent longer on the experiment as a whole, concept mapping being inherently more time consuming then note taking, subjects using the concept mapping tool actually spent less time looking at the material screens (time is in minutes).
Material Access Tool Total screens facilities Concept 17.4 mins. 3.8 mins. 32 mins. 53 mins. mapping Note taking 17.3 mins. 3.1 mins. 14 mins. 35 mins.Table 1: mean percentage times on the various facilities available to the subjects.
We can further ask whether the concept mapping tool had any qualitative effect on the sort of knowledge that subjects learned. It could be argued that since concept mapping places an emphasis on the relational aspects of the domain, then it may help in the learning of this type of information over an above any general effects on learning. In order to test this, one of the questions (with a maximum score of 18) in the post test required learners to draw an aerial diagram of the layout of the stone circle main site. There was no aerial diagram in the material contained within the system, and it seemed that concept mapping may focus learners on the relationships between the components of the circle that made up the main site, helping them to produce a better diagram than the note tool subjects. Analyses of the results reveal that the scores for the relational questions when expressed as a percentage of the total score were in fact very close: 31% for concept mapping subjects and 29% for note tool subjects. Thus although subjects using the concept mapping tool scored higher than note tool subjects on the relational question, the degree to which performance was enhanced was no greater than that for factual questions.
Figure 2: a network from the study showing spatial clustering, superordinate link names
The final and most precise from of organising observed was executed by linking concepts together and providing a label for what the objects have in common. Note that this is not structuring since there is apparently no attempt by the subject to try to integrate the concepts by specifying an explicit relationship. In figure 2 the three link names relate to commonalities between the linked concepts. A number of different types of structural relationships were created, but these were too varied to derive any useful analytical scheme from them. Overall there were fewer structural links than organisational links created (4.1 compared to 4.9 on average).
Figure 3: a network showing the use of nameless (in this case arbitrarily named) links.
Assuming that the end product of concept mapping should be a well structured concept map, it was interesting to observe that only one of the eight subjects achieved anything like this objective. Often concepts, some of which had been spatially organised, remained unlinked (see figure 2), concepts that had been linked using organisational links often remained so, with little attempt by subjects to specify them as structural relationships. This last point may be explained by subjects' apparent reticence to revise their maps; links that had been used for organising would have to be deleted, or at the very least renamed, if they are to be replaced by structural links. Subjects made on average 1.6 revisions to the maps that they created. The fact that students revised their maps very little may have other implications; it is known that during the process of learning a individuals understanding of the domain will change, often resulting in a degree of restructuring of knowledge [11], [12]. If concept maps are supposed to be relatively faithful representations of the learner's understanding, and their role in communication suggests that they should be, then they should also be restructured accordingly as this understanding changes.
In truth any attempt to evaluate computer-based tools is always going to encounter problems as to how well the results can be generalised to other tools, tasks, materials and subjects. This study should therefore be seen as contributing information to a growing corpus of data which can inform our understanding of issues. On this point there is some evidence that problems analogous to those encountered in this study occur in real-world situations, [13, 14] and it is likely that they are not simply artefacts of this particular study. The issues mentioned above have implications for the effectiveness of concept mapping tools in general. One of the assumptions behind tools such as SemNet is that forcing learners to link concepts encourages them to think about structural relationships. The results of this study indicate that simply requiring learners to link is not enough as links can be used for purposes other than structuring such as organising. Additionally all concept mapping tools seem likely to suffer from the problems caused by the reticence of learners to impose structure on knowledge and restructure extant networks that they have created, simply because they require learners to use semi-formal representations. Shipman and Marshall [13] discuss a number of cases where learners apparently fail to organise and structure effectively in areas as disparate as design and office filing. It seems that the desire of individuals not to commit themselves to a structured representation prematurely may be a major stumbling block for concept mapping. On the topic of restructuring Fischer [14] states that: "Despite the fact that in many ways users could think of better structures, they stick to inadequate structures, because the effort to change existing structures is too large."
These problems may be more easily addressed in tools intended for learning than for other uses such as design. Whilst expert designers may, quite rightly, find the process of restructuring their networks to be counter productive, restructuring and revising for learning may itself be a useful learning experience.
References
[1] Anderson, J. R. (1990) Cognitive psychology and its implications. (3rd Ed.) San Franscisco, CA: Freeman.
[2] Frase, L.T. (1975). Prose processing. In Bower, G.H. (ed.) The Psychology of Learning and Motivation, Vol. 9. Academic Press, New York.
[3] Hammond, N.V. & Allinson, L. (1989) Extending hypertext for learning: an investigation of access and guidance tools. in A. Sutcliffe & L. Macaulay, (eds.) People and Computers V, Cambridge University Press, 293-304.
[4] Charney, D. (1991) The impact of hypertext on processes of reading and writing. In C. Selfe & S. Hilligoss, (eds.) Literacy and Computers, Modern Language Association, New York.
[5] Novak, J.D. & Gowin, D.B. (1984). Learning How to Learn., Cambridge University Press, Cambridge, England.
[6] Fisher, K.M. (1990). Semantic Networking: the new kid on the block. Journal of Reasearch in Science Teaching, 27, (10), 1001-1018.
[7] Kozma, R.B. & Van Roekel, J. (1986). Learning Tool. Santa Barbara, CA: Intellimation.
[8] Kozma, R.B. (1992) Constructing knowledge with learning tool. In P.A.M. Kommers, D.H. Jonassen, & J.T. Mayes, (eds.), Mindtools: Cognitive Technologies for Modelling Konwledge. Springer-Verlag, Berlin.
[9] Bliss, J. & Ogborn, J. (1979) The Analysis of Qualitative Data. European Journal of Science Education, 1. (4), 427-440
[10] Johnson, N.E. (1988) Mediating Representations in Knowledge Engineering. In D. Diaper,. (ed.) Knowledge Elicitation, Academic Press: London, pp 179-193.
[11] White, B.Y. & Frederiksen, J.R. (1990) Causal model progressions as a foundation for intelligent learning environments. Artificial Intelligence, 42, 99-157.
[12] Rumelhart, D.E. & Norman, D.A.(1978) Accretion, tuning and restructuring: three modes of learning. In J.W. Cotton & R.L. Klatzky, (eds.) Semantic Factors in Cognition. Lawrence Erlbaum Associates. Hillsdale, New Jersey.
[13] Shipman, F.M. & Marshall, C.C. (1993) Formality condidered harmful: experiences, emerging themes, and directions. Submitted to InterChi `93.
[14] Fischer, G. (1988) A critical assessment of hypertext systems. Panel Session in Proceedings of CHI `88: Human Factors in Computing Systems, 223-227. ACM: New York.
[15] Lee, J. & Lai, K. (1991) What's in design rationale? Human-Computer Interaction, 6 (3 & 4), 251-280.
[16] Schuler, W. & Smith, J. B. (1990) Author's Agumentation Assistant (AAA): a hypertext-based authoring tool for argumentative texts. In A. Rizk, N. Streitz & J. André (eds.) Hypertext: Concepts, systems and applications. Proceedings of the European Conference on hypertext. INRIA, France, November. Cambridge University Press.
[17] Streitz, N. A. & Hannemann, J. (1990) Elaborating arguments: writing, learning and reasoning in a hypertext based environment for authoring. In D.H. Jonassen. & H. Mandl (eds.) Designing Hypermedia for Learning. NATO ISI Series, Springer-Verlag, Berlin.