COMPUTER-BASED TOOLS TO SUPPORT LEARNING FROM HYPERTEXT: CONCEPT MAPPING TOOLS AND BEYOND

Will Reader and Nick Hammond

ABSTRACT

Although hypertext can be a useful way of delivering learning materials it is problematic in that there is no requirement to actively manipulate knowledge as part of the interaction. The mindtools, or concept mapping, approach requires learners to construct graphical representations of the information that they cover forcing them to engage more actively in the information as a consequence. Advocates of concept mapping argue that by encouraging learners to represent their knowledge using a node-link formalism, learners are forced into activities that aid the organisation and integration of knowledge, and that the map itself can serve to communicate the learner's knowledge more effectively than text. This paper reports the results of a study designed to test the effectiveness of a concept mapping tool in aiding student learning from a hypertext system. It was found that use of the concept mapping tool enhanced the scores on a post-test when compared to standard note taking. Qualitative analyses of the process of network construction, and of the maps produced argue that there may be a case for offering learners more support in the networks that they construct, to explicitly encourage structuring activities and it is hoped integration, and more encouragement to revise networks so to enhance their use in communication.

1. INTRODUCTION

Hypertext is suggested as a useful medium for the development of CAL systems because it can act as an educational resource that supports independent learning. The basic hypertext philosophy is that learners forge their own paths through the richly interconnected information-base in a self-directed manner, assembling the course materials in accordance with their educational goals, rather than having to slavishly follow some form of linear tutorial.

There are however a number of problems with hypertext that are specifically relevant to its use as an educational medium. It is well known that in order to promote effective learning learners must actively engage in the information that they are reading, processing the material in an elaborate manner [1,2]; however hypertext is basically a passive medium and there is no requirement that learners do this. One of the methods that has been adopted by developers of hypertext CAL systems has been to provide the learners with batteries of questions, usually in the form of a quiz, that encourage the learner test themselves and thus promote more effective learning [3]. Such approaches fall short of the ideal because the nature of hypertext means that the questions may not address the information that the learner wishes to learn, resulting in redundancy. This problem also applies to printed texts, which are also by nature passive entities, but in the case of hypertext it is compounded by the instability of the discourse; because learners can follow many different paths through the same information, hypertext will often not have the coherence provided by paper-based texts. As Charney points out [4], hypertext cannot rival the ability of print to support rational, deductive, goal-directed discourse. This lack of coherence means that learners may often fail to obtain an overview of the way that the information fits together to form a whole [3]. It therefore seems that learners may need support whilst learning from hypertext, first to encourage them to be more active processors of the material, and secondly to help them to form more coherent representations of the information. An approach such as that offered by mindtools or concept mapping tools may offer help in both these areas.

2. CONCEPT MAPPING TOOLS

Novak [see 5] was one of the first advocates of concept mapping who used them as an aid to meaningful learning. A concept map is, at its simplest, a graphical representation of domain material generated by the learner in which nodes are used to represent domain key concepts, and links between them denote the relationships between these concepts. The putative usefulness of concept maps has lead to the development of computer-based tools that allow learner to create graphical representations of knowledge using semantic net type representations, such as SemNet [6] and Learning Tool [7].

There are a number of claims as to why concept mapping may be effective in aiding learning which can be divided into claims about the process of concept map construction and claims about the product, that is the completed map.

Claims about the process of map construction tend to view concept mapping as a problem-solving exercise [6], the various restrictions built in to the tool forcing or encouraging the learners to engage in these activities more effectively than if they were not using the tool. Although concept mapping tools vary as to the degree to which they support various learning activities, by and large most of them encourage the learner to organise and structure their knowledge. Organising is where the learner places a loose structure on the material, clustering concepts that are similar, and perhaps specifying what these concepts have in common. Structuring is where learners make explicit the relationships between concepts, to provide a coherent, integrated network. It is proposed that the explicit activities of organising and structuring can, in turn, have implications for the organisation and integration of new knowledge into schemata.

Other claims indicate that the map itself can have educational benefits. Kozma [7] argues that since the network is at the level of the overview it may act as a more efficient aid to memory than normal text-based notes, freeing up short term memory and allowing students to devote more effort to performing meaningful activities. Other theorists such as Novak and Gowin [5] emphasise the role that concept mapping can play in communicating the learner's knowledge. Ideas may be communicated to an instructor, to other learners, aiding collaborative learning or to the same learner over time. It is argued that because concept maps require more explicit representation of knowledge than does text, ideas can be communicated more readily then text. This is one of the reasons why formalisms such as systemic grammars and semantic networks have been used as intermediate or mediating representations in knowledge engineering [9], [10].

Constructing concept maps therefore not only encourages the learner to engage with the material more actively, it also requires them to assemble overviews of the material covered, both of which are claimed to be useful activities for use in learning from hypertext.

3. STUDY: EVALUATING CONCEPT MAPPING TOOLS IN LEARNING FROM HYPERTEXT

In order to evaluate the effectiveness of concept mapping tools in learning from hypertext a study was performed with two aims in mind: first to see if the use of concept mapping had any effect on learning when compared to standard note taking, and secondly to assess some of the issues in using concept mapping as a learning activity.

3.1. Method

3.1.1. Materials

The concept mapping tool that was used was developed for the purpose of the experiment using HyperCard. Concepts were represented by small page icons which could be named by the subject to denote particular domain concepts. Each node contained a pop-up text field that allowed the subject to make notes; multiple text fields can be opened at any one time. Concepts could be linked using either typed or untyped links to represent the relationships between the concepts. Concepts and links could be renamed and deleted as the subject saw fit (see figure 1).

_

Figure 1: the concept mapping tool used in this experiment, containing a well structured network.

A hypertext system was also developed to be used as a learning resource, the system was a HyperCard stack containing approximately 50 cards on the topic of the Callanish stone circle. This was chosen as a suitable domain both because it would be novel to most subjects, thereby reducing the effects of prior knowledge to a minimum, and also because the information contained a large number of different types of relationships, concerned with the structure of the circle, its function, the people who built it and so forth. The system contained most of the standard hypertext access facilities with the exception of a map as it was thought that learners may merely copy the map when constructing a concept map.

In order to measure learning, a test was developed which consisted of a series of questions that related to information relevant to the study goal. There were 16 questions, with a maximum score of 44. The questions were of the short answer format, and required different types of information to answer them, some were factual, whilst some were more relational in content (see results section).

3.1.2. Subjects

Sixteen male and female students from the University of York took part in the experiment.

3.1.3. Design

The study was a simple two condition experiment: a concept mapping condition requiring the use of the concept mapping tool, and a notes only condition in which subjects were provided with a tool that allowed them to make on-screen notes. The principal dependent variable was the score on a post-test.

3.1.4. The learning task

The task that subjects were given was designed to simulate fairly closely a real world hypertext interaction under controlled experimental conditions. Subjects were given a study goal which did not directly map onto any of the access facilities present in the hypertext; this was so that they would have to browse the material rather than being able to access it directly. All subjects were told that they were to learn the material in order to answer a series of questions at the end of the experiment. Additionally subjects in the concept mapping condition were told to create a concept map that represented the concepts and relations of the domain information, whilst subjects in the note tool condition were told to use the note tool in order to record any useful information that they might encounter. Thus, apart from the type of tool that subjects used, the only difference between the two conditions was the instruction to create a concept map. The experiment was open ended, in that subjects were allowed as long as they felt they needed to complete the task, but they were told that it should not normally take more than one and a half hours. The purpose of this was to allow for individual differences in learning style, reading speed, familiarity with the Macintosh interface and so on, and yet still try and establish some form of baseline for the length of the interaction.

3.1.5. Procedure

The experiment was divided into two sessions: a practice session and an experimental session. The practice session allowed subjects to become familiar with hypertext, using a system that was structurally similar to the experimental system; and allowed them time to practice using whichever tool they would be using in the experimental session. The hypertext system used for this session contained different material to the one in the experimental session, but had similar facilities and structure. Familiarisation took the form of an introductory tutorial followed by a task similar to the one that they would perform in the experimental session. When subjects felt comfortable with the two systems they were allowed to continue to the experimental session. At the beginning of the experimental session subjects were given their study goal, and the instructions for use of the tool, following this they were allowed to use the systems in any way they felt fit. When subjects felt that they had fulfilled their goals they were asked if they would like a further five minutes for revision purposes, after this they were given the questions to answer.

3.2. Quantitative results

The results of the post-test revealed differences in scores between the two condition, with subjects in the concept mapping condition obtaining a higher score than the note tool subjects (means of 47.9% for concept mapping and 24.3% for the note condition). A one way analysis of variance revealed this result to be significant to the 98% level, (F [1,14] = 8.53, p < 0.02). The standard deviations (17.8 for the concept mapping condition, 14.4 for the note tool condition) show a high degree of between subject variability in the scores. This was expected given that there are many variables that effect performance in addition to the independent variable.

One possible explanation for the substantial difference between conditions is that for some reason subjects in the concept mapping condition spent more time reading the material in the hypertext system than did those in the note tool condition and thus retained information more effectively, relevant data are shown in table 1. Table 1 shows that whilst concept mapping subjects spent longer on the experiment as a whole, concept mapping being inherently more time consuming then note taking, subjects using the concept mapping tool actually spent less time looking at the material screens (time is in minutes).

                   Material     Access          Tool            Total           
                   screens      facilities                                      
       Concept    17.4 mins.    3.8 mins.       32 mins.        53 mins.        
       mapping                                                                  
   Note taking    17.3 mins.    3.1 mins.       14 mins.        35 mins.        

Table 1: mean percentage times on the various facilities available to the subjects.

We can further ask whether the concept mapping tool had any qualitative effect on the sort of knowledge that subjects learned. It could be argued that since concept mapping places an emphasis on the relational aspects of the domain, then it may help in the learning of this type of information over an above any general effects on learning. In order to test this, one of the questions (with a maximum score of 18) in the post test required learners to draw an aerial diagram of the layout of the stone circle main site. There was no aerial diagram in the material contained within the system, and it seemed that concept mapping may focus learners on the relationships between the components of the circle that made up the main site, helping them to produce a better diagram than the note tool subjects. Analyses of the results reveal that the scores for the relational questions when expressed as a percentage of the total score were in fact very close: 31% for concept mapping subjects and 29% for note tool subjects. Thus although subjects using the concept mapping tool scored higher than note tool subjects on the relational question, the degree to which performance was enhanced was no greater than that for factual questions.

3.3. Discussion of quantitative results

From the results of the study outlined it seems that concept mapping tools can have a positive effect on learning from hypertext, although the mechanism of the effect is not forthcoming from the results themselves. It does appear that concept mapping aids the acquisition of both relational and factual knowledge, rather than the effect being localised on the acquisition of relational knowledge. In some senses this distinction may be erroneous: it may be argued that concept mapping does exert its effect on the acquisition of relational knowledge, but that this has a knock-on effect which aids the acquisition of factual and other sorts of knowledge. It is well known in psychology that having a well-developed schema for a particular domain allows factual knowledge to be accommodated more readily than if the schema is poorly developed, and it could be that concept mapping by placing an emphasis on the construction of relational networks, is aiding the development of domain schemata. It is, however, impossible to ascertain from the results obtained whether this is a plausible account, and it seems to make sense to adopt the more parsimonious explanation that the effect is probably more prosaic than this. Perhaps concept mapping, at least in the early stages of learning as measured here, serves to focus the learners' attention on particular items of information by forcing them to think about concepts and relationships more than they might do when simply taking notes. A possible confound is that using a novel tool such as the concept mapping tool may be intrinsically motivating, resulting in learners engaging in the task more than if they were simply using a note tool. Although it seems unlikely that novelty alone would account for the sizeable difference between conditions, it is difficult to escape entirely from such criticisms; only further studies using learners familiar with concept mapping tools could address these issues directly. It is, however, worth pointing out that in this study subjects used a number of novel facilities such as the hypertext system itself, and even the on-screen note-cards were new to a number of students. The novelty factor also has a down side: learners may have found the concept mapping tool difficult to use, impeding the creation of useful notes and thus resulting in poorer learning.

3.4. Qualitative results

In addition to being interested in the effects of concept mapping on learning, we were also concerned with the way that the actual tool was used. In order to evaluate this, screen recordings were made which could allow us to observe the process of map construction. In the introduction it was indicated that concept mapping tools may be useful because they encourage learner to engage in activities of organising and structuring in an explicit way. Recall that organising is where the learner starts to group and partition information that seems to share something in common, whilst structuring is where learners identify specific relationships that appear to exist between items of information or concepts. Using the screen recordings it was possible to observe all of the on-screen actions that the subjects performed. These were categorised in an attempt to enumerate the activities that seemed to be aimed at structuring and those that were organisational in nature. One of the loosest forms of organising that was observed was the use of spatial clustering to denote commonalities among concepts. Here subjects would form groups of concepts in different parts of the screen. Figure 2 shows evidence of spatial clustering, with three clusters denoting information relating to the structure of the stone circle (middle left), its function (top right) and its builders (bottom right). A more specific from of organising is by using nameless links to declare an, as yet, unqualified relationship between concepts. Figure 3 shows an example of this where, in the main, numbers are used to label the relationships.

Figure 2: a network from the study showing spatial clustering, superordinate link names

The final and most precise from of organising observed was executed by linking concepts together and providing a label for what the objects have in common. Note that this is not structuring since there is apparently no attempt by the subject to try to integrate the concepts by specifying an explicit relationship. In figure 2 the three link names relate to commonalities between the linked concepts. A number of different types of structural relationships were created, but these were too varied to derive any useful analytical scheme from them. Overall there were fewer structural links than organisational links created (4.1 compared to 4.9 on average).

Figure 3: a network showing the use of nameless (in this case arbitrarily named) links.

Assuming that the end product of concept mapping should be a well structured concept map, it was interesting to observe that only one of the eight subjects achieved anything like this objective. Often concepts, some of which had been spatially organised, remained unlinked (see figure 2), concepts that had been linked using organisational links often remained so, with little attempt by subjects to specify them as structural relationships. This last point may be explained by subjects' apparent reticence to revise their maps; links that had been used for organising would have to be deleted, or at the very least renamed, if they are to be replaced by structural links. Subjects made on average 1.6 revisions to the maps that they created. The fact that students revised their maps very little may have other implications; it is known that during the process of learning a individuals understanding of the domain will change, often resulting in a degree of restructuring of knowledge [11], [12]. If concept maps are supposed to be relatively faithful representations of the learner's understanding, and their role in communication suggests that they should be, then they should also be restructured accordingly as this understanding changes.

3.4.1. Accounting for these results

These problems may in some ways be artefacts of the study itself. First the session may have been too short, if subjects were given more time, or more demanding tasks then they may structure and revise their networks more than they did here. Second, because the task had no importance to the subjects outside of the experiment, it may mean that they were prepared to make do with inadequate maps more than they might do if the task was considered important, for example if it was a course requirement. Third, subjects only had limited experience with concept mapping tools. None of the subjects had ever used such a tool before, and it may not be too surprising that they produced maps that were perhaps not as well structured as they might have been. Novak and Gowin [5] argue that in order for concept mapping to be a truly effective learning activity, learners must put in many hours of practice constructing networks. Given the limited experience subjects had using the tool the results were surprisingly good (see figure 1, for an example). Fourth, since the material itself was novel, and in many ways not `real' material, this may also have implications for the way that networks were constructed.

In truth any attempt to evaluate computer-based tools is always going to encounter problems as to how well the results can be generalised to other tools, tasks, materials and subjects. This study should therefore be seen as contributing information to a growing corpus of data which can inform our understanding of issues. On this point there is some evidence that problems analogous to those encountered in this study occur in real-world situations, [13, 14] and it is likely that they are not simply artefacts of this particular study. The issues mentioned above have implications for the effectiveness of concept mapping tools in general. One of the assumptions behind tools such as SemNet is that forcing learners to link concepts encourages them to think about structural relationships. The results of this study indicate that simply requiring learners to link is not enough as links can be used for purposes other than structuring such as organising. Additionally all concept mapping tools seem likely to suffer from the problems caused by the reticence of learners to impose structure on knowledge and restructure extant networks that they have created, simply because they require learners to use semi-formal representations. Shipman and Marshall [13] discuss a number of cases where learners apparently fail to organise and structure effectively in areas as disparate as design and office filing. It seems that the desire of individuals not to commit themselves to a structured representation prematurely may be a major stumbling block for concept mapping. On the topic of restructuring Fischer [14] states that: "Despite the fact that in many ways users could think of better structures, they stick to inadequate structures, because the effort to change existing structures is too large."

These problems may be more easily addressed in tools intended for learning than for other uses such as design. Whilst expert designers may, quite rightly, find the process of restructuring their networks to be counter productive, restructuring and revising for learning may itself be a useful learning experience.

4. Future directions: using argumentation to augment learning.

The results of this study indicates that although concept mapping may have some positive effect on learning, the qualitative analysis of the results indicates that students may need more support in the formation of structural relationships, and encouragement to revise maps. In addition to providing a more useful process, this may also result in maps that communicate the learner's ideas in a more structured and efficient way. It is not sufficient simply to force learners to link concepts because, as we have seen, linking can be used for things other than providing structural relationships. What seems to be required is a jigsaw type method in which pre-defined concept and link types are provided, and it is the learner's task to fit their knowledge or incoming information within this framework. In providing such as facility it is important that the learner is not provided with so much support that they become straight-jacketed. A possible compromise is that learners be given a framework at a high enough level of abstraction to allow flexibility of representation. Argumentation is a possible method and one that has been used in a number of areas of application such as design [15], the production of argumentative texts [16], and also learning [17].

References

[1] Anderson, J. R. (1990) Cognitive psychology and its implications. (3rd Ed.) San Franscisco, CA: Freeman.

[2] Frase, L.T. (1975). Prose processing. In Bower, G.H. (ed.) The Psychology of Learning and Motivation, Vol. 9. Academic Press, New York.

[3] Hammond, N.V. & Allinson, L. (1989) Extending hypertext for learning: an investigation of access and guidance tools. in A. Sutcliffe & L. Macaulay, (eds.) People and Computers V, Cambridge University Press, 293-304.

[4] Charney, D. (1991) The impact of hypertext on processes of reading and writing. In C. Selfe & S. Hilligoss, (eds.) Literacy and Computers, Modern Language Association, New York.

[5] Novak, J.D. & Gowin, D.B. (1984). Learning How to Learn., Cambridge University Press, Cambridge, England.

[6] Fisher, K.M. (1990). Semantic Networking: the new kid on the block. Journal of Reasearch in Science Teaching, 27, (10), 1001-1018.

[7] Kozma, R.B. & Van Roekel, J. (1986). Learning Tool. Santa Barbara, CA: Intellimation.

[8] Kozma, R.B. (1992) Constructing knowledge with learning tool. In P.A.M. Kommers, D.H. Jonassen, & J.T. Mayes, (eds.), Mindtools: Cognitive Technologies for Modelling Konwledge. Springer-Verlag, Berlin.

[9] Bliss, J. & Ogborn, J. (1979) The Analysis of Qualitative Data. European Journal of Science Education, 1. (4), 427-440

[10] Johnson, N.E. (1988) Mediating Representations in Knowledge Engineering. In D. Diaper,. (ed.) Knowledge Elicitation, Academic Press: London, pp 179-193.

[11] White, B.Y. & Frederiksen, J.R. (1990) Causal model progressions as a foundation for intelligent learning environments. Artificial Intelligence, 42, 99-157.

[12] Rumelhart, D.E. & Norman, D.A.(1978) Accretion, tuning and restructuring: three modes of learning. In J.W. Cotton & R.L. Klatzky, (eds.) Semantic Factors in Cognition. Lawrence Erlbaum Associates. Hillsdale, New Jersey.

[13] Shipman, F.M. & Marshall, C.C. (1993) Formality condidered harmful: experiences, emerging themes, and directions. Submitted to InterChi `93.

[14] Fischer, G. (1988) A critical assessment of hypertext systems. Panel Session in Proceedings of CHI `88: Human Factors in Computing Systems, 223-227. ACM: New York.

[15] Lee, J. & Lai, K. (1991) What's in design rationale? Human-Computer Interaction, 6 (3 & 4), 251-280.

[16] Schuler, W. & Smith, J. B. (1990) Author's Agumentation Assistant (AAA): a hypertext-based authoring tool for argumentative texts. In A. Rizk, N. Streitz & J. André (eds.) Hypertext: Concepts, systems and applications. Proceedings of the European Conference on hypertext. INRIA, France, November. Cambridge University Press.

[17] Streitz, N. A. & Hannemann, J. (1990) Elaborating arguments: writing, learning and reasoning in a hypertext based environment for authoring. In D.H. Jonassen. & H. Mandl (eds.) Designing Hypermedia for Learning. NATO ISI Series, Springer-Verlag, Berlin.