wairua » publications » evaluation of a web-based teaching session

• Introduction
• Participant Feedback
• Feedback on Session
• Comments
• Educational Philosophy
• Session Structure
• Lesson Format
• Web-based Tutorial
• Conclusion
• References
• About the Authors
• Appendices
• Appendix A - Student Handout
• Appendix B - Lesson Plan
• Appendix C - Web Site
• Appendix D - Sample Pages

Introduction

This paper evaluates the teaching session "Building a home network", which was presented on Saturday November 11, 2000. The session format was that of an introduction, which included casually assessing the group's level of subject knowledge and stating the learning outcomes. The first part of the session was a presentation using a slide show, aimed at introducing the audience to the basic concepts of networking. This included some examples of networking components to assist participant familiarity with the type of equipment used. The second part of the session was a self-paced tutorial delivered via the Internet. This also contained a demonstration question and answer component that was designed to allow the participants to build a specification for their own home network. Although the two sessions were run concurrently, the design principles behind this session included the use of the web site at a later time to re-enforce learning in the classroom.

In this paper, we will firstly review the participant's feedback from the session and then evaluate the educational philosophy and value of the session. Finally, we will identify the key design issues for both the tutorial and the web-based component of the teaching session.

Participant Feedback

The conclusion of the teaching session was the completion of an electronic feedback using nine closed, multiple choice, questions and one free form comments field. Eight participants took part in the session and all completed the feedback form.

The session was designed as an introduction to home networking but assumed some level of understanding of computers. Six participants reported that they considered their own knowledge of networking to be "medium", one "low" and one "very low", making this a suitable group for the presentation.

Feedback on Session

The following feedback was provided by participants in relation to the structure and content of the session itself:

Comments

Some of the comments provided in the feedback included:

• ". the presence of a plan with the components placed in a schematic form . would be useful. I would like to be able to take this form to say Noel Lemmings and buy the network."
• "Very easy to navigate my way through the Website."
• "The language was simple and MOST acronyms were explained."
• "Very clear and logical sequencing of presentation."
• "Excellent structure and the English used was spot on, with the odd typo. The amount of detail was excellent but very easy to find."
• "A useful tutorial for the average home user."

Educational Philosophy

This section we will describe the philosophy that was used as a guideline in determining the general criteria that needed consideration when deciding upon the educational technology to use in order to achieve a successful learning outcome. Scaradamalia, Bereiter, Swallow & Woodruff (1998, p.51) stated:

It should be up to students not the computer to solve problems, make planning, and set the learning goals. The role of computers should be to promote and facilitate learners to maximize use of their intelligence and knowledge. In other words, the intellectual tools design should focus on providing quality scaffolding that entails metacognitve guidance to facilitate students learning how to learn (the 'effect of' technology), rather than off-loading and task dividing that try to ease students' cognitive burden (the "effect with" technology). The idea of distributed cognition is relatively new yet crucial.

Software design company, I.C.O.N. (2000), defined certain design principles, or steps, that need to be met to ensure that a written programme can be converted into a fully functional learning software program, these are:

• Determine the educational goals: What the writers want the students to learn.
• Determine the target public: Who the prospective students are.
• The language: For example, translation between English and French.
• The technical tool: CD-ROM, Web site, hybrid CD-ROM, etc.

Using this approach the writers set the following framework:

Criteria	Approach
Determine the educational goals	How to build a home networkGain an understanding of the different types of network available Understand the terminologyThe advantages of a home networkThe hardware and technical knowledge required to build a home network by oneself
Determine the target public	Peer group studying 06.817 Pedagogical Strategies for the use of IT
The language	English
The technical tool	An introduction using a slideshow presentation and an interactive self-paced tutorial delivered via the Internet.

Based on this framework the writers looked at determining a definition to design a web site in such a way as to effect a predetermined learning outcome, namely that the user with little or no technical knowledge of networks or home networking, will on completion of an interactive self-paced tutorial delivered via the internet, have gained adequate understanding of the requirements needed to set up a home network system and feel confident enough to set up a home network. The writers considered the Internet to be the most appropriate medium for a classroom activity based on the availability and ease of use of HTML and supporting tools and software packages, which has made it possible to develop learning environments that reflect, to varying degrees, constructivist principles easily transferable to the web environment. The principles considered most appropriate to this session were based on the research of Love & Gosper (1995), defined in their paper "Developing interactive course materials: Using HTML to integrate conventional and internet resources."

These principles are summarized as follows:

• Learners construct their own meaning and interpretations of instruction.
• Instructional goals will be negotiated, not imposed.
• Task and content analysis would focus less on identifying and prescribing a single best sequence for learning, but would instead identify several alternatives.
• Evaluation would be less criterion-referenced (for example, might include portfolios).

And are appropriate for:

• anchored or problem-based instruction where learners have to discover rules to solve a problem
• multiple representations of reality
• viewing events from multiple perspectives
• hypermedia environments

In trying to implement the teaching session, the writers found it difficult to meet the demands placed by these principles for designing a web site that would provide the user with a complete learner centred activity. This difficulty was a result of the limited time available to test the site and thereby allow the development of a more constructivist approach to the given learning environment. The writers were faced with the dilemma of having a lot of information but only half an hour to test the success of the web site as a learning tool. The problem that needed to be solved was introducing the material to users in such a way as to make use of the experiences, convictions and constructs that learners already possess, that the site be structured so that it can be easily understood and used by the learner and that the design facilitated exploration and extrapolation of information needed to meet the learning objective; in this instance the knowledge needed to build a home network.

The decision was made to use the dreadful acronym KISS - "Keep It Simple Stupid" - as the main design concept for the web site. As Tanguay (2000) stated:

Theoretically the task of a Web developer is simple. On the one side you have a particular user group who will be using your Web site. On the other side you have an unprecedented overload of information, services and products, which might interest them. Your job is to create a Web interface, which filters and packages this overload so that your target group can use it effectively.

Jonassen (1994) suggests a six-point conceptual framework to create Web-based instruction. The writers felt that this framework was manageable and provided principles that gave measurable outcomes based on the predetermined tasks for the learning exercise. These are that the Web-Based Instruction must:

• provide multiple representations of reality, representing the natural complexity of the real world: Different types of networking systems.
• present authentic tasks that conceptualise rather than abstract information: Determine a network type most relevant to the user.
• provide real-world, case-based contexts, rather than pre-determined instructional sequences: Determine the need , including network hardware, based on the user's own home environment.
• foster reflective practice: Can the user complete the task themselves or would a technician/electrician be needed to install the cabling, hardware, etc?.
• enable context and content dependent knowledge construction: Could the user sensibly use the information, knowledge and skills gained by the exercise?
• support collaborative construction of knowledge through social negotiation, as opposed to competition among learners for recognition: Use the knowledge gained to determine exact needs and to negotiate the purchase of a home network system, through a company of their choice, with the knowledge gained by completing the exercise.

Session Structure

Having reviewed the philosophy behind the session, we will now reflect on the structure of the session itself. The teaching session was divided into two parts; the in-class presentation and the self-paced online tutorial.

Lesson Format

It was decided to introduce the basic concepts to the students using a slide presentation, providing graphics and limited text and to then let the students do the on-line tutorial, this allowed the writers to provide a minimum level of pre-knowledge in a structured fashion, thereby attempting to ensure that the student focused on the requirements needed to set up a home network and was able to start to understand the interrelationship between the constituent parts, such as network cards, cable and hub, rather than focussing on understanding the terminology itself. This approach was based on the premise that, using an interactive on-line tutorial as a cognitive tool, and having given the students a basic pre-knowledge, the students could, in working through the tutorial, determine their own home networking requirements. This understanding of the broad concept was re-enforced by use of the web based tutorial, which allowed students to refer back to this at any time they felt that more information was required or should they come across unfamiliar terminology (which was considered likely given the complexity of networking and the level of knowledge of the target audience).

Web-based Tutorial

The web-based tutorial was designed to be simple and non-linear in approach in order to provide the user with the ability to navigate any part of the website site interactively:

Interactivity in instructional design takes on a more complex meaning. In good instruction, interactivity refers to active learning, in which the learner acts on the information to transform it into new, personal meaning. In a constructivist sense, the learner co-constructs meaning by exploring an environment, solving a problem, or applying information to a new situation that he/she helps to define (Campbell, 2000).

The writers intended that the participants would use the information in the website to determine their own needs for a home network. Therefore, the web environment was constructed to encourage active learning, based on learners making decisions about the task given, based on the content, navigation, presentation, and assessment.

The tutorial was designed with usability in mind. As Nielsen (2000) observed, to engage the user a web site must download quickly. The Internet is about understanding the user, including their technology base and media size is a particular issue if the product is targeting domestic consumers (Siegel, 1999). For this reason, the design was attractive but graphically simple (avoiding complex animations or plug-in requirements). According to Nielsen, site design must appropriately reflect the content, in this instance it was important that the site appeared professional and authoritative and that, as a teaching tool, the language must be relevant and appropriate. Both Nielsen and Siegel have argued that not only graphics but also page sizes should be kept as small as possible, to achieve this the tutorial was divided into logically grouped pages. Analysis of participant responses suggest that the design was both successful and appropriate to the target audience.

The website itself was structured in accordance with the principles of information architecture (Parker, 2000). As Parker explained, this is the process whereby the designer creates a blueprint for the proposed site with an emphasis on navigation, logical grouping of information and consistent design. For the tutorial, this involved using an inverted "L" approach to navigation, representing each section in a horizontal menu bar at the top of the page and sub-section navigation in a column on the left hand side of the page. Content was consistently placed to the right of the sub-section menu and a textual navigation bar representing the site hierarchy was placed below the content. Finally, to ensure that the user was always able to locate themselves within the site, the page title repeated the hierarchy, for example:

Home Network: Tutorial: Network Cards

Conclusion

This session was designed using sound pedagogical principles to target students with little prior knowledge of home networking. The self-paced web tutorial was developed with an understanding of good information architecture practices and with regard to the bandwidth issues for potential users.

To ensure a consistent introduction, a lecture-based approach was used to transfer knowledge of the basic concepts of home networking and to ensure that learners had a context for the session. Again, understanding that students had different levels of prior knowledge and that their requirements would be different, the second part of the session was self-paced and relied on an Internet-based tutorial and question and answer tool to both increase theoretical knowledge of the subject and, more importantly, to allow the student to determine a networking solution that would work for themselves.

The session ran with eight students, seven of whom had a level of prior knowledge appropriate to the session and one whose knowledge was already reasonably high. Feedback from the students was that they all found the session instructive and that their level of knowledge had increased. It was observed by participants that the avoidance of complex technical terms and simple presentation of the web-based material made the subject more understandable and, therefore, less daunting for the novice. Given the writers intent of focusing on the overall network model, not on the terminology or technology per se, this reflection is seen as positive.

In a real world environment, the tutorial would remain available after the session, allowing students to re-enforce and repeat learning as a need arose and this was a primary consideration in the design of the site. Even in the classroom setting, the session was seen by the participants as making them more likely to be able to select and even install home networking products.

References

• Campbell, K. (2000). The Web: Design for Active Learning. Retrieved November 16, 2000 from the WWW: http://www.atl.ualberta.ca/articles/idesign/activel.cfm#introduction

• I.C.O.N. (2000). Learning Software (learningware) publishing. Retrieved November 14, 2000 from the WWW: http://www.ikon.ch/Customer/ikon/Ikonstr.nsf/htmlmedia/about_us.htm

• Jonassen, D. (1994). Thinking technology. Educational Technology, April, pp. 34-35

• Love, P.K. & Gosper, M.V. (1995). Developing interactive course materials: Using HTML to integrate conventional and internet resources. Paper presented at ASCILITE95, University of Melbourne, Melbourne

• Nielsen, J. (2000). Designing web usability. Indianapolis, IN: New Rider

• Parker, R. (2000). Built 2 Order. USA: Publish. Retrieved November 8, 2000 from the WWW: www.publish.com/features/0007/feature4.htm

• Scardamalia, M., Bereiter, R. S., Swallow, M. J., & Woodruff (1989). Computer-supported intentional learning environment. Journal of Educational Computing Research, 5, 51-68.

• Siegel, D. (1999). Futurize your enterprise. New York, NY: Wiley

• Tanguay, E. (2000). My Advice for Beginners: Learn HTML, ASP and Database Connectivity. Retrieved November 16,2000 from the WWW: http://www.webdevelopersjournal.com

Appendices

Appendix A - Student Handout

An Introduction To Computer Networking In The Home A short session to introduce participants to the concepts and terminology of computer networking and to help participants gain an understanding of their own home networking needs. Learning Outcomes Participants will have learnt the basic concepts and terminology of computer networking, with an emphasis on networking in the home. Participants will have the opportunity to further examine areas they are unsure of or require more information on at their own pace via the website. Participants will better understand their own home networking needs. Participants will be able to reflect on their learning experience and re-enforce the learning by visiting the website at a later date. Structure This session is in two parts: Part A A presentation designed to introduce students to some of the basic concepts of home networking, to describe the different options that are available and inform participants as to the relative pro's and con's of each method. This session will include practical examples of networking equipment. Part B The second part of this session will be self-directed. Students can review an introductory tutorial on home networking through the course website and take part in a question and answer session that is designed to assist student in designing a home network that suits their needs. Reflection The website contains a feedback page, which is a simple questionnaire to help us to evaluate your learning experience, it would be appreciated if participants could complete this at the end of their session.

Appendix B - Lesson Plan

Outcome

By the end of this session, participants will have learnt the basic concepts and terminology of computer networking, with an emphasis on networking in the home.Participants will have the opportunity to further examine areas they are unsure of or require more information on at their own pace via the website.Participants can follow the web-based question and answer tutorial to help them design their own network.Participants will be able to reflect on their learning experience and re-enforce the learning by visiting the website at a later date.

Topics

Length	Plan	Resources	Outcome
5m	Introduce the session, the presenters and the learning outcomes for the session (AW)Ascertain current level of knowledge amongst group (AW)	Group	Group is familiar with the presenters and vice versa.Group understands purpose of session and has been briefed on structure and timing.Presenters are aware of group skills levels.
20m	Presentation of networking basics, including use of props to demonstrate network equipment (PB)	SlideshowPC/DatashowProps (NIC, hub, CAT5 cable)	Group is introduced to the high-level concepts and requirements for building a network. They should now be familiar with some of the concepts, equipment and terminology.
5m	Introduce interactive session so that participants are able to follow the online tutorial and question and answer utility (AW). Presenter to ask participants to complete feedback page on website at the end of their session.The purpose of the website is to re-enforce the presentation by allowing participants to review networking at own pace. Secondly, the tutorial provides a self-paced question and answer session (demo) to help them design their own network.	Website UR(www.wairua.com/development/hnb)	Students are aware of how to access the website and have been introduced to the website structure and purpose.
10-20m	Participants to review the website tutorial and question and answer model at their own pace (group).	PC'sInternetAccess to website	Students have used this self-directed time to re-enforce their own knowledge by using the online tutorial and to work through their own network requirements using the question and answer model

Flow Into Next Session

This is single session, but students are expected to re-visit the online tutorial as their own needs develop. The website is designed to be used to re-enforce learning and for the student to explore specific areas in more detail.An evaluation of the session is to be carried out using participant feedback from the website.

Appendix C - Web Site

Appendix D - Sample Pages

About the Authors

Andy Williamson is Principal of Wairua Consulting, an eCommerce and new media consultancy specialising in Information Architecture, and a Director of The iE3 Group Ltd, a management and IT consultancy. Andy's work helps organisations to understand the Internet, plan their web presence and achieve real results. He writes about the Internet and its impact on society, lectures on eCommerce and Information Architecture, is Chair of the IT Advisory Committee at UNITEC, a member of the executive of the National Advisory Committee for Computing Qualifications, a Fellow of the Institution of Analysts and Programmers (UK) and a member of the US-based ACM and the Internet Society.

Patrick Baker is an educator with 21 years of teaching experience. He has held numerous teaching and management positions in education and has served on various committees and advisory bodies dealing with education. Patrick has, until recently managed a Transition and Careers Department at Kelston Deaf Education Centre, Auckland. He is presently the Principal of a PTI Teachers Training College in Auckland. Patrick is happily married, has two children, 12 tropical fish, 5 tadpoles and a cockatiel to keep him occupied.