To take the quizzes that are a part of this talk, use Netscape 4.5 up to 4.75, or Internet Explorer 5. The quizzes will NOT work with IE 4.x, nor will they work with Netscape 6.x.
The quizzes are intended to illustrate seven testing formats available with our software, and NOT to measure your learning.
This talk was intended to show how software tools can help to make it easier to use the Web to structure teaching.
Before we begin on this, I thought some time should be spent discussing Web teaching in general. You'll find MOST of this in my book on Web teaching.
This Web talk works best when used with Netscape as a browser. If you are using Internet Explorer when taking a quiz, and you get an error message like this:
use the back button in the browser to go back one page, and try again!
How does one give a talk about software? I decided to create a lesson-like page that I might want to use for a given audience, say one learning about Web teaching.
I made this page using my tool. Here you see a product made using software tools..
For the second part of the talk, I illustrated the tool use in terms of this specific content. That is, I showed how this talk was made. I've since modified the page; there's quite a bit more here than actually was presented during the talk.
My hope was to make the example concrete, and to choose content that would have value beyond what I had by illustrating these features with -- for example -- a chemistry course for teachers.
The layout fir this page consists of --
The layout is entirely up to the teacher. You probably can adapt the tools illustrated in my talk to any consistent format. The purpose of the tools is not to make you conform; it is to help you to work fast.
If there is a surprise about the Web, it is rooted in early successes.
At UNL, Jim Partridge has been a pioneer with his plant pathology instruction. Bill Glider is another pioneer, and his work has enjoyed seminal inputs from Christy Horn and Roger Bruning. John Markwell is still another pioneer. These folks see learning gains -- some times BIG gains. Supplements where Web materials are devised to address the learning of difficult content within the context of a traditional (on campus, synchronous) course seem to be the clearest cases so far of where the Web has improved the amount of student learning (all other things being equal). These improvements have been dramatic, approaching what Bloom calls the 2-s (two sigma) effect.
There has been a tremendous amount of work in delivering distance courses at UNL. There has been notable success in this area; early pioneers include many members of the Department of Educational Administration. This effort was nurtured by the insightful efforts of Ward Sybouts. A small portion of the distance clientele so far has included students who otherwise would be without access. Providing access to this clientele has proven very successful as judged by their responses to course offerings.
Of these two early successes, the first seems to be economically viable. Once good materials are developed, they are handled repetitively at low cost. The economic viability of the second success, in my view, remains to be established.
There have been failures, too. Some courses have been duds. In an early masters program at TC using Notes, the faculty never did get to learn how to use Notes.
I had a Web site for teaching chemistry, and while students DID learn, the rate of learning was very slow.
Just throwing a course or parts of a course on the Web, and offering 24-7 access does not mean that effective learning will come about.
The are many issues that faculty need to think about when considering Web instruction.
The most important issues concern instruction. After all is said and done, instructional design matters. If the content is easy, instructional design will determine the level of student praise (or bitching). If the content is difficult, instructional design will determine how many students are successful with that content. Design matters.
I think there is a minimal Web presence that a teacher should have. This can be accomplished using Blackboard.
In a minimal approach, a teacher makes the following things accessiable on the Web:
To do more, add:
To do still more:
Be very careful about posting lecture notes. The literature reports successes, not failures. I've heard of many cases where posting notes led to lowered performances! (Also, keep in mind something that I heard a colleague from SOTL [UNL Scholarship of Teaching and Learning Community] say recently -- "data is not the plural of anecdote.")
The server is the computer where your course software resides. Servers are a big issue. You can use just about any (ethernet worthy) computer as a server. {Translation: if you can plug it into the UNL network and use it for e-mail, it probably could be a server.}
You need a fixed computer address (IP address), and you probably want to get some appropriate name (like www.chem.unl.edu or www.unl.edu/teaching). You need software, too. Probably server software and firewall software as a minimum. {Sever software is what lets the computer 'talk' on the Web. Firewall software tries to control such things as inappropriate access, or putting stuff on your server surreptitiously that is used to 'attack' other Internet-linked computers.}
You may want some other software, like Timbuktu, that will let you take control of your computer from someplace other than sitting right in front of it (like from your home.)
I run several "private" servers. By private server I mean that they are mine in the professorial sense of that usage. They are UNL servers that I run and control. I try to decide what's on them, and i am responsible for keeping them going. A couple of the private servers are working servers. If you are accessing this talk on the WWW, you are accessing what I call my main 'private' server. Other servers in my lab are developmental servers; we experiment and create on those servers. {The people in my lab wearing casts on broken limbs are the ones found messing with a non-developmental private server without permission.}
With private servers, many things are possible -- research, training, immediate access to special things.
However, there are headaches. The server must be maintained (24-7), and this is not easy.You need some special software, and you need to know how to use it. I fear the day that someone directly or indirectly loads something onto my server that is inappropriate. But, if they can do it to microsoft, they can do it to me.
I could not do what I do without "private" servers. They are integral parts of my teaching and research.
For most faculty, running a private server is nuts.
For most of what you might want to do, some mix of management using Blackboard or testing using eGrade will serve your needs.
If you have a special need, it's probably worth your while to 'lunch' an 'official' server manager at the Cornhusker (or even the French Cafe) and get what you need than to do it yourself.
It is possible that you might have a chance to put course materials on another server -- not yours, and not one at UNL, but one run by some other organization. If the support is really there, a 'third party' server may give you the best of both worlds.
Setting up a server with a local company, however, probably also goes under the heading, 'nuts.' That may go under the heading of really, really nuts.
Getting a good estimate of how much time you will need to spend is at the heart of being successful in your professional life as a Web teacher.
For ordinary courses, setting up materials on the Web is a straightforward matter. Learning how to use Blackboard is just not that tough, and there will be many places to get help while learning -- Teaching and Learning Center, ITG Short courses, courses organized by individual colleges, etc.
Putting up your syllabus is sort of a minimum. I actually use very interactive syllabi. I use special software to create these syllabi. They are not used in my Web courses.
At the other extreme from a minimal Web intervention is the course where you plan to create multimedia materials specifically for your discipline. For the most part, multimedia materials are overrated. In some situations they are absolutely essential, however.
Before you begin working on a multimedia intense project, you should contact a UNL colleague with some experience in that area. Katherine Ankerson from architecture has done some lovely work.
John Markwell has been engaged in developing some really nice animations.
If you want to use video, you should have a conversation with Chuck Ansorge. Video quality is an issue, and Ansorge has developed a system of pressing CD-ROMs to provide students with quality resources.
Simulations are becoming ever more Web worthy. However, they aren't worth much if they don't work, and getting some of them to work can take an enormous amount of time.
Most instructors find that the time they spend in Web teaching modes is greater -- oftentimes much greater -- than in conventional courses.
Why is the time greater? Many students will try hard to establish a line of communications with you, and that usually means e-mail.
Much of this e-mail is very personal, and it is extremely likely that you will learn about the student's family or pets. Indeed, much of what I've seen on screens can be described as 'love me, love my dog' conversation.
In the early days, faculty thought Web teaching would be very impersonal. Experience has shown that, at least in many respects, Internet-based teaching is more personalized than is traditional teaching.
This personalization extends to between-student one-on-one interactions when the e-mail addresses of the students are shared in the course. (You may not want to do this in a big course; there may be some serious privacy issues.)
Web discussions can be extremely rich. Since students have time to think about what they want to say, sometimes Web discussions are better than they are in class.
Also, different students seem to end up participating in Web courses than in traditional face-to-face courses.
The key to a successful Web discussion is to create good initiating questions.
Chris Marvin has proven to be an outstanding manager of discussions, and she has many tips for managing successful discussions.
I follow a model for learning that was developed by Gregg Schraw. Gregg is a wonderful colleague, and I miss having him here very much. Collaborating at a distance, while possible, is not the same as collaborating nearby. (Yes, this still is a Web talk, I'm still advocating 24-7 distance methods, and I did just say that some distance things don't work for me. I would have liked to link to Gregg, but he doesn't seem to have a page at UNLV. Some things never change.)
An important thing to keep in mind is that prior knowledge is the best predictor of new learning. This means any instructional methods offer contextual practice at the same level as that intended for testing usually give excellent results.
Mastery learning suggests that students must demonstrate mastery of the learning of a particular body of content before proceeding to the next body of content.
The Keller Plan, widely adopted as college approach to learning during the period 1965-80 based upon mastery learning, was very successful.
Kulik, C. C. & Kulik, J. A., (1987). Mastery testing and student learning: A meta-analysis. J. Educational Technology Systems, 15, 325-345. Kulik, J. A., Kulik, C. C., & Bangert-Drowns, R. L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60, 265-299. Kulik, J. A., Kulik, C. C., & Cohen, P. A., (1979). A meta-analysis of outcome studies of Keller's personalized system of instruction. American Psychologist, 34, 307-318. Kulik, J. A., Kulik, C., & Carmichael, K. (1974). The Keller Plan in science teaching. Science, 183, 379-383.
Kulik, C. C. & Kulik, J. A., (1987). Mastery testing and student learning: A meta-analysis. J. Educational Technology Systems, 15, 325-345.
Kulik, J. A., Kulik, C. C., & Bangert-Drowns, R. L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60, 265-299.
Kulik, J. A., Kulik, C. C., & Cohen, P. A., (1979). A meta-analysis of outcome studies of Keller's personalized system of instruction. American Psychologist, 34, 307-318.
Kulik, J. A., Kulik, C., & Carmichael, K. (1974). The Keller Plan in science teaching. Science, 183, 379-383.
Why was the Keller Plan abandoned? Both teachers and students disliked it. Teachers disliked it because they had to work too hard. Students disliked it because they had to work too hard. Experience shows us that improved learning outcomes are not necessarily sustainable; they are sustained by successful learning environments. Success is defined by what the system decides is successful, that system being an amalgam of students, teachers, and administrators.
Today there are several systems that will deliver and, to one degree or another, evaluate student homework. Studies of several of these systems suggest that they are very useful in improving learning.
Repeatable testing is at the heart of the Keller Plan. Many of us used repeatable testing. I used it while in chemistry ("Repeatable Testing,"; Moore, J. W.; Brooks, D. W.; Fuller, R. G.; Jensen, D. D. J. Chem. Educ. 1977, 54, 276.), and I still use the most important features of it in my graduate courses.
Cal Garbin is a UNL faculty member who has excelled in getting students to perform better through repeatable testing. He uses the term practice instead of the term testing. Garbin's work is noteworthy because the content covered is generally perceived as being difficult.
Far and away, the most important contributor to repeatable testing at UNL is John Orr. Orr's ingenuity, tireless dedication to his effort, and responsiveness to faculty feedback have made his program vital to our efforts. Orr's work is reflected in his software product, eGrade. The UNL Online Testing Center is the result of Orr's (and others) efforts. Learn more about eGrade and the Testing Center from a FAQ page.
The UNL Online Exam Room:
If you have a server, you can do some research. Research may be important. Much of what we study as faculty in terms of the Web is at a low level. We survey our students for the -- love it/hate it -- stuff. We look at Web statistics.
A few people have looked at learning outcomes. Bill Glider (with Horn and Bruning) deserves much credit for this approach. So does Jim Partridge.
I try to measure learning at the server level by capturing as many of the keystrokes as possible. I can't time them, but I can time the lag between when a quiz item goes out and when a response comes back.
There is some hope that this research can have deep payoffs. Two approaches -- knowledge space theory, and rule-space models -- suggest that we can try to elucidate information from test performances that imply strategies for sequencing instruction. This is very important in the sciences, for example.
It seems obvious that, if you know something about sequence, you'll know something about creating your curriculum.
We also can test other learning theories like cognitive load theory and cognitive flexibility theory. These deal with how you design instruction.
Cognitive load theory suggests that, if the material of this talk is difficult, then the various hyperlinks will get in your way. On the other hand, if this material is easy, cognitive flexibility theory suggests that having the links in this document will improve your hypermedia skills.
Using worked-out examples is a much touted strategy ("Learning from Examples: Instructional Principles from the Worked Examples Research," Atkinson et al, Review of Educational Research, 2000, 70, 181-214). Both eGrade and my systems afford learners significant opportunities to use worked out examples.
Quiz 1 below is a chemistry example from UNL Web course for chemistry teachers. The point here is that MANY questions are possible, and that the worked-out example fits exactly the particulars of the problem given.
The key software is HyperCard. Next most useful is Dreamweaver, and then my browser. I prefer Netscape 4.75.
The logic of a HyperCard Hub is that one HyperCard program runs numerous files (called Stacks). Consider when I ask a quiz question. The HTML code buried in the Web pages calls for a HyperCard file (stack). For this talk, it calls for FirstTuesday. It also carries information about which question to ask.
Nerd special: When a user clicks a 'Quiz' button the message comes to my server. WebStar generates an AppleEvent. HyperCard detects the AppleEvent, and uses that to gather information -- in this case the Program name (FirstTuesday), the task (make a quiz), and the item(which question to make. FirstTuesday then makes that question. encoded in the question is information that allows the very same question to be reconstructed. Nothing is stored. Grading goes the same way, except that the encoded information helps FirstTuesday to reconstruct the question and calculate and answer. All of these pages are calculated, not stored.
There are four pages involved in this presentation.
One page, prepared in advance, sets the frames.
A second page determines what is at the top. In this case, it says something about First Tuesday. In my course, it includes some navigation.
The other two pages -- the list at the left, and the content at the right, are calculated using HyprCard.
The page at the right is assembled by taking 'chunks' of preformatted text from HyperCard. If there are related quiz questions, they are included at the time this content page is 'calculated.'
The selector page at the left is just a list of the topics used in the program. Six levels of headings are possible. Each of these is a link. HyperCard calculates the links. it starts with the list on the top page of the stack (top of the file), and uses that. Embedded in the content side (right side) are anchors. these are calculated at the same time that the conent page is calculated.
Because the content page is assembled from 'chunks,' you need to either be able to code these pages or use tools to help you.
HyperCard can send these pieces to a powerful tool and then import them back. It can be ste to nearly any tool you might want to use, Dreamweaver and GoLive being the current gold standards.
I think it's foolish to say that one doesn't need to know HTML to make good pages. But, one sure as hell doesn't need to know much, and powerful tool access is very simple.
The HyperCard stack is able to create 10 types of quiz formats, and seven of these are illustrated in this talk. The other three rarely are used in our courses.
The quiz-types are:
In our courses, student answers to essays are stored and retrieved by the instructor for evaluation.
Programming the short answer questions is done on an individual basis, and takes a great deal of time. Teachers have used the blank format as a workaround.
You could use the HyperCard stack to write quiz questions which were then downloaded into eGrade.
I prefer the immediate-response-type item, and build that into my pages. You may prefer a different approach where items are bundled together and then tested as a batch.
My software can do bunched questions, too. However, once the concept was proven, we switched to this format. If there is interest, I can include that format.
There was a question at the talk that was NOT well answered by me. Why do this, instead of eGrade? After all, it just adds another step. My colleague, Dr. Crippen, reminded me that we do all of our test creation off-line, and that's a tremendous time advantage. As far as I know, eGrade must be done online.
Remember, these are not competing systems.
Resources at UNL in some areas are very thin. In my view, the best resources are the faculty.
Blackboard: Paul Erickson
Testing Help: John Orr
Discussion Help: Chris Marvin
Multimedia Advice: Kathy Ankerson, John Markwell, Jim Partridge, Charle Ansorge (video)
Course Supplements: Bill Glider, John Markwell
Running a Web Course:
Low tech: Chris Marvin High Tech: Jim Partridge, Bill Glider
Low tech: Chris Marvin
High Tech: Jim Partridge, Bill Glider
Big Help Needed (nerds, servers, etc.):
ITG (Bruce Sandhorst) Continuing Studies (Nancy Aden, Marie Barber) DEAL Labs (Ron Roeber) Colleges
ITG (Bruce Sandhorst)
Continuing Studies (Nancy Aden, Marie Barber)
DEAL Labs (Ron Roeber)
Colleges
Money:
Teaching Council (Del Wrioght) TLTR (Charles Ansorge, Pam Holley-Wilcox) Associate VC for Extended Education (Jim O'Hanlon) VCAA NSF, Dept. of Educ., Dept. of Agric., etc.
Teaching Council (Del Wrioght)
TLTR (Charles Ansorge, Pam Holley-Wilcox)
Associate VC for Extended Education (Jim O'Hanlon)
VCAA
NSF, Dept. of Educ., Dept. of Agric., etc.
Faculty Development:
Teaching and Learning Center First Tuesday 2nd Wednesday
Teaching and Learning Center
First Tuesday 2nd Wednesday
First Tuesday
2nd Wednesday