Instructor's Guide to Chi Tester
 
 

Contacts:  Developers	Contact:  For Pricing and Licensing Information
Craig Gundy  cgundy@weber.edu	Richard Alston, President
Eric Jacobson  ejacobson@weber.edu	Chi Squared Software, Inc.
Jeff Willden  jwillden@weber.edu	4181 Bona Villa Drive
Weber State University	Ogden, Utah 84403-3246
2405 University Circle	Phone:  801.626.6061
Ogden, UT 84408	ralston@weber.edu

TABLE OF CONTENTS

1. Introduction and Overview

1.1 Life Cycle of an Exam
1.2 Browser Variability

2.1 Accessing an Exam
2.2 Taking an Exam

3.1 A Sample Exam
3.2 Basic Question and Answer Formats
3.3 Question Types
3.4 Adding Feedback to Questions
3.5 Question Categories
3.6 Question Weights
3.7 Inserting Multimedia (Graphics, Video, Sound)

4.1 Access to Chi Tester
4.2 Instructor Registration
4.3 Logging In

5.1 Submitting New Exam Text
5.2 Test Setup
5.3 Final Test Preparation in Exam Management

6.1 Proofreading and Editing
6.2 Edit Exam Parameters
6.3 Changing Exam Parameters
6.4 Exam Passwords

7.1 Data Ordering in Reports
7.2 Instructor Report
7.3 Student Reports
7.4 Item Analysis
7.5 Rescore Results

9. Surveys and Evaluations

10. Archiving and Deleting an Exam

10.1 Save Exam Text
10.2 Resubmitting Saved Tests
10.3 Delete Results
10.4 Delete Tests and Results

1. INTRODUCTION AND OVERVIEW

Chi Tester is an Internet Test Driver (ITD) that presents and scores exams, surveys and question-based tutorials on computers. All interactions with Chi Tester, whether it's students taking exams, instructors submitting exams, instructors accessing exam results, or managers enrolling new instructors, occur through the World Wide Web (WWW) on standard browsers such as Netscape Navigator or Microsoft Internet Explorer. Chi Tester and WebTester, its predecessor, have been used at Weber State University since 1998 in courses in Health Sciences, Art, Economics, Business Administration, Physics, Music, Information Technology, Psychology and other fields for both distance learning students and those enrolled in on-campus classes. Chi Tester is also being used at university campuses throughout the U.S.and in New Zealand, by private commercial test delivery companies, and by human resources divisions in major corporations for employee training and assessment. Chi Tester is owned and distributed by Chi Squared Software, Inc. Complete, current information about Chi Tester can be found at http://chitester.weber.edu/. This is the active site at Weber State University where hundreds of instructors test thousands of students every semester. Temporary access for potential licensees is available on a selective basis. Contact Richard Alston for more information concerning licensing and pricing (see Contacts above).

In this document all people who answer questions in Chi Tester--respondents in surveys, learners in tutorials, students being evaluated--are referred to generically as "students." Those presenting the questions--survey researchers, teachers, evaluators--are referred to generically as "instructors."

Chi Tester presents multiple choice, multiple response, short answer and essay questions along with specified graphics, sound or video. Depending on the nature of the question, the student selects responses by clicking with the mouse, or by typing an entry into an answer box. In the testing mode, Chi Tester evaluates responses against the correct answers stored with the test. In presenting surveys, answers are recorded without evaluation. Chi Tester reports results on demand.

A variety of security options enhance fairness and reduce the chances of cheating. Question selection from a larger test bank and the order of question presentation can be randomized so no two students see exactly the same exam. Tests can be restricted to only designated computers and to only students with specified identification (ID). A password may be required for a particular test. By providing that password only to a specific test proctor, the instructor can ensure a specific level of monitoring.

In test mode, Chi Tester presents evaluative, graded exams, with or without feedback to the student during post-test review. In survey mode (the mode used for such things as voting, obtaining student input in opinion polls, and for course evaluations), no correct answers are specified in advance by the instructor and the "student" may be allowed to answer anonymously. In tutorial mode, feedback information stored with each question is given after a response. Sections 2 through 7 of this guide describe Chi Tester in base-test mode. Sections 8 and 9 describe special procedures for surveys and tutorials.

1.1 Life Cycle of an Exam.

A Chi Tester exam has the following life cycle:

All interactions with Chi Tester occur through internet browsers. All users, those submitting and managing tests and surveys, and those taking them, do so through a browser on the WWW. The predominant browser products are Netscape Navigator and Internet Explorer. Different versions of each exist at any one time and, in addition, individuals modify the browser environment in personal ways. It is not possible, therefore, to describe in all cases exactly how Chi Tester screens will appear and operate across this diversity of systems. The descriptions here are accurate for most systems. However, some differences in detail can be expected.

2. TAKING AN EXAM

2.1 Accessing an Exam.

Any computer connected to the Internet and running Netscape Navigator or Internet Explorer is a potential exam site. However, an instructor can restrict access to:

To access Chi Tester, a user initiates Netscape Navigator, or a similar Net browser, and enters the name of the website (Universal Resource Locator better known as the URL) which has been established for the particular Chi Tester installation being used. For example, students at Weber State University (WSU) type in http://chitester.weber.edu, the URL for WSU users. The URL for other licensed users is specified by the Chief Administrator, as explained in the Administrator's Guide. The home page offers the option to "Take a Test." When this option is selected, the test taker is asked to:

If the student is a first time user of Chi Tester, he or she will be asked to supply a name and an email address (optional). This information is stored in a database and on subsequent tests it is called up when the student ID is entered.

If the test is to be strictly proctored, the proctor would enter the password and it would be unknown to the student taking the exam.

2.2 Taking an Exam.

Questions are displayed one at a time on the computer monitor. Multiple choice questions are shown with the answer choices listed below the question. The student uses the mouse to click on the desired answer, or answers if there are more than one possible acceptable (i.e., multiple response type questions). If the question type is short answer, the student is given a blank field below the question into which a line of text can be typed. For essay questions, a text entry box is given which allows for the typing of many lines. Text is scrolled within this box to accommodate long essay answers.

Pictures, graphs, animations and sound can accompany questions just as with any Web document (see Section 6.1.4).

Questions may be grouped into categories and tests may be generated by randomly selecting questions from each category. If the instructor chooses, the order in which students view questions may be randomized. The latter option provides enhanced security when more than one student are taking the same examination at the same time and in the same testing center.

Students can move back and forth through exam questions. This allows them to change and revise answers before stopping and submitting the test. The student selects a STOP option when finished. If the instructor has allowed the option, the student is given his or her score on exiting the exam.
 

3.  QUESTION PREPARATION

Chi Tester questions are formatted very much as they would be for presentation on paper. Each question is followed by descriptors indicating answer, feedback, question weight, and question category. Only the answer descriptor is required.

The general format is shown in the example that immediately follows.

Example Question formats:

1. Question text goes here followed by a hard return.

a. Distractor (i.e., possible answer choice) goes here if it is a multiple choice question.

b. Have as many distractors as you wish, from a to z.

ans: b (this is mandatory and, depending on what is entered, tells Chi Tester the question type)

Category: Chapter 2 (optional; used for selecting random questions from a larger test bank)

Feedback: (optional; the various types of feedback are shown immediately below)

General: This feedback will be displayed if the test taker clicks the feedback button regardless of whether the question was answered correctly or not.

Correct: This is shown if the student answers the question by selecting the correct answer(s).

Incorrect: This is shown if the student answers the question by selecting an incorrect answer(s).

a: This feedback is shown if the student selects distractor a as his or her answer.

b: This feedback is shown if the student selects distractor b as his or her answer.

Weight: 2 (optional; used to vary the value of questions in the examination for scoring purposes)

2. In which Utah city is the State capitol building?

ans: Salt Lake City; SLC; Salt Lake

Notes on the two sample questions and general formatting for test preparation:

Question 1 is a multiple choice type question. Question 2 is a short answer question. Two other types, multiple response and essay, are also allowed and are discussed in Section 3.3.

If there are more than one distractor in multiple choice or more than one short answers which are correct, they are separated by a semicolon. The instructor may place a space after the semicolon for readability during editing, but it is not required.

The question parameters "Category," "Feedback," and "Weight" may be placed in any order, but if one or more are present, they must appear after the correct answer designation "ans:" and before the next question. If weight is included, it must be a number which multiplies the value of the question. In this case, question 1 is worth twice as much as question 2 because it has a weight of 2. If omitted, the default weight of the question is 1.

The feedback parameters "General," "Correct," and "Incorrect" may be placed in any order, but if one or more are present, they must appear after the line "Feedback:" and before any distractor-specific feedback, and before any other question parameters such as "Category" or "Weight". The distractor-specific feedback, indicated by the label a: may also be designated as a., since that is the way they are used above as distractors.

Most instructors will probably find it easiest to prepare test questions in a word processor and upload them, Section 5.1.1. Test files uploaded into Chi Tester must have been saved in ASCII or text format. Alternatively test questions can be typed directly into Chi Tester, Section 5.1.3, and major additions and changes can be made with the Chi Tester editing options, Sections 6.1 - 6.2.

The next section shows a short sample exam illustrating Chi Tester format and options. Full explanations are given in subsequent sections.

3.1 A Sample Exam.

The following exam is typed in Chi Tester format. A GIF image (Section 3.7.1) of a parabola (that is, a graph or drawing of a parabola) will be added to question 3 when it is edited in Chi Tester (Section 6.1.4).

1. Mary has two more apples than Jane. Heather has twice as many apples as Jane. Mary has 8 apples. How many apples does Heather have?
ans: 12; dozen
category: word problem
feedback:
correct: very good

(Comment: This is a multiple response type question and will be counted as correct only if student selects all three of the required answers because  the string ans: a, c, d separates the distractors with commas. If they had been separated by semi-colons, then the answer would have been counted as correct if any one of the three correct responses had been selected. Feedback is provided for the distractor selected by the student. The value of the question is 3 times greater than was question 1.)

3. This is the graph of which function?
a. x^2 - y^2 - 20
b. 3x - 7
c. x^3 - 12
d. y^2 + x^3 - 31
e. 3x^2 + 7x - 12
ans: e
weight: 2
category: analytic geometry
 

(Comment: This is a multiple choice type question. Only one distractor is the correct answer. There is no feedback to the student. Note the use of carat (^) to indicate exponents since HTML does not support subscripts and superscripts.)

4. Which would you prefer for this course?
a. more tests and less homework than usual
b. fewer tests and more homework than usual
c. more tests and more homework than usual
d. fewer tests and less homework than usual
e. the usual amount of tests and homework
ans: multiple choice
category: evaluation
 

(Comment: This is a multiple choice type survey question and is unscorable. The student will only be able to pick one distractor for their answer. The question will not be scored as correct or incorrect but the answer will be recorded and included in instructor's report. This is typically used in survey, assessment, or evaluation settings (see Section 3.35).

5. Which topics do you need some additional help on?
a. solving equations
b. graphing functions
c. story problems - deriving equations from real situations
d. quadratic equations
e. logarithms
ans: multiple response
category: evaluation
 

(Comment: This is a multiple response type question. Student may select more than one distractor as his or her answer. See the comment in question 4, above; see also Section 3.3.5.)

6. Describe one situation in which this course has helped you solve a real world problem.
ans: essay
category: word problem
 

(Comment: This is an essay type question. Note that, with the exception of the answer descriptor -- here, ans:essay -- the other components (i.e., weight, feedback, category) are optional.)

3.2 Basic Question and Answer Format.

Questions must be numbered sequentially beginning with one (1.) and each number must be followed by a period. When Chi Tester finds the next hard return followed by a number followed by a period in an exam, it begins the next question.

At the end of a question, Chi Tester identifies the correct answer from the answer descriptor string ans:

Text which follows this string up to the end of the line is stored as the correct answer and is used for scoring student responses. The exception is ans:essay, which is treated as non-scorable. The student's typed-in answer is stored for later report(s) that may be accessed by the instructor.

Optional descriptors, Feedback, Category and Weight, are described in Sections 3.4, 3.5 and 3.6, below. Note, these descriptors, if used, must follow the string ans: descriptor.

3.3 Question Types.

3.3.1 Multiple Choice Questions. (See Section 3.1 Sample Exam, question number 3)

For multiple choice questions the student is offered several possible answers, called distractors. In Chi Tester a radio button is placed next to each distractor and the student selects one by pointing the mouse at it and clicking. Only one response is recorded. If a student clicks on a second distractor, the selection of the first distractor is erased.

Distractors are designated with a single alphabetic character followed by a period. Either upper or lower case is acceptable. On a paper exam these letter labels remain with the distractors and the student answers by circling the answer, by writing the letter, or by marking the letter on a scoring sheet (e.g., Scantron). In Chi Tester the letters are replaced with radio buttons.

The correct distractor is designated with the letter label: e.g., ans: b . More than one distractor can be designated as alternative correct answers by separating the distractor letters with a semi-colon: e.g., ans: c; e . In this example, either c or e would be scored correct, but only one of the two would be allowed.

3.3.2 Multiple Response Questions. (See Section 3.1 Sample Exam, question number 2)

Multiple response questions are the same as multiple choice except that more than one distractor is correct. Chi Tester allows any number of distractors to be selected and will score the answer correct only if those selected exactly match those listed as correct by the instructor. If distractors a and d were correct then the answer descriptor would appear like this:

ans: a, d

If multiple distractors are separated by commas then the question is multiple response and all correct distractors must be selected for a correct score. If multiple distractors are separated by semi-colons then the question is multiple choice and only one of the listed distractors can be selected for a correct score (see Section 3.1.1).

3.3.3 Short Answer Questions. (See Section 3.1 Sample Exam, question number 1)

A short answer question is identified by listing a word or phrase as the correct answer. Here are two examples:
 

23. Who is buried in Grant's Tomb?

ans: Ulysses Grant

23. Who is buried in Grant's Tomb?

ans: Ulysses Grant; U. S. Grant; Mr. Grant

In the first example, only "Ulysses Grant"  or "ulysses grant" would be scored correct. In the second, "Ulysses Grant" "U. S. Grant" or "Mr. Grant" would be accepted as correct. Note that that Chi Tester is not case sensitive (i.e., the difference between upper and lower case letters is ignored), but the periods in "U.S." and "Mr." are required. "Mrs. Grant", "Mr Grant", or "US Grant" would be wrong in either example. If the instructor wants to count misspelled entries as correct, he or she may enter as many variants as can be anticipated, separated by semi-colons. Alternatively, the instructor may want to code the question as an essay-type and score the responses manually after accessing the results report with all answers listed (Section 7.2.1).  Yet another alternative, after the exam is completed by all students, is to access the instructor report with all answers listed (See Section 7.2.1). After visually inspecting the various spellings submitted, the instructor would use the "Edit Selected Exam" option in "Exam Management" (see Section 6.1) to add any desired "spelling variants" to the list of answers. Rescoring the results will then give the students credit for their creativity (see Section 7.5). Experience shows, however, that few instructors can "outguess" the creative and imaginative spelling of modern students. Many have found it simpler to rewrite questions as multiple choice to avoid the issue of spelling entirely.

3.3.4 Essay Questions. (See Section 3.1 Sample Exam, question number 6)

Essay questions are coded in Chi Tester by entering the word "essay" as the string answer descriptor, like this:

ans: essay

With this designation, Chi Tester records the answer typed but does not include it as a scorable part of the exam. The instructor can read and score essays by using the "All Test Answers" option when requesting a report (Section 7.2).

3.3.5 Questions with No Correct Answers. (See Section 3.1 Sample Exam, questions number 4 and 5)

An instructor may wish to ask questions for which there are no correct answers. Survey questions are of this type. If the instructor simply wants the answers recorded and not scored then the name of the question type should be entered for the string ans: . For example, if the string appears like this ans: short answer then Chi Tester will present a blank for the student to insert an answer just as with any short answer question. The answers which are typed by students are saved for later review by the instructor and are not scored. Other string ans: codes which will prevent scoring are multiple choice, multiple response, and essay.
 

Another example:

23. Why did you enroll in this course?
a. required for my major
b. my advisor recommended it
c. friends told me it was good
d. sounded interesting from the catalog
ans: multiple response

3.4 Adding Feedback to Questions. (See 3.1 Sample Exam, questions number 1 and 2)

Four mutually compatible types of feedback can be added to a question:

a) general - feedback given regardless of answer;
b) correct - feedback, given if a correct answer is submitted;
c) incorrect - feedback, given if a wrong answer is submitted; and,
d) answer - option specific feedback, given only if the specified answer option is selected by the student.

Feedback:

The question example below shows all the feedback options and how they are coded:

23. What state is north of Pennsylvania?
A. Indiana
B. New York
C. Missouri
ans: B
Feedback:
General: Geography of states is important.
Correct: Very Good.
Incorrect: Wrong. Study the chapter again.
A: Indiana is west of Pennsylvania.
B: Yes, New York is north of Pennsylvania.
C. Missouri is west and south of Pennsylvania.

Geography of states is important.
Wrong. Study the chapter again.
Indiana is west of Pennsylvania.

3.5 Question Categories.

One way in which Chi Tester can generate randomized tests is by selecting a certain number of questions from each question category (randomized selection is explained in Section 6.2.2). The category of a question is defined by the code, category: followed by a string designating the category name, e.g., category: Byzantine. Any string can be used to name a category. The short answer type questions below are placed in the category "rural."
 

23. What is hay made of?
Ans: alfalfa
category: rural

24. What is a Holstein good for?
Ans: milk
category: rural

If in "Test Preparation in Exam Management" (see Section 6) the instructor specified that six questions should be selected from the category "rural" then these questions would be among those which might be chosen (See Section 6.2.2).

3.6 Question Weights.

An instructor can weight questions by using the weight: code. For example, weight: 3 increases the score value of the question to three. If no weight is designated the question has the default weight of one.

In the short answer questions below, if a student got the first question right and the second wrong the score would be one because, since no weight is specified, question 1 automatically defaults to a weight of one. It the student got the second question right and the first question wrong, the score would be two. If the student got both questions correct, the score would be three.
 

23. What element has the symbol Na?
Ans: sodium

24. What is the product of solar fusion?
Ans: helium
weight: 2

3.7 Inserting Multimedia (Graphics, Video, and Sound).

Chi Tester presents multimedia stimuli using standard Hypertext Markup Language (HTML) links to multimedia files. These files must be prepared or modified for the question context. They must be in a form that the student's browser can read and they must be copied to the Chi Tester server along with the text of the test. Copying multimedia files to the server is explained in Section 6.1.4.

3.7.1 Multimedia File Types.

When a multimedia file is inserted into a Chi Tester test (Section 6.1.4), a support code for standard multimedia file types, designated by standard file extensions, is automatically inserted. Multimedia types for which default html code is automatically inserted are listed below, by their extension.  Because Chi Tester is implemented with web pages, you can include any multimedia type that can be displayed on a web page, given any required plug-ins are properly installed.  You may have to insert into the question the appropriate html and plug-in code to make the multimedia element appear correctly.

Some browsers or computers may not have the necessary plugins to support a particular multimedia file type. Before any multimedia files are used, the instructor should confirm that the computer(s) on which the test is to be taken can support them.

3.7.2 Preparing Multimedia Files.

Images and sounds are presented along with Chi Tester questions and so they must be compatible with those questions. Using an appropriate editor, graphics should be prepared to be the same size as they will be on the actual exam, and small enough so that there is space remaining for the text portion of the questions. Multimedia videos will appear with a control bar for pause, replay and volume. This bar will require approximately 24 additional pixels in the display. Thus, videos should be sized to allow room for the question, video and control bar.

If a graphic has many subtle shades of color (e.g., photographs), it should be saved as a JPEG file with extension .JPEG or .JPG added to the end of the filename. For pictures with very few colors, the GIF format, with .GIF extension, often results in a smaller file. Some computers used for testing may have low performance video systems, so images which depend on high resolution or fine color discrimination may not appear clearly on the computer monitor.

Questions with large multimedia files (over 40K) may load slowly, especially over modem connections.

3.8 HTML Codes and Special Characters.

Some characters are used for special purposes in HTML, the presentation language of Chi Tester. If placed in the test, they will be interpreted as HTML commands instead of text to be viewed. The question will not appear correctly and error conditions are likely. For these symbols, it is necessary to place the HTML alternatives into the question. HTML codes for some commonly used characters and formats are shown below.

4. CHI TESTER ACCOUNTS AND LOGGING IN

4.1 Access to Chi Tester.

Each Chi Tester site will establish a testing home page for access to its Chi Tester services. The development address for Chi Tester, http://chitester.weber.edu/ will be used for examples in this manual. Instructors should use the appropriate address for their site.

The home page for Chi Tester at your site has links to all Chi Tester functions, for students, instructors and managers. Only registered users with previously established login names and passwords can access instructor, manager, and administrator functions. The level of access for each registered user will be set by the chief administrator.

4.2 Instructor Registration.

To use Chi Tester, instructors must register and obtain a login name and password. Each Chi Tester site will establish policies and procedures for issuing these accounts.

Contact the Chief Administrator at your site for further information.

4.3 Logging In.

Instructor functions are accessed from the Chi Tester home page by selecting "Exam Management." This selection brings up a login screen at which time the instructor enters the login name and password. After successful login, the first screen  the instructor sees is a list of his or her exams. The instructor can select one of those to edit or otherwise manage, or can opt to create a new exam. Of course, the first time an instructor accesses "Exam Management," the test list menu will be empty.

5. SUBMITTING THE EXAM TO CHI TESTER

Simple test submission consists of logging in, submitting the test to Chi Tester (Section 5.1) and setting up the test administration parameters (Section 5.2). Although this is sufficient for giving a test, two other optional management procedures are recommended. These are: 1) test proofing and editing with the insertion of graphics, sound if desired; and 2) customizing the test environment by editing additional exam parameters. These are described in Section 6, Test Preparation in Exam Management.

5.1 Submitting New Exam Text.

After selecting "Exam Management" and logging in with username and password, the instructor selects the option, "Create New Exam." The name of the new test must be entered. This name will appear on test lists for the instructor and student and so the test name should be understandable to both. It cannot exceed 30 characters. This name can later be changed on the "Exam Parameters" page, Section 6.2.

The new test can be submitted using

a) File Upload (Section 5.1.1) with a file which is saved in ASCII or plain text format; or,

b) Text Box and

i) pasting an exam previously copied into the clipboard (Section 5.1.2); or

ii) typing the test directly into Chi Tester (Section 5.1.3).

5.1.1 File Upload.

The file to be uploaded must have been saved in plain text, or ASCII format. The File Upload option must be taken and the name of the file must be entered in the "Select a file:" box. Some browsers will allow the file name, with full path, to be typed directly into this box. Some will not. All browsers should allow the instructor to use the "Browse" option to search for and to select the test file. The standard "browse" routine on some browsers may be preset to select files with ".html" extensions. To see and select other files, the instructor must change to ".txt" files or "all" files. After the correct test file has been typed, or selected from the "browse" routine the "Submit Test" button is clicked. Within a very short period (usually only seconds), Chi Tester will return a "Submission Results" screen (Section 5.2).

5.1.2 Pasting from Clipboard into Text Box.

Before entering Chi Tester the instructor should open the test in a word processor, select all of the test text and use the Copy function to copy it into the system Clipboard. The instructor should then log in to Chi Tester and begin the "Create New Exam" procedure described in Section 5.1. When the "Text Box" option is selected the instructor receives a window with a text entry box. The cursor should be placed in this box. From the browser Edit menu the instructor should select Paste and the previously copied test will be pasted into this box. The "Submit Test" button should then be clicked. Within a very short period (usually only seconds) Chi Tester will return a "Submission Results" screen (Section 5.2).  Note that depending on which browser you use, there is a size limit on the amount of text that can be submitted in a text box.  This limit is probably at least 24K.  In past experience, this may give room for between 50 and 75 questions.  If you have a test over 100 questions, it is recommended that you use the file upload option to import your test.

5.1.3 Typing Directly into Test Box.

After selecting "Text Box" to submit the test, a text entry box appears. The cursor should be placed in this box and the test can be typed directly into this box, using the format described in Section 3. When finished the "Submit Test" button should be clicked. Within a very short period (usually only seconds) Chi Tester will return a "Submission Results" screen (Section 5.2).

5.2 Test Setup.

After the Submit Test button has been clicked, Chi Tester shows "Submission Results," listing the questions of each type which were identified in the submitted test. If major formatting errors are apparent from this list, the instructor can select the "Delete and Resubmit" button. Before resubmitting, the test can be reformatted in the original file.

If after submission the instructor chooses to "Continue Setup," the next screen is "Test Setup." In "Test Setup," the instructor chooses the basic conditions under which the test will be presented to students. Three conditions must be set:

On the Test Setup page the instructor selects one of these modes:

When a mode is selected, values are set for test parameters which are generally appropriate for the type test/survey interaction named in the menu. For example, one default parameters set in the Survey mode is "no ID required." Instructors can change these preset, default parameter values by using "Edit Exam Parameters," Section 6.2.

5.2.2 Dates and Times.

On the "Test Availability - Dates & Times" page, the instructor enters the starting date and time and the ending date and time for the test. Tests will not appear and cannot be taken by students except between these two points in time. Default dates and times are the entire day the test is entered in Chi Tester. The dates and times can be changed by the instructor at any time by using "Edit Exam Parameters," Section 6.2. This means that an examination may be reopened after the original date or time expired. Instructors often find, for example, that one or more students "missed" the exam for a legitimate reason and this feature minimizes the hassles involved in accomodating them.

5.2.3 Test Sites.

The final choice for the instructor in "Test Setup" is the site or sites at which students may take the test. Each Chi Tester installation can establish testing centers by entering IP addresses for a given set of computers. A test which is assigned to a particular testing center can only be accessed on the computers registered at that center. On the "Test Availability - Sites" page, the established testing centers are listed and the instructor selects the center or centers at which the test is to be taken. An automatic entry on all site lists is "Everywhere Else." This is the set of all computers on the web which are not in testing centers. A test which is assigned to "Everywhere Else" can be taken by anybody on the web and is appropriate for surveys, take-home tests, and tests or tutorials which need no proctoring.

Instructors must make a selection of at least one site, or the "Everywhere Else," option by selecting (clicking on) the desired entry or entries on the site list. If no selection is made, the exam is not available anywhere and cannot be taken.

5.3 Final Test Preparation in Exam Management.

After "Test Site" selection, basic "Test Setup" is complete and the test (or survey or tutorial) will appear at the prescribed times and sites. In most cases, however, additional work will be necessary in "Exam Management" (Section 6). Final preparation procedures in "Exam Management" include proofreading the test to ensure that it appears correctly on the web; adding multimedia files to test questions; and customizing exam parameters for the precise test/survey/tutorial environment desired. This final work can be done immediately at this point after new test creation and setup, or can be done later by logging into "Exam Management."
 

6. TEST PREPARATION IN EXAM MANAGEMENT

An instructor enters the Exam Management page at the conclusion of the exam creation sequence, or at a later time by logging in from the Chi Tester home page, Section 4.3. At the end of this login sequence, the instructor selects the exam to be managed from the list of exams the instructor has submitted. After making this selection the instructor can proceed to the following editing and preparation functions for that test.

6.1 Proofreading and Editing.

One of options on the Exam Management page is "Proofread & Edit Exam." When this option is selected, the first question in the selected exam is shown. It appears as it will be presented to the student, except that the correct answer is shown with a checkmark. After reviewing a question, the instructor can choose to see the next question or the previous question, or can use the "Jump To" option to select a particular question from the test. This feature allows the instructor to examine any or all of the test questions.

If, when a question is presented, the instructor chooses the Edit option the question is brought up in edit mode. The question text, the answer text and the designation of correct answer(s) can be edited on this page. Text is changed by placing the cursor on the area to be corrected and typing as with a word processor. Correct answer designations on multiple choice and multiple answer questions can be changed by clicking on the appropriate buttons. In changing correct answers on multiple response questions, the instructor must click both on new distractors to make them correct, and click on previously correct distractors to make them incorrect.

Placing a multimedia file into a test (Section 6.1.4) causes HTML code referring to that file to be inserted into the question text. To remove the multimedia, this HTML code should be deleted from the question in the same manner as any other unwanted text, i.e., select (highlight) the code with the cursor and press the delete key.

Additional editing can be done through these options on the edit page:

6.1.1 Add a Distractor.

A text entry box is appended to a multiple choice or response question if the "Add a Distractor" option is selected. The instructor can then type in the text for this distractor and can click on the associated button to make it a correct answer. After a distractor has been added, it may be necessary to add a new feedback message for this distractor (see Section 3.4 and Section 6.1.3).

6.1.2 Delete Last Distractor.

The "Delete Last Distractor" option will remove the last listed distractor. Only the last distractor can be deleted. After deleting this distractor, the instructor should confirm that a valid correct answer remains. After a distractor has been deleted it may be necessary to delete the feedback message for this distractor (see Section 3.4 and Section 6.1.3).

6.1.3 Other Editing Options.

Question weight, category, feedback and question type can be changed or added new on this screen. Text boxes appear for weight, category and varieties of feedback. Text is entered or changed in these boxes by clicking the mouse in the box and typing the appropriate changes.

A pull down menu for "Question Type" lists all Chi Tester types (seeSection 3), and allows the instructor to select a new type of question. This change of structure has implications for several question parameters and should be done with care. When a multiple choice or response question is converted to short answer, the correct answer is listed as the previously correct answer distractor letter or letters, e.g. ans: c, e. This would need to be corrected on the question edit page. When a question is converted into multiple choice or multiple response distractors, must be added through that option (Section 6.1.1). Feedback may also have to be changed to avoid inaccuracies and confusion.

6.1.4 Add Multimedia.

Section 3.7 should be consulted concerning the types of multimedia files which Chi Tester supports. When an instructor selects the "Add Multimedia" option, a screen appears with two choices: "Add New File to Question" and "Add Selection to Question." The second choice, "Add Selection to Question," allows the instructor to add a file which is already in the test to a new question. To get a file into the test initially, the "Add New File to Question" option must be used.

With this "Add New File to Question" option, the instructor must enter the file name (along with path or folder location) in the file name box provided. Some browsers will allow the file name, with full path, to be typed directly into this box. Some will not. All browsers should allow the instructor to use the "browse" option to search for and to select the multimedia file. The standard browse routine on some browsers may be preset to select files with ".html" extensions. To see and select other files, the instructor must change to ".txt" files or "all" files. A browse option is offered to allow a search and selection of the target multimedia file. Once entered, the instructor clicks on the "Add New File to Question" box and the multimedia file will be presented with the question.

Every file which is added to a question on the test is also added to a list which appears above the "Add Selection to Question" option. The name of the file appears on this list along with a radio button. If the file name is clicked, the multimedia file is shown or played, so that the instructor can confirm that it is the desired file. If the radio button is clicked followed by the "Add Selection to Question," the file will be added to the new question. It will appear with both the question for which it was previously submitted and the new question.

HTML code to link to the multimedia file is inserted by Chi Tester into the target question. If the file is of a type which is not directly supported by Chi Tester, then additional code appropriate for this file type must be inserted here by the instructor (Section 6.1). To remove a multimedia file from a test, edit the corresponding HTML code out of the question from which the file is to be removed (Section 6.1).

6.2 Edit Exam Parameters.

When an exam is initially created, the instructor specifies test times, sites and presentation mode (e.g., test, tutorial, survey or evaluation) (see Section 5.2). The "Edit Exam Parameters" option allows the instructor to make detailed changes in these individual parameters and to define new custom modes for presenting questions.

Exam parameters are organized into six groups: Test Information, Question Information, Site Information, Feedback, Test Review and Security. Each exam parameter and its options are listed and explained in subsections listed below:

6.2.1 Test Information Group.

Test Name: The previously entered test name appears here and can be edited.

Owner: The submitting instructor's login name appears here and can be changed to another Chi Tester instructor using the button, "Transfer Exam to Another Instructor."

Availability: Starting and ending days and times for the test are entered here and can be changed if desired. Chi Tester will present tests only between these points in time.

Instructions: Text entered here will appear at the beginning of each test. The instructor might type in a greeting, a warning, or instructions for special questions.

6.2.2 Question Information Group.

Total Questions: The number of questions each student will receive (out of those in the loaded test) is entered here. If the instructor wishes Chi Tester to give only a subset of the total questions, that number should be entered here. For example, if the student should only see and answer 30 of the 50 questions submitted, then the number 30 would be entered here. Specific questions are randomly selected for each student.

Randomization: If this box is checked, then questions will be presented in randomized order in the test. This option is especially useful when students are taking a particular exam at adjacent computers in a testing center environment. With this option, random selection is made from all questions in the test. For random selection from categories see the following paragraph on Categorized Randomization.

Categorized Randomization: For tests in which questions are categorized, the instructor can specify the number of questions to be selected from each category. If this option is selected, the instructor is given a list of categories on the test and picks the number of questions to be selected from each category. Questions for which no category has been named cannot be selected on a test of this type. Section 3.5 describes question categories.

6.2.3 Site Information Group.

Sites: The instructor can select a testing center from the list of registered sites provided or indicate that the test can be taken "Everywhere Else" on the Web. Sites are testing centers whose computer IP addresses have been registered with the Chi Tester administrator. Tests will only be offered on computers with IP addresses at the sites selected from this list. "Everywhere Else" means anywhere that is not in a registered testing site.

Site Planning: The number of students expected and the expected test duration can be entered for use by the test site personnel in facilities planning. This is optional, however.

6.2.4 Feedback Group.

Score: If this option is selected, the student is given both the number correct and incorrect upon completion and submission of the test.

When: The instructor designates when test feedback, if any, will be shown, either during the test or during later review.

6.2.5 Test Review Group.

"Enable Test Review": If the instructor wishes to allow students to review the test after completion, then this option is selected. This feature allows the student to re-access the test for review. During the review, the student can see questions, an indication of the correct answer(s) and feedback (if any), but cannot change his or her answers. If the review option is selected, the following options concerning when, where, what, and for how long review is possible are relevant:

Password: Any string typed here is the test password and must be entered by the student or proctor before a student can get access to an exam.

Take Test: The instructor can allow any student with any ID to take the exam. Otherwise only students with IDs entered in the list on this page will be allowed to take the exam. Or, the instructor can allow access to the test anonymously, that is, with no ID. Finally, the instructor can restrict the test to be given only once for a particular ID.

Go Back: When this option is selected, students can go back and change answers to previous questions in the test before the test is finally completed. Otherwise, the student cannot return to earlier questions.

Privacy: This option prevents the instructor from seeing names or IDs of students. This would be important for course evaluations, polls, and surveys.

Who: Specific ID=s can be entered here. Only students with IDs on this list will be allowed to take the test.

Allow One Testing: If selected by the instructor, students are not allowed to repeat the test. If this option is not selected, repeat tests are allowed. In the latter event, all scores are saved and reported to the instructor.
 

6.3 Changing Exam Parameters.

Exam parameters can be changed at any time by accessing the "Exam Management" area as described above (Section 6).

6.4 Exam Passwords.

If established by the instructor, a password can be required before an exam is presented to a student. Giving the password to the students who are to take the exam will restrict access to that select group. For even greater security, the instructor can give the password only to an exam proctor, not to the students. This proctor can require identification from the student exam-taker before the proctor enters the password, allowing the student to take the exam (see Section 6.2.6).
 
 

7. EXAM SCORES

"Exam Management" offers three different score reports and a rescore function. The options are:

7.1 Data Ordering in Reports.

Student exam results are presented in alphabetical order by name (the default) or in ascending order by date. The selection of name or date is made by the instructor from a pull down menu on the "Report for Instructors" page. Question results are presented in original file order. That is, Question 1 is the first question in the original test file; Question 2, the second; and so on. Since questions on a particular test may be presented in random order as it is being taken, this may not represent the question order seen by any particular student during the exam. Assuming post-test,  in-class review, this reporting protocol means the instructor need not be concerned about the actual order that any student experienced during the exam. On the report, Question 1 will be the same for all and will match the order on the original exam.

7.2 Instructor Report.

If the instructor selects this option he or she is given a menu of possible items for the report. The instructor first clicks the desired items for the report, including a choice of having the report sorted by name or date student(s) took exam (from a pull down menu on the screen). Then the instructor selects the "Generate Report" option and the report with the desired information appears on the screen. Summary statistics, mean, median, standard deviation, range and high and low scores are shown at the bottom of the report. The browser print option can be used for hardcopy, or the Spreadsheet format can be chosen to obtain a report which can be saved into a file and imported into a spreadsheet gradebook.

The report can include student name, ID, email address, score (raw and percent), time and duration of test, and the site and IP address at which the exam was taken. Information fields are reported left to right in the order they appear on the menu. Additional individual score and format options are described more fully in Sections 7.2.1 through 7.2.4, below.

7.2.1 All Test Answers.

A report with this option, selected shows the actual answers submitted by the student. This allows the instructor to evaluate essay questions and other questions which do not have correct answers. The report format is adjusted to accommodate the additional information. If desired, the instructor can include a page break between individual student results in order to get more readable printouts when printed through the browser. However, different browsers display the same HTML code differently so the report format may vary unpredictably. If the appearance is unsatisfactory, try a different browser.

7.2.2 Show Incomplete Tests.

If a testing session is terminated without going through the normal exit routine, the answers are saved, but the test is "incomplete." Incomplete results are not shown in a report unless specifically requested by the instructor by selecting this option. Students who use the normal stop option, but who have not answered all questions, are considered to have completed the exam and their scores will appear in the default report. Incomplete results are only those from students who exited the browser without exiting the test, or whose computer crashed in the middle of the test.

Summary statistics such as test mean, median, etc., do not include incomplete test results.

7.2.3 Delete Buttons.

If this option is selected, a button is placed in the requested report next to each student's results, under a column labeled "Delete." This allows the elimination of repeated tests, if retakes have been allowed, tests taken by students who have dropped the course or any other extraneous results which should not be included in summary or report information. Such deletions are permanent.

To delete results, the delete button should be checked for each test to be removed, and the "Delete Checked Results" button should be selected at the end of the report. Only when this last box is selected are results actually deleted. When this option is selected, the requested report minus deletions is regenerated.

7.2.4 Spreadsheet Format.

When the Spreadsheet option is selected, items in the report are provided in an output format that is appropriate for import into a spreadsheet. After the report is generated, the requested information will appear on the browser window sometimes shown in a run together mass. It should then be saved using the browser's FILE/SAVE AS option. (On some browsers it is necessary in saving the file to set the file type to "Source" so that it will be read correctly by the spreadsheet.) The saved file can be imported into a spreadsheet with each student's information on a separate row and each item in the report in a separate column.

Some care should be used in the spreadsheet format as these reports will be somewhat different in appearance in the spreadsheet. The field width in the spreadsheet may be too narrow for the report field. For example, the Date/Time field may be truncated and may require extension. The format for test duration on the web report is hours, minutes, and seconds. In the spreadsheet, it is simply total seconds. The percent correct on the web is likely to be rendered as the default proportion in the spread sheet, e.g., 66% on the web, .66 in the spreadsheet. Finally, if the test includes essay questions, it would probably be inappropriate to request all answers in spreadsheet format, since essays will extend as a long text string far to the right.

7.3 Student Reports.

Individual score reports, formatted so that the instructor can print, separate and distribute them to individual students, are available. When this option is selected, reports for all students who took the exam or survey are displayed in the browser and can be printed from there for distribution.

7.4 Item Analysis.

This option, if selected, displays the number of students who selected each question distractor. The instructor can choose to see this analysis in graphical or tabular form. The graphical output includes a bar graph (histogram) for each question, with bars representing the frequency of each distractor selection. Filled bars represent the correct answer(s).

In the table output, the data is presented in the order the questions appeared in the original file, one row per question. The columns represent the distractors which are the possible answers, the first column for distractor a, the second column for distractor b, and so on. The table entries are the number of students who selected the column distractor on the row question. For short answer questions, two columns are displayed. The entry in the first column is the number of students who answered correctly. The entry in the second column is the number who answered incorrectly. For essay and other non-scored questions, a single column is displayed showing the number of students who responded to this question.

This report can be saved into a file using the browser FILE/ SAVE AS menu option. The file can then be imported into a spreadsheet or other analysis program.

7.5 Rescore Results.

It is sometimes brought to the instructor's attention by an unhappy student that the answer key is wrong. In Chi Tester, this is equivalent to entering the wrong answer after the ans: code. This error can be corrected by editing the question text and supplying the correct answer, (see Section 6.1). Since Chi Tester evaluates answers as they are submitted, this in and of itself will not correct erroneous grading. If the "Rescore Results" option is taken, however, Chi Tester will reevaluate answers with the presently correct designations, thereby correcting the scores. After rescoring, the instructor can re-generate desired reports.

Caution: If the test is to be reused after changing correct answer designations, the instructor should check the test feedback to insure that it is consistent with the new correct answer.
 

8.  TUTORIALS

In a tutorial, students answer questions and are given informative feedback to their answers. They are allowed, or even encouraged, to repeat the tutorial and review the material that it contains. The purpose here is one of  learning rather than evaluation.

To construct a tutorial, the instructor should include feedback with each question, Section 3.4. This feedback will be a fundamental part of the educational experience, so it should be clear and content oriented, either explaining concepts or directing the student to text materials which explain the topic.

When a tutorial is submitted and set up, (see Section 5), the instructor should select the Tutorial option in Test Setup (Section 5.2.1). The particular presentation parameters which are set for Tutorials can be changed in Exam Management (see Section 6.2), if the instructor wishes to modify the tutorial interchange. Default parameters which might be of particular interest are Test Review, which is not enabled; question order which is not randomized; Password, which is not set so that the tutorial can be accessed without password; and ID, which is not restricted, so that any ID can be used.

Student answers are recorded and evaluated as normal and the instructor can generate reports in the normal way to assess progress (see Section 7).
 

9.  SURVEYS AND EVALUATIONS

In a survey, answers are recorded, but not evaluated. This is accomplished in Chi Tester by providing no correct answers with the questions. The answer codes, "multiple choice," "multiple response," "short answer" and "essay" should be used as appropriate, (see Section 3.3.5).

When a survey is submitted and set up in Chi Tester, (see Section 5), the Instructor (e.g., surveyor/evaluator) should select either the "Anonymous Survey" or "Course Evaluation" option in "Test Setup" (see Section 5.2.1). The particular presentation default parameters which are set by Chi Tester can be changed in "Exam Management," (see Section 6.2), if the instructor wishes to modify the survey format. Parameters which might be of particular interest are question order, which is not randomized; "Password," which is not set so that the survey or evaluation can be accessed without password. For the anonymous survey option no ID is required. For the "Course Evaluation" option an ID is required, but if the instructor (i.e., surveyor/evaluator) wishes to restrict access to only students in a specific course then the IDs for these students must be entered (see Section 6.2.6). For the "Course Evaluation" option, although an ID is required, it will not be shown on reports and the instructor cannot access it.

Probably, the most useful way to report survey or evaluation results for multiple choice or multiple response questions is the "Item Analysis" option, (see Section 7.4). This provides a count of each distractor choice for each question. This report can be saved into a file and imported into a spreadsheet. For short answer or essay questions, the "All Test Answers" option should be used, (see Section 7.2.1).
 

10. ARCHIVING AND DELETING AN EXAM

When all students have taken an exam and the scores have been properly recorded, the instructor should delete the results from the Chi Tester database. The test will remain and for a new test group the instructor can edit the exam parameters in "Exam Management" to set a new testing period (see Section 6.2.1). If the test will not be taken again soon the instructor should delete both the results and the test. If the test is to be deleted, but might be given again at a later date, the instructor can save the exam onto his or her own computer system, thereby saving any editorial changes which were made in Chi Tester. These three functions can be performed from the "Archival & Deleting" section of the "Exam Management" page, which is accessed from the Chi Tester homepage.

To perform these functions the instructor must log in to "Exam Management" (see Section 4).

10.1 Save Exam Text.

When the instructor selects this option, he or she is asked whether the exam parameters should also be saved. If this option is not selected, only the exam text will be saved. What happens after the "Save Exam Text" option is selected depends on the browser. It may save the text automatically and it may request a name for the file to be saved. Alternatively, it may display the test text on the screen. In this case, the instructor should select FILE/SAVE AS from the browser FILE menu and specify the name and the path where the file should be saved. For ease of later identification and possible reuse, the test file should be given a name that codes the test and the time it was administered.

10.2 Resubmitting Saved Tests.

The test file saved using procedures described in Section 10.1.1 can be directly resubmitted to Chi Tester as described in Section 5.1.

10.3 Delete Results.

After selecting this option the instructor is given a confirmation page. If the "don't Delete" option is taken nothing is deleted and Chi Tester returns to the "Exam Management" page. If the "Delete Results" option is taken the results are deleted.

10.4 Delete Test & Results.

This option is identical in form and function to "Delete Results" (see Section 10.3), except that the test is also deleted.