HOME – Previous section of this example
Next section of this example -->
College Life: Testing the model
The student's basic problem: within constraints that I cannot control, how much of my time do I allocate to which activities?
The goal is to turn the situation that contains the problem into a system ... then use the system to test possible solutions to the problem.
Where I am: I have developed a model in the form of a flowchart The model can generate daily schedules for the student.
What I do next: I need to make sure that the model is accurate, that it has no logical flaws, and that it fits the needs of the student. The best way to do the tests is to invite some students to use the model to help them allocate their time. Then I can ask them for feedback regarding our concerns.
Then, based on the results of these tests, I will probably need to revise and/or refine the model to improve its accuracy, logic and feel.
The General Method
Testing is not an attempt to prove that your model is right. Instead, it is a serious attempt to prove that your model is wrong.
It often surprises people to hear that there is no way to prove, beyond any doubt, that a systems model is correct. But there is not. No matter how much evidence you gather to support what you have done, there is always at least some chance that you have missed something that matters. So I don't bother trying to prove that I am right.
Instead, I work at trying to prove that I am wrong. I say this: "if I can't prove that the model is bad, then probably it's OK."
So, testing begins by showing the model to critics and potential users. I ask the critics to find errors or faults in the model. I ask the potential users to try the model out and tell us about anything that they don't like.
Important Note: Testing usually produces a many questions and concerns. Each of these has to be addressed - sometimes requiring new data, or even building a whole new model. The testing process might have to be repeated many times. The small number of examples I'm going to show here do not reflect the many weeks of time that would go into the testing of an actual model of this size.
Testing for Accuracy
Is the data I used in building the model accurate, and are the methods I used in the model are complete and correct?
• As noted in the previous tutorial, the model lacks details as to how the student will resolve conflicts when two events overlap.
I try to gather more facts, and after talking to a number of students, I find that different people do this differently. Everyone agrees that red events have priority over pink and yellow ones. E.g., if my work hours conflict with a course, then I have to choose a different course ... because I simply cannot quit my job. But beyond that, the decisions may be very difficult and each person will have to come up with his or her own method. E.g., if work hours conflict with soccer practice, then I may have to try to find a new job ... or maybe the coach will let me miss one session each week, or ... maybe something else.
Consequently, there is no way I can provide a basic method for resolving conflicts. All I can say is: "the student has to decide which event to keep and which one to drop."
However, even this is a benefit. The process of building the model has shown us that scheduling can force students to face some very difficult choices. This is useful information. Perhaps the college's counseling center should offer services to student who need to make tough choices like these. And maybe the model could then include an option called: "if decision is too difficult, go talk to a counselor."
Again ... this points out the basic fact that the thinking process is important here ... not the model itself.
Critical Testing
Are there any logical reasons why this model might not work?
• I might question the basic assumption that the student's problem has to do with time management and scheduling. Maybe the student is just lazy ... or maybe the student is depressed and just doesn't want to be in college at all.
Of course, if this is the case, then an attempt to use the model will fail. So maybe the model could be used as a first step ... and would turn into a kind of diagnostic tool if the student still couldn't make a schedule.
• The model is set up in one hour chunks of time. Many of the events do not fit neatly into hour long units. Maybe the model should have fifteen minute chunks ... or even one minute chunks.
This is easy enough to do. However, the more lines I put in the printed result, the more complicated it looks to the reader. I like the approach I used, which has a table off to the side that shows the exact times, and uses the one hour chunked graph as a visual reminder. But it's not what I like that matters ... if students prefer a different time scale, then that change will be made.
How Does It Feel?
Do you like this model? Will it solve your problem? Will it satsify your needs?
• The flow chart is very confusing.
This is a serious concern. I don't think I would want to hand the flow chart to a student and say, "follow this." But ... the flowchart can be represented in a number of different formats. It's just a series of questions and answers. It could be put in the form of a booklet ... or it could be coded up as a computer program. One way or another, this will need to happen.
• One of our testers might point out that she values time spent with her friends just as much as time spent in class. I have made "friends time" yellow, and she thinks it should be pink.
The student decides what priorities his or her events will have. Maybe a student works part time in his mother's company, for example, and can decide his own hours. In that case "work time" could be yellow. The first step in the scheduling process is to identify the events that need to be scheduled and figure out what priority each needs to have. The rules for setting priorities need to be clear, and I think they are, but the model will have to be tried out in practice to be absolutely sure about that.
• Another student found our use of "R1", "Y1", etc. to be confusing.
Maybe I could just number the events "1", "2", "3", and so on and let the colors speak for themselves. On the other hand, the "R1", "Y1", etc. labeling shows the priority of the event in the tables where color cannot be used. I'll test this further; if most students are not confused by the current labeling, I'll keep it.
Am I Done?
Maybe. It depends on what I was hired to do.
Suppose, for example, that I was hired as a consultant by a company that wants to design software for sale to colleges. While the company has its own experts, they wanted an outside opinion on how to proceed with their project. In this case, I would write up my report and hand it to them. Assuming they were satisfied, they would pay me, and I would have no more to do with the project.
On the other hand, suppose that I work in the student affairs division of a college. My boss may have asked me to evaluate the division's methods of advising students on how to choose their courses. Having finished this report, I may be asked to help make changes in the current way of doing things.
In either case, my system model is just the starting point for the work to come. This is always the case. The model is a thought tool whose purpose is to provide insight into the problem. Construction of the solution to the problem uses the model as a starting point, but procedes under its own steam.
Next section of this example -->