It might be worthwhile for people to mentally review what exactly a single study means in the realm of science.
It is neither more nor less than evidence. It may have been misinterpreted. It may be the beginning of a solid proof. As yet, though, you can’t know which is which; it is merely an interesting tidbit.
Preparing to use this test in a real college environment or decrying the study as rank elitism are both premature.
I’m prepared to believe that perhaps the standard teaching methods are questionable and that with other methods the currently “unreachable” students might yet learn what is meant. In fact, this test is useful simply by virtue of providing a discriminator; much of psychology works that way. We can’t actually know whether a person has “schizophrenia”, because we don’t even know exactly what “schizophrenia” is, but we can study how two populations, divided by a standard “schizophrenia battery test”. Without that tool study is borderline impossible.
On the other hand, I’m prepared to believe that the ideas hinted at by the study are true and there are people who just can’t get it. I saw senior-level students who I seriously wonder if they could pass that test, and after four years of experience I daresay teaching methods should be of minimal consequence; the teaching method of the compilers is both constant and rigorous.
It’s worth further study. If it does turn out that there is a segment of people who literally can not understand computer programming, and it turns out we can identify them with high reliability, then we owe it to those people to so identify them before they spend years going down the wrong (for them) path.
“Encouragement” is not an intrinsic virtue. “Encouraging” somebody who has been mountaineering for a week to take on Everest is not a positive thing, it’s postively murderous. Encouraging somebody who can’t learn to program to learn to program is not a positive thing, it’s evil. If this study does reveal something, then the authors are not evil elitists, they are the heros of every non-programmer the test identifies.
Hey, I didnt do to well in algebra or physics, but I got a 3.99 GPA in programming, I did however such in English (“but you coders seem to have ZERO respect for the actual language us humans use”) I guess that statement might be true.
I remember looking at code before attending my first “Introduction to Programming” class and not understanding how something was “equal” to something else. This was one of those “aha” moments where the pieces came together. I would have failed the question before the class but that simple stepping stone opened worlds to me.
While I agree that the language specifics isn’t really a problem, they are not meant to get it correct, only show a consistant model the C style asignment operator is confusing to everyone I’ve seen exposed to it for the first time. “:=” is a little better, and then something like “let x = 10” next, and then something like “let x be 10”.
I wonder how many students would have answered false in the question if it had been there… mind you, I taught a guy a language after he had read the teach yourself language in 21 days, and been on a 2 week course and had a degree in computing… still didnt know what an integer was.
having been taught programming as part of an engineering course, where the course book was along the lines of ‘C for idiots’ (which was actually a good choice as it turned out ), by a lecturer who understood FORTRAN and didn’t have a clue what ‘C’ was let alone how to use it…
I spent many an hour with the only other person on the course who understood anything (far more than I did) running side classes to explain things. ironically the lecturer ‘attended’ these classes as well.
these tests would have been a very good idea. put the people who pass into one stream and the people who fail into another, teach to peoples abilities, then a bit further the stretch peoples minds.
there is not point baffling people, programming isn’t hard once you get the basics. its just that ‘getting’ the basics can take time and needs to be taught well, by which i don’t mean the all to common answer during my education of “you don’t need to know that to pass the exam” which really annoyed me…
Is C (based) syntax. You have to learn C to know the correct answer. If someone does not know C (or any other programming language) they could easily not know that = in this case is an assignment.
Assuming the student being tested to know such syntax is ridiculous (after all whats the point in the int keyword? or the semi-colons?). The person who designed the test obviously forgot that they themselves know how to program. Either that or they were just lazy to think up a more generic language.
All the same, interesting stuff. Might be good to see the test conducted on actual so called professional programmers as well (not just students). I wonder how many “professional programmers” have no aptitude for programming?
44% of students formed a consistent mental model of how assignment works (even if incorrect!)
39% students never formed a consistent model of how assignment works.
They didn’t just look for “correct” answers, they looked for consistency. So, if you don’t know that a=b means the value of b gets assigned to the value of a, then the group in the first bullet would create a mental rule for what it means, and consistently apply that rule throughout the test. The folks in the second bullet would ‘change’ that mental rule for each question.
Now, the question is, if you didn’t know, and you were guessing, did the folks from the second bullet alter their rules in order to get some of the questions right, instead of applying the same rule throughout the test. The whole, “well, I don’t know how to do this, but I want to get some of these right” attitude.
As an individual who has gone through many MANY programming courses, it’s well known to most people that these two groups exist. I was dismayed in my first year of engineering study when 90% of the class aknowledged that they had already learned Java (program we were studying) and many of the topics covered by the class. I was not one of them. By the end of the term, the class average was somewhere around 50% (tough university as well), with most students failing and struggling to even understand many of the basic concepts taught in class. Regardless, without much effort, I managed to pull out one of the highest marks (sorry if it sounds like i’m bragging).
So why does this divide exist? Obviously it has nothing to do with how smart an individual really is, but more so how they think. In my opinion, actual programmers are clearly able to see the big picture, and how little bits and pieces relate and affect everything else. Sadly, with how simple programming is to teach/learn, the industry will continue to churn out huge amounts of incompetent programmers.
Exactly the same thing happened in the Kalmar Laszlo Computer Sciences School (Or whatever is the english name of it) in Hungary. Btw it seems to be true: “The two most common elements in the universe are Hydrogen and stupidity.”
While the question is indeed vague and the asignment vs. equator could be an issue, I don’t think it really is. Given that the answers are not “true” or “false” the function of = is fairly obvious.
The C style syntax is not a problem either. What they are looking for is a consistant mental model. If it’s some crazy language where = moves the value from b into a and leaves b empty, then that should be reflected in all of their answers.
A double hump is not limited to programming. In elementary geometry, there is a classic problem called the Pons Asinorum (“Bridge of Asses”). Some students handle it swimmingly, others require extensive handholding. It has also been said that regarding advanced mathematics, only about 15% of secondary students really “get it” and the other 85% just follow along. Knuth once remarked that in Stanford CS classes, only about 1-2% of the students were at all fluent when it came to programming. Gerald Weinberg in “The Psychology of Computer Programming,” noted that unlike most fields of endeavor, where the difference between the best and worst practitioners might be a factor of 2, in programming, even after being measured with a wide variety of metrics, the difference can be as much as a factor of TEN.
Readers should note that the dedicated professional teachers all recognise the two humps.
I have attended many teaching conferences where young aspiring teachers wonder what they are doing wrong. Eventually, they recognise the two hump problem. Some students just don’t get it.
To all those who are insulting the article’s authors’ teaching ability, please consult the literature on teaching introductory programming.
And point of the test was not to find a way to discriminate between goats and sheep; as pointed out in the article (please read it) this was a complete accident. This was discovered by subsequently analysing the results. So this “so called study” is simply pointing the way. The lovely thing about this paper is it presents what they discovered and offers entertaining insights and opens up a very useful approach for studying potential programmers with a view to improving teaching. It is far more useful than so many of those so called “real” studies.
I completely agree with Scatman Dan, “I’m disappointed by the number of software engineers who read this article and fail to realise this!”
"So why does this divide exist? Obviously it has nothing to do with how smart an individual really is, but more so how they think. In my opinion, actual programmers are clearly able to see the big picture, and how little bits and pieces relate and affect everything else. Sadly, with how simple programming is to teach/learn, the industry will continue to churn out huge amounts of incompetent programmers."
Daiken on July 17, 2006 10:46 PM
*right u are!
From the beginning I would like to say that a programming course does teach openminded students something…
I think the test is a good tool both for teachers and the students. The students receive a signal that they might pretty much be wasting their time and money for soemthing they-re not suited for, or, in case, let’s say…they weren’t yet showing interest by the time of the second testing, they should start doing it, if they really want to be in this business…My ideea is that there are many bright students that “wake up” pretty late during faculty and start seriously working as programmers and maybe be miscathegorised by a test like this, and maybe a “wake up call” early in the process of learning would be good…I sure could have used one earlier
I think this problem is a reality, I sure know some of the “ones that get it” and some who don’t get it, no matter how much I try to explain to them… Whether this is because I’m a bad teacher or because they won’t ever get it is left to study, but there sure is an hell of a difference between those two kind of people in the amount of effort they need to attain comprehension of the basics !
However I wouldn’t say this is specific to CS, I encountered the same phenomenon in Maths (though the ones who are good in Maths aren’t always good programmers). Of course this doesn’t appear as much in college because Maths courses start way earlier and the Maths student are all in the “get it” part…
), by a lecturer who understood FORTRAN and didn’t have a clue what ‘C’ was let alone how to use it…