Interesting article, I enjoyed it.
Extending on what you said, if the a=b test is for assignment and sequence, I would suggest that coding a solution for the Knight’s Tour
would be a good test for recursion / iteration. To me this begs the question, what would be the concurrency test?
An intelligent, long-form, highly referenced treatment of the subject is in the book “making software” by O’reilly. It’s chapter 7, “Why Is It So Hard to Learn to Program?” by Mark Guzdial.
44% of students formed a consistent mental model of how assignment works (even if incorrect!)
39% students never formed a consistent model of how assignment works.
8% of students didn’t give a damn and left the answers blank.
Oddly, approximately 9% of computer science students are unable even to perform basic tasks such a comprising a portion of 100%.
The authors of the paper have rejected their own thesis in the meanwhile!
Please update your article!
“We cannot claim to be separating the programming goats from the non-programming sheep”
[Source: http://crpit.com/confpapers/CRPITV78Bornat.pdf ]
Indeed, as of July 11, 2014 author has now retracted the paper in full. The retraction is available here: http://www.eis.mdx.ac.uk/staffpages/r_bornat/papers/camel_hump_retraction.pdf
Thanks for that, quoting the conclusion of that retraction:
There wasn’t and still isn’t an aptitude test for programming based on Dehnadi’s work. It still appears to be true that novices who answer ‘consistently’ in the test are more likely to pass a programming course. Current work, by others, begins to suggest reasons for the phenomenon and open future research avenues.
So, what was the right answer? I don’t know how to program, but after I read this test,
Read the following statements and tick the box next to the correct answer.
int a = 10;
int b = 20;
a = b;
The new values of a and b are:
a = 20 b = 10
Because, we have a=b, so you can change numbers vice versa. This is my logic from the statement I see.
I must say that I take offense at the title of this article, which implies that greater intelligence is a barrier to accomplishment in the field of computer programming. We raise goats, and I can assure you that goats are smarter, easier to train, don’t smell half as bad, and are better looking that sheep. Sheep, on the other hand, are so dim that they can be used as a light-sink. They’re so stupid that describing them as being “as dumb as a box of rocks” is an insult to the rocks. Sheep smell so bad that skunks avoid their pasture. The only thing more stupid than a sheep is another sheep. Sheep will not only bite the hand that feeds them, they’ll trample that hand and whatever it’s attached to in a panicked attempt to get away from the feed, which they believe might be out to attack them. Goats, on the other hand, are cheerful, playful, affectionate, and…OK, pointlessly and brutally violent, but only to other goats. If ever I was to attempt to teach a ruminant to program I’d choose a goat for my experiemental subject - although they’d probably want to eat the keyboard and stand on top of the computer.
I haven’t read the original thing, or the retraction quoted, but a shallow skimming of this makes it clear that there are a lot of issues with that thing. Two quick things: the quoted question is extremely bad since it appeals exactly to how CS in general has completely butchered the meaning of =. Similarly, I can show you x = x+1 and ask what it means – people who haven’t programmed see an obviously bogus statement, and people who have see mutation. The second note is about the double hump – it’s not like I have any proper data, but from my private experience of teaching a course that traditionally always had a double hump I concluded that this is due to improper teaching. The thing is that it’s extremely easy to get carried away and sail away into faraway lands where things are very confusing to the average student. When this is done, some small group of excited students will actually follow you there and will actually know what you’re talking about, but a larger portion of the class gets more and more lost, and at some point they just give up and switch to automatic emergency mode of trying to write as much stuff as possible and maybe they’ll get it later. What almost always happens is that they don’t get it, and they get left behind to make the lower (and bigger) hump. I can see how an aptitude test that appeals to what people know about programming is likely to stumble on people who are excited about it pre-college, and therefore are more likely to be in the sailor group, and OTOH this is a popular subject (lots of money, right?) which leads to more people trying to go for it than what they actually can do.
That link is dead (but this post’s popularity is not).
Maybe the proper link now is http://www.eis.mdx.ac.uk/research/PhDArea/saeed/paper1.pdf, via http://www.eis.mdx.ac.uk/research/PhDArea/saeed/?
I think that the issue boils down to the developmental progression of Concrete Operations to Formal Operations to Post-Formal. People vary in not only where they are but in how far they might possibly go. Formal functioning is learned in adulthood, if one engages in the sort of things that work that muscle, but College is late to begin accumulating the 10,000 hours needed to be good at something. So, in short, If you have aptitude for something, Then don’t wait until shortly before you have to support yourself to start learning it - on your own and as avidly as you can. Else …
I do not understand the confusion about the assignment statements. In school people see things like “y = mx + b”. There is no uncertainty about what that equal sign means: ASSIGNMENT! It shouts at you. How else can anyone see it? How could someone wonder about the right to left or whatever? If people are that far behind in schooling, they need other things before learning programming.
DarrellP, your Mother’s view is called the “Concrete Operations” developmental stage. No, such a person cannot generalize situations to form an algorithm. They did not learn such a skill early enough in their life to be at all likely to ever learn it now.
So how come I intuited off the bat that “a = 20 / b = 20”, and I’m no programmer?? Have you ever had trouble with trying to read a software manual? No answer in this “academic” paper, and most all the comments give no clear solution for “non-programmers” — to simply have and learn from a clear example, for backwards correlation. Sorry guys, but I gotta say that programmers can be an insulated lot it seems, and they appear to like the power of such priesthood. Sheep, goats, such insulated-inbred reasoning defies communication and ignores the social awareness to teach, seems to me. Logical or quantum proof or poof, please!
Oh yes, because it is so difficult to understand that the new value can be the same as the old value.
Hey fred, I bought a new car!
Wow nice, you finally got rid of that Fiat Panda! What’s your new car?
Another Fiat Panda.
So difficult!
okok… old topic but very interesting.
Just consider viewing it from another point of view. If the biggest flaw of this test is knowledge about the rules, why not explain them? The test could be:
= means copy a value from right to left. The value on the right is not changed.
Every line is executed one after the other from top to bottom.
a= 10
b= 20
a= b
What is the value of a?
What is the value of b?
So what do you think how many people would pass the test now? Probably around 100%, right? Of course you can inflate the topic by saying you want people to apply a consistent rule, blah blah, but just explaining a clear rule makes a lot more sense.
But only if you are interested in results…
The point why the test is how it is is pure bigotry. This test tells a lot more about the testers than the testees. It is just made this way to prove some kind of prejudice and to rise over others. No Rocket science, just academic bigotry.
This is such an incredible stupid question. Why would you assume that the student who was never exposed to coding knows that a=20 is an assignment statement, or that a=b assigns value of b to a.
In minds of ‘normal’ people ‘=’ establishes some kind of equivalency. The meaning of ‘assignment’ is a coding convention and you are asking people who are not familiar with such conventions to guess them out of the thin air. Ridiculous.
B did not start with a value. There is no old value for B, it’d just be null. Since B was assigned a value, it is the new value. Contrast this with A which was also unassigned to start but it was assigned a value and if you stopped there then that value would have been the new value. However the algorithm continued and then A got reassigned to a new value, thus rendering the previous value the “old value.”
B did change. It changed from null to the new value.
I think you missed the point. Because, I decided to do this, too. In fact, it also states that they presumably teach this topic. When i found people who couldn’t pass the assignment test, i tried doing the same thing, and they still failed. So, the question was, “Ok, why are they failing.” The issue, in particular, is that they got it stuck in their heads that = means equality. So when you show them b=a, their mind automatically assumes that A and B must have the same answer, and then they rush to the multiple choice section and pick a result. I get the same problem when addressing issues of hex-decimal and binary as well: people can’t understand how 0x10 == 16, and it breaks them mentally. Even when you directly tell them that the symbol has a different meaning in this context, it goes in one ear and out the other, and if you tell them they can’t put something like 2*x-4 = 10; and expect x to hold the value of 7, they break yet again. It’s like taking a flat earther into space.
And there are those people who insist in ‘a’ can’t be anything but an ‘a’ even if you say ‘a’ equals 10 because they can’t see meaning as fluid.
There people also look at binary data can not seem to see FF or 255 are the same number just a presentation. And they end up making math function take string and return string all over the place to able to work with such number!!