Separating Programming Sheep from Non-Programming Goats

What pointless emphasis on the mechanics of software! Writing software is a creative process, much like writing a story. Like any other subject taught in schools, you love it, are indifferent to it or you hate it. Grasp of a language and programming concepts are the required underpinnings for being a software engineer, but do not begin to describe the experience of creating successful programs, any more than a comprehensive grasp of English will make you Stephen King. It is facile to try to identify a successful formula for churning out talented logicians, mathematicians, writers or any other human endeavour. It simply identifies the limitations and prejudices of the so-called “educators” conducting the study, and their fears of being caught out if they depart from the line of pretending to teach the self-motivated and ignoring the unmotivated.

How do people with no capacity to grasp basic programming concepts function in real life?

Programming is modelling; It’s a map / territory thing. Someone who can’t learn to program likely can’t read or draw a map of any kind. How do they engage the real world? A human needs a sufficient understanding of real world processes to draw a map of those processes with enough truth to predict the consequences of their actions. A grasp of the rules of gravity, for example, is quite useful.

Interesting - the last I read on the the nature/nurture debate splits things at 50/50. Looks like there is some hardwired circuitry for programming that is either there or not.

The naysayers should realize that the article is based on experience not an educational theory.

What an odd question. Actually, I have no background in programming, and will start my first programming class in a month or so. Logically, a cannot equal b, so you have to look for other options. Without any context and without any history to base your decisions on, the answer could be any of them, because it is impossible to determine the value of =. = could very well be a modifier for the word ‘int’, then create a completely different meaning. If ‘int’ meant ‘multiply this by 0’, that changes the whole context, and a=b is no longer a logical difficulty. I’d say they’re insinuating that there’s a definition of what ‘int’ and ‘=’ mean, and expecting you to know that. Or… what if ; meant stop doing everything here. A would = 10, and b would never occur, and the closest answer for b would b 0.

My background is actually in psychology, and I’m going back to school for UI design. That’s a straight up invalid test, similar to the ‘IQ’ tests that favor one culture over another because of the language used. They’re assuming that the person has at least seen a programming language before. Personally, I wouldn’t want them as teachers.

Bring back cobol

01 A PIC 99.
01 B PIC 99.
MOVE A TO B.

Its not rocket science.

As someone who has been programming for a while, I’d like to say that that is a pretty bad question. Sure, it’s glaringly obvious to us, but it doesn’t just test assignment to variables; it also tests the ability to comprehend syntax by just looking at it.

Lvalues and rvalues are a trivial concept, but it’s pure convention that says “the left side gets the result of the right side.” It’s implied to some extent when the variables are initialized, but this tests the ability to immediately comprehend foreign syntax correctly. If I had never seen code before, I’d say that that either they’re both 10 or that they’re both 20 - but I wouldn’t be at all sure.

I think it’s certainly true that some people have brains that are “wired” for programming, and some don’t - like anything else, it’s a talent that people have to a greater or lesser degree. However, the concept of assignment is simple enough that it can be taught to almost anyone with not too much difficulty.

I find it mind-boggling that people couldn’t learn the concept of assignment after 3 weeks of class, to the point where I think the error is as much in the instructor as the students. Assignment is as simple as putting an object in a box - that’s it.

I’ve had to take a “beginning computer programming” course before, already knowing the language, and I’ve found that instructors, even the good ones, can be pretty obtuse to beginners. On the first day, students don’t need to know that a variable represents a specific location in memory, how it’s possible to overflow a variable, or the representation of a floating-point number. Instead, they should get a solid idea of assignment in the abstract: you’ve got boxes that can hold one piece of data, and each box has its own name. If you put a new object in a box, you’ve got to take the old one out first.

Once the students have CONCEPTUAL understanding, you can talk about implementation, syntax, and everything else. Talking about the stack and the instruction pointer on the first day just makes students’ eyese glaze over, and reinforces the idea that “nobody but a supergenius could ever learn this stuff!” (direct quote from a person whom I helped in a programming class, who went on to understand the basic concepts pretty well).

Teach the concepts, and all else will follow.

“I find it mind-boggling that people couldn’t learn the concept of assignment after 3 weeks of class”

That is not what this study is about. At the first test, the students had no experience with programming. That meant that to solve the questions, they had to create their own model/ruleset about the way assignment works. Students who were able to create a model and use that model consistently for all the questions in the first test, scored on average better on the course exam, than students who didn’t consistently use the same model.

The key word here is consistently. Did the student create a model and stuck with it, or did he use different models for different questions (or was he just guessing). What particular model he used didn’t matter.

The course exam probably didn’t just test understanding of assignment, but also if else constructs, loops, and other subjects. The point is that consistency in the study’s test is a good indicator of how well the student would do in the exam.

So, where’s the FRIGGIN test???

And why is the captcha always Orange? Is this blog based in Florida?

Some comments here (and elsewhere) attack the test’s validity on the basis that it requires the taker to have some prior understanding of a programming language or at least programming concepts.

I’d like to support Thijs by reiterating that this is NOT what is being tested - the marking criteria, as can be seen at http://www.cs.mdx.ac.uk/research/PhDArea/saeed/ are based on the consistency of response and have nothing to do with the “correct” answer under a given programming paradigm.

It is not culturally specific (or at least the culture that it specifies is incredibly broad and hardly exclusive); all that is needed is an understanding of the relationship implied by an equals sign, which anyone with any sort of background in mathematics can deduce in this context.

Okay, just finished reading all the comments and the actual paper and results. Just to give you some background, I am someone who made it about half-way through a CS degree before dropping out due to a divorce, I have been a professional programmer for about 8 years, I’ve written successful products, and programming frameworks from scratch, and these days I can keep up just fine with PhD CS folks (not trying to brag, just trying to keep my background accurate and short).

Let me see if I have this right: The results indicate that there is a correlation between people who can derive a consistent mental model for answering these questions, and people who make it through an introductory programming course.

First of all, I think I appreciate the objectivity of the paper better after reading it–at first, like many posters, I saw this experiment as extremely biased towards people with a previous understanding of the assignment operator. Now I see how that is eliminated that by allowing multiple “right” answers so long as a consistent model is applied.

All that said, I think there are several really dangerous assumptions that this paper draws from these results, and which it implies in its conclusion.

  1. That the “consistent mental model” required to learn programming is something inherent, that learning to think in the consistent manner necessary for programming cannot be taught.

  2. That the 61 students across only TWO courses surveyed at TWO universities is sufficient data for drawing such conclusions.

  3. That these two courses (or any intro CS course) provide suitable instruction for students to learn the “consistent mental model” necessary for programming.

Now obviously this last point happens to line up with already established evidence (based on far more more data) that apparently very few intro CS programs succeed in teaching basic programming to all students who enroll (the double-hump phenomenon). But there are a lot of other factors at work here:

  1. Most universities don’t have the facilities to support getting everyone who enrolls in their CS program through the whole program.

  2. Most intro CS classes are over-crowded.

3)The previous programming experience of students in intro CS classes is extremely diverse, with some students already being semi-skilled programmers, and others having no exposure at all. The lowest common denominator is REALLY far away from the highest.

While one could argue that all this is irrevelant to the research, my point is that these conditions are prevelant (though not univeral) in colleges across the world, and that it affects the way that intro CS teaching gets done. This study rushes to the conclusion that these results point to something inherent about the students, without considering it may have something more to do with widespread conditions placed on the classrooms and teachers. Just because a lot of teachers in the trenches have experienced the double-hump phenomenon and couldn’t seem to beat it doesn’t prove it is inherent to some wiring in the students. If you live in the Arctic, you’ll quickly assert that all bears are white–that doesn’t make it true.

I’m just supposing here, but I bet those students who need some practice and attention (perhaps some help that would go towards learning a more consistent mental model) don’t get the attention and encouragement they need. They are thrown into the midst of cryptic syntax, alongside a bunch of folks who already seem to understand it naturally, and they get discouraged very quickly because they don’t pick it up as fast. The classroom is huge, the teacher has way too many students, and doesn’t have time to bring the stragglers up to speed. Especially when the stragglers may become quickly demotivated in an environment where the subject matter comes so easily to some students. Just my opinion, but I think this phenomenon comes much closer to explaining the double-hump.

Of course, I do not submit all this not as a counter-theory–I have no data–but it is an argument that this study is pretty anecdotal and not good science. Many extremely subjective statements are presented as fact and the author’s bias is made obvious in the introduction.

“We point out that programming teaching is useless for those who are bound to fail and pointless for those who are certain to succeed.”

That’s a pretty broad statement, don’t you think?

So here are some anecdotes of my own (and at least I present them as such):

I taught two semesters of programming in a continuing education program (voluntary, small classroom, no grades, no pressure) and everyone could do basic programs by the end of it. No drop-outs. Of course, the make-up of my student population was biased because no one was trying to meet “core requirements”–everyone wanted to be there. Nonetheless, one student had an extremely difficult time, and required a great deal of one-on-one attention. As another teacher described, after enough practice, one day, it finally just “clicked” for her.

What is the difference between this student and the thousands that drop out of intro CS courses every year? I’ll tell you I think it has nothing to do with some hard-wired gene for programming. That is just old-fashioned bigotry.

Personally I think there are two simple factors:

-The teacher's skill and experience with teaching, their patience and motivation, and their time-capacity for encouragement and one-on-one attention
-The students attitude and motivation to learn the material

Those two things added up, and I believe there is very little that cannot be learned.

Hello, before I lay down my views on the subject, a few things about me. I’m 25 yo, and I studyed programming and computer science for 12 years, starting from fifth grade. From 5th to 8th there were 2 hours of programming a week, on old Z80 / BASIC machines - very “light” (so to speeak) stuff - basic programming, basic control structures etc - and if memory serves, the course was suspended during 7th and 8th grade. From highschool(wich was CS) began the real heavy teaching, pascal, structured programming, intensive etc. And then there was the university - electrical engineering and CS. I am currently working as a programmer, employed by a software company - and I am very successfull in my work. Here are my opinions, based on my personal experience as a student of various institutions (I never taught):

  1. There is definitely a programming gene, a natural, inborn ability to program. And yes, those who lack this will never be able to program computers. I still remember the first programming course - I had seen and played on computers before, but had no ideea about programming. From that very first course I was like: “hey, I understand this perfectly, this makes perfect sense, this is so easy - this is NATURAL”. And I was perplexed that the others couldn’t understand programming. After only a couple of courses, I was one of the two or maybe three “freaks” in the class, those who could understand programming perfectly. Thoughout highschool and university, I have seen bad teachers, excellent teachers, good and bad teaching, but the double hump was allways there, and nothing could be done about it. There were those (few) who could, and those who couldn’t (well eventually, after years of hammering, they could write basic programs, and had a general grasp of programming, but not enough to become programming professionals - most of them don’t even work in IT). And another true thing - the teaching is too slow, tuned to the rythm of the “goats”. The “sheep” were devouring the books, learning several lessons in one session, were 1-2-3 semesters ahead of schedule, reading 10th grade manuals in 9th grade, languages, general algorythms etc. No other course was like this, there were no students lo learn math ahead of schedule, or physics, or whatever. Meanwile, the rest were struggling and going through all imaginable hell. I was perplexed - why can’t they learn, what is so difficult, what is their problem? In the end, many were relieved to have finished highscool, and never have to see programming ever again.

  2. About sexism - nobody can tell me there is no link between gender and programming ability. I once watched a documentary about the human mind, basic psychology and stuff, and one ideea was this: men have an inborn ability towards exact, mathematic, and engineering fields, and make “excellent nerds”, while women are far better at psychology, human behaviour and interpersonal skills. This is clear as night and day. CS highscools were almost all male with as few as 2 girls out of 30 in one class. CS and electrical profiles in the faculty were all male. Languages, psychology were almost all female. I recently went to job interviews to several software companyes. The programmers/technical were all males, the same “nerds” and “geeks” types I saw in highscool and faculty. Human resources were all female, all of them.
    As a sidenote, both my parents are programmers, my father is an expert, my mother barely strugles with some basic database software.

  3. I am not shure about the test. The results sound interesting, but I am not convinced. I tryed to imagine how I would have fared back then, before any programming course - and frankly, I don’t know. I do believe that programming is special, set aside from all the other objects, and needs specialised testing and evaluation - and specialised aproach of all aspects. As I said, I am not convinced that some test for some basic mind patterns could be the panacea. What I sugest, before admitting students, teach some basic programming, logic, algorythms - just a couple of courses - and then do lots of evaluations. As I said earlier, the sheep are identified very quikly.

  4. For a teaching, introductory language, I swear by Pascal. People say it’s a “comunist language”, but it forges a coding discipline and clarity that I personaly value greatly. I consider myself lucky to have studied pascal, before moving to c/c++. Hated JAVA…

These I my personal and highly subjective opinions. Please excuse my english…

Hi, I’m a professor of physics. And a woman. There’s definitely another factor at work here: confidence. If you were unconfident you would probably switch up the rules you were using to answer the questions just in the hopes of getting it right some of the time. That says nothing about your innate intelligence, just your confidence. I’ve given a lot of oral exams to students in a 101-level course and have found that the unconfident ones often know the material but it needs to be coaxed out of them because of the barrier that lack of confidence presents.

I’d also like to add a few other things that need to be said:

  1. This experiment does not do a thorough enough job of testing the possible parameters. These profs should collaborate with a wide range of other CS profs at different colleges who teach classes of varying size and use different teaching techniques. I know a woman CS prof at U of Maryland who can teach anyone to program.
  2. If the authors of the study really want to investigate this fully they should do several case studies where they do in-depth assessments with students from all over the double hump. Ask the students what their background is with computers, how comfortable they feel using them (ie, confidence level), etc.
  3. Dragos, watching a documentary does not make you an expert. Any and all “facts” on television carry a bias. Part of the lack of women in your CS classes has been the slow-to-change phenomenon of available mentors. As more women become the teachers of CS classes more women will take them and feel like they belong. Also, in many cultures (including US culture) the patriarchal way of life keeps women down and discourages them from achieving more than their male counterparts. These seemingly small effects add up to a society that lacks women programmers. It is a social commentary, not a genetic one.

-Kristine

1 Like

in response to dragos:

  1. There is definitely a programming gene
    re:
    The problem i see with this opinion is that it is based on experience, but it doesn’t take into account nurture. Perhaps in your first programing course, you had already been introduced to logical relationships in one way or another, and understood some basics about giving instructions and potential pitfalls; whereas the individuals who couldn’t grasp it, had never really been introduced to these concepts. It could be that we are trying to teach a baby to run before they know how to crawl… which would only result in lots of skinned knees and a few concussions. Since you already knew how to crawl, it was easier for you to stand up and run.

  2. gender differences
    re:
    Documentaries can be wrong, and a lot of them have been. I could argue, in the same manner, that since most girls play with dolls, it is the doll, not gender, that causes a lack of interest in programming.
    Could it be that because as a culture, we assume girls will like pink and dolls, and then go ahead and stick them in a pink baby room with doll, that they like dolls because of the situation, and not because of a predisposition? How many countless families perpetuate this view, and thus perpetuate the situation in our culture. The classic case of self-fulfilling prophecy. Because we assume girls have this predisposition to psychology, we (as a net sum) unknowingly force them in that position.
    In my highschool, when a conflict arose, boys were given gym, and girls were given home-ek… is it no wonder that most of the boys at my school assume girls liked home-ek and vice versa. It really had nothing to do with their predisposition, but instead having to do with the presumptions of faculty. Is it that far off basis that, as a whole, most schools do the same thing when it comes to psychology and comp-ed courses; thusly creating this mass assumption that boys are better at it than girls?
    Correlation does not equal causation.

programing is the act of instructing a computer to do things, and using the application of logic to make it do things that you want it to. Instructions is a form of communication, in essence, you are coaxing a computer to do something for you. And logic is an abstract form of looking at how we solve problems and arguments. It typically takes a lot of logic to preform well in math. And the instructions usually require an understanding of how a computer works (ie, writing to the appropriate audience).

I bet your code isn’t very object-oriented.

I’ve seen a comment here that complained about programming teaching being biased towards the imperative paradigm, and making it the supposed reason for the situation where so many people just can’t get it.

I’ll admit that I’m a novice at functional programming, and I know almost nothing about logic programming, but anyway…I still can’t understand something: Is it really easiest to understand everything in computing in terms of functional programming?

Say, how can you explain a data structure such as a hash table, or say, explain how a garbage collector operates at runtime, with the functional programming mindset?

Another thing, why is it that pseudo-code, the preferred formal method for describing algorithms [at least in academic papers that is], is distinctively “imperative”? I’m just talking out of my own experience here … Maybe that has changed in recent times? Almost all of the programmers I know are more comfortable with imperative programming, though there is no argument functional programming is vastly more expressive/powerful in many cases. The best programmers can think in whatever paradigm suits the problem best, but I can’t think of a single programmer who can easily understand functional programming, and still struggles with imperative programming.

I remember my ( Stage 2 ) lecturer talking about some code a student was having trouble with :

int i = 0;
while(i100)
anArray[i] = 0;

Now, this was in a second year course, with 2 pre-req papers, both of which cover a C-like language.

Personally, i think that these crap^W bad programmers exist because of the flawed teaching system. The stage 1 courses cover OOP, indentation, drawing flowcharts, comments, GUI buttons, etc but miss one important thing : teaching students about what code is. E.g., what happens when you run “a = b;”. Students who don’t instinctively get this fail because they never learn it properly and instead spend hours messing around in a circle. Also, at a higher level i see students who abuse recursion, or write f-ing massive switch statements when a simple bit of bit-or/and-ing, etc would have made it 2 lines.

Once i saw the source to this chess game a student was writing, he had used a separate array for each piece for each side. So he had a blackPawns array, a whitePawns array, and so on. Yep. So you can imagine what a mess the code to draw the board with these 12 arrays was. He hadn’t done the code for the simple A.I. the computer side was going to have, because (surprise) it was going to be very hard. So i recommended the he use a 8x8 array of Pieces for the board, and have the Pawn class extend the Piece Class, and have white/black flag(s), empty spots are null, etc. Right ? (Hopefully) This is a pretty simple and standard way of writing it. His first response was “How do you check if a spot is empty?”, i was about to say that you check if the element is == to null. His way of doing it ? “Catch the NullPointer exception.” He didn’t take my advice to use a 2D array for the board, and as far as i know is still trying to write the huge amount of code to handle the movement checks or A.I. To give you an idea of the extra work it would need :
His code :
bool isEmpty(int x, int y){
for(int i = 0; i …;i++)
if(writePawns[i].getX() == x whitePawns[i].getY() == y) return false;
[ copy + paste * 11 ]
return true;
}
My Code ?
return board[x][j] == null;

What a weird choice of “sheep” and “goats” as an analogy, and what a weird way to assign which were the sheep and which were the goats. Anybody that knows anything about sheep and goats knows that sheep are really really not very smart at all, and being called a sheep is a bit of an insult, whereas goats are much more clever and resourceful, much harder to keep confined, impossible to herd, devour anything and everything.

Given that managing programmers has commonly been described as “trying to herd cats”, mapping good programmers to sheep seems particularly inept.

as far as i can see the test has nothing to do with imperative programming. one could devise a mental model which concurrently processed the 3 statements and still pass the test IFF you use the same model on the other questions.

Let the compiler/assembler/interpreter sort it out. Everything else is subjective.

“…not a perfect divider of programming sheep from non-programming goats…”

I don’t like that for some reason. I think we’re programming goats, and they’re non-programming sheep.

Actually goats are the more intelligent animal. Sheep are mindless. While from the perspective of archaic theology that values conformity over independence or intelligence, sheep may be preferred, the metaphor should be reversed for dividing programmers from wannabes.