Answering "Why" people do the things they do is always a risky proposition. Be that as it may, I'll try my hand at answering your two-questions-in-one question. I believe that the answers are interdependent, with each feeding the other. This makes it hard to split this question into a pair of questions for the site, which I would normally recommend.
Why are pointers a key topic in a CS curriculum?
There a five reasons that come to mind for making pointers a key topic:
- They are an integral part of many of the current in-use languages, and should be understood by the students if they're going into the field as developers
- Pointers (whether explicitly in the foreground, or implicitly in the background) enable the Object Oriented paradigm
- They are perceived as difficult, and the belief is that repeated exposure, and emphasis, can pound the concept into the students' brains
- Understanding pointers aids in further abstract thinking
- Everyone else is doing it.
Integral to many of the currently in use programming languages
Many of the programming languages commonly used in software development enable pointers. They might have a different name (reference perhaps?), or their actual use can be cloaked behind other layers of abstraction. When available for use, many developers, or their supervisors, think that they must be used. Oft times their use over the basic types serves no purpose to the code or the operation of the software at all, any benefits being in the mind of the developer using them. Sometime, "because I can," or "I like them," is sufficient justification for their inclusion. Since very few of the students will transition straight from school to lead developer on a project, they will likely be working with the code of others that has a significant amount of pointer used. Working with such code will require that they have a working comprehension of pointers, their operation, and their merits, within the language of choice.
Pointer use in the Object-Oriented paradigm
I have not examined the "behind-the-scene" implementation of the OOP paradigm in great detail for any language, nor even know all the languages in which it is implemented. Nevertheless, without the ability to use pointers, by the compiler if not the code itself, I don't think OOP would be practical. It may not even be fully realizable at all without pointers. Whether or not pointers are used explicitly by the student, or the programmer, they are probably using pointers anyway. Understanding what a pointer is, and how it works, can make some of the OOP concepts easier to comprehend, and increase the facility with which the students utilize OOP in their own code projects.
Repeat the hard stuff until they "get it"
If the instructor, (or department head, textbook author, curriculum developer, etc.) believes pointers are "something people don't get" they try to make them "get it" by repeating the use of pointers in different places, across multiple courses, or from different approached. They hope is that with enough repetition, even difficult concepts will become comprehensible. If, for whatever reason, the concept presented is hard for the student, such a layered, multi-pronged approach may be helpful. If, however, the student does grasp the concept and is able to use it effectively, the repeated implications that they still don't "get it" can make them begin to question if they really do understand, or if the instructors might be correct, and they only thought they "had it" when in fact they don't. Teaching to the lowest common denominator can be just as destructive as teaching to the test.
Pointers: one layer, among many, of abstraction
Pointers are a concrete manifestation of indirection. Technically speaking variables are an indirection, but somehow we manage to address that just fine. With pointers we now have two, or more, "leaps" to get to the "data" and not only do we have to know that we have to double hop to the get data, but we also have to remember to handle the hops the right way. For example, if I do
a = b where
b are numbers, I can change the value of
a will not change to match. If I do
a = b where
b are pointers, I can change the value of what
b points to (
*b = 4) and the value of what a points to (
*a) will be changed as well. The syntax, or nomenclature, helps the developer keep things straight, but the develop still has to know what's happening with the pointers.
The good part is that once the student has a grasp of pointer operations, they are able to move into higher levels of abstraction with confidence. As noted in a comment by Scott Rowe above, in the computer science realms, pointers are the crossing point into formal operations from concrete operations. Crossing that threshold of understanding in any discipline develops the mental abilities to deal with formal operations, and symbolic thinking in any discipline, from computer programming to medical research and structural engineering.
Everyone else is doing it
Probably one of the most common reasons, and the worst reason because it is not a reason, is to follow the crowd, or to gain acceptance of their work. I have seen educational materials where the author attempts to present pointers, in some fashion, that end up looking like a blind person leading a deaf person into the swamps. They don't have a real grasp on pointers themselves, but try presenting them because they think their works will be judged negatively based on their mere exclusion. They could be right, I don't know. They don't seem to have a good understanding of pointers, indicating they don't consider pointers important enough to learn for themselves. Yet, because everyone else is doing it, they include an attempt at explaining pointers in their materials.
Why are pointers oft cited as something people don't get?
The claim that pointers are something people don't get, or that they are hard to understand, could be founded on any of five different reasons (or beliefs):
- Instructors have tried to teach what they don't understand
- It was hard for the instructor to grasp, so it must be hard for everyone to grasp
- Teachers have been warned that some people don't get pointers, so they know some of the students just won't "get it"
- Students, or programmers, not ready for abstraction will not "get" pointers either
- Everyone else is saying it's true
Passing on knowledge not possessed
If the instructor does not have a very clear understanding of pointers, it is very difficult to help the students understand them. Saying that pointers are difficult, and that some people just don't "get it" serves as an opportunity to blame the subject, and the student, for the instructor's failure.
It was hard for me, so it must be hard for you
A natural human tendency is to take our experiences and project them onto others in the form of expectations. When something was difficult for us to master, for what ever reason. we will believe that it is difficult, and will expect others to have the same difficulties. Also, how one learned about pointers the first time can have a strong influence on how they later teach them. For a general question in that vein see this old question: How is your teaching affected by how you learned?.
My (Education or CS) instructors said it was difficult, so it must be
Presuming that the instructor has been instructed in the art of instruction, the instructor's instructor could have embedded a self-fulfilling prophecy. (See Merton, Robert K. (1948), "The Self Fulfilling Prophecy", Antioch Review, 8 (2 (Summer)): 195.) If told that it is hard to teach pointers, or that students have a hard time understanding pointers, and then hearing it, or reading it, from other sources, it can become "fact" in the instructor's mind, and then acting on that "fact" in the classroom they make it hard for the students to understand pointers.
Pointers: one layer, among many, of abstraction
There are two schools of thought about abstractions. One is that the use of abstractions eliminates the need to have any concept of the processes hidden by the abstraction. The other is that abstractions are only a convenience for encapsulating processes already known, and solved, so that higher levels can be solved. In either case, if someone is not yet able to grasp the indirection and abstraction, the subject matter, already maligned, gets the credit for the student's lack of development or ability.
Everyone else is saying it's true
If it is said often enough, from many diverse corners of the field, it soon becomes common knowledge. The longer it survives as common knowledge, the more likely it is to become conventional wisdom. Unfortunately, conventional wisdom may not be all that "wise," and common knowledge may not be correct. For example, triskadekaphobia is so prevalent that many, newly constructed large buildings still do not have a 13th floor, or a usable one at least. This despite the fact that our era is supposedly enlightened and has moved beyond myths.
Are pointers difficult to use in practice? Or, are pointers hard to learn?
Pointers, in whatever guise they assume, are not difficult to use in practice, if the language supports them in a native manner. They are also not very difficult in other languages, though perhaps slightly more so than a regular variable. I have used "pointer" functionality using
POKE in BASIC in the late 70's, before I even knew what pointers were, and had no difficulties. They do not require any more diligence from the programmer than should be exercised with the rest of the code. They can, however, turn out to be a nuisance if not used responsibly. In the languages where garbage collection is not automatic it is the programmer's responsibility to track what they've done and undo it later. That shouldn't be any worse than making certain all loops have an exit condition that will happen, or that all conditions for a logic construct have an assigned branch.
Pointers should also not be hard to learn. Pointers are everywhere in our lives, and we have no problem understanding the ones outside of a program. Why, then, should they suddenly become mysterious inside a program. My phone number is a pointer, or an indirect reference, to my house: someone who knows my (1) name will look up my name in a directory listing, which lists my (2) phone number; they dial that number and someone answers the phone; they then ask the (3) person answering the phone to speak with (4) me. A very long, and complicated series of pointers that people have beed executing since before computers were built. The only excuse for pointers being hard to learn is the instructor, hopefully innocently, making them difficult. Once the concept of "variables" is apprehended, pointers, arrays, hash tables, and "complex" data structures should naturally fit in the same understanding. The more complex the structure is the more effort that is required to remember "where you put" the data. That is not because there are pointers, rather it is a factor of the quantity of elements involved. When following a route through a large city, the more turns you have to make, the greater are the chances that one of the turns made will be wrong. It has nothing to do with turning being difficult, only that a line in the instructions was skipped, or a street name misread.