What things aren't obvious about reading documentation? How can I explain to others how to do it?

Question

As an experienced programmer and frequent user of Stack Overflow, I often run into this problem. I try to tell people calmly to read the documentation for whatever it is they are trying to use - but this often doesn't resolve the issue even when it seems like it ought to. Sometimes a direct link is needed (with an anchor to the appropriate part of the page); perhaps a relevant quotation, or even my own rephrasing or explanation.

Because I'm accustomed to doing my own research, and to the actual process of reading and understanding documentation, I'm often a little mystified here. I get the suspicion that common "documentation-ese" writing conventions are just not intuitive for a lot of people, whereas I've become so accustomed to typical documentation writing style that I don't notice any issues.

Am I correct in my suspicion, for example, that new students commonly get confused because they don't know how to parse the usual descriptions of a function signature (e.g. with optional parameters in square brackets, the special meaning of ..., the way that Python documentation used / to separate positional-only arguments for C-level functions even before native Python support was added for them...)? If so, are there resources that explain this sort of thing? I've evaluated many "learn to program with X as your first language" books over the years, and I can't recall any that get into that material.

What other things cause confusion? Is there something about, for example, the layout of a typical documentation page that inhibits students from drawing connections? Recently I got feedback complaining that some documentation didn't appear to "indicate what classes the methods come from" (which seems rather dubious, and which inspired me to ask the question).

Knowing what needs to be mentioned, is any special approach needed to explain the task of reading documentation? Or should I think of it like any other subject? I have it in my long-term plans to write one of those learn-to-program books myself, and I'd like to break the mold by including something useful on the topic.

Answer 1

I have a partial answer, perhaps, as to why it is so hard for novices to learn from documentation. But the answer isn't universal and doesn't apply to all products/languages.

There are some companies (I remember Sun Microsystems) that employ people to write documentation who are expert writers rather than expert geeks. They need to be able to talk to geeks and to write in such a way that the uber geeks agree is accurate, but their focus was on reader understanding. A friend of mine once did this for Sun.

But there are other places in which the geeks themselves write the docs. There might be a lot of reasons for this and it might require the backing of a large company to do otherwise, but such documentation can fail users.

I recently reviewed the documentation and tutorials for a product in development. The docs were written by the same folks that did the product. The problem was that in order to understand the documentation you also needed to already deeply understand the problem the product was trying to solve (not a novice attribute) as well as the basic overall UI that the product used. This was probably comprehensible to someone already deeply immersed in the problem space as well as having a lot of experience with similar UI structures.

One of the reasons, perhaps, that you can get away with it is that you already have the background of the entire platform structure that novices don't have. So it is all "Greek" to them. ("Geek" to them??).

A solution, though expensive, is to have different documentation for different experience levels. This is why textbooks exist, actually, and why clear explanation is so valued there. Professional level books are pretty much closed to novices before they go through the apprenticeship needed to grok it properly. And, I remember that after going through the three semester Calculus sequence in college, I then went through several semesters of Advanced Calculus and Analysis, where it became clearer.

Few shortcuts to real learning, I'm afraid.

Answer 2

I think that both of the posters so far (Fureesh and Buffy) have great insights. I would only add one more piece I've observed from my own students. It starts with the observation that most people don't particularly enjoy reading dry text, and will give up easily if it becomes impenetrable.

I can't find it now, but I recall some literacy studies that suggest that kids start to give up on text if more than one in ten sentences are too hard. Since I can't find it, take the number ten with a giant grain of salt. What's important here is that there was a number, and that it was possible to characterize motivation as a function of the percentage of non-understood items within the overall text. What's also important is that adults aren't truly so different from kids.

So, take a look at Oracle's documentation for Java's String class. It is, by my estimate, pretty excellent documentation. It has nice text at the top that outlines the basics of the structure, a list of method calls, and then a write-up of each call. It's very helpful, and I've referred to it many times over the course of my career.

But now view it from the perspective of a real beginner who is just learning about Strings (think about how beginner that is), and the entire document becomes very different. We start with a bunch of totally impenetrable header stuff, like implemented interfaces.

When we get to a text introduction, we start with:

"The String class represents character strings." All string literals in Java programs, such as "abc", are implemented as instances of this class.

So far ,that might be okay. For the beginner, character isn't a fully fleshed-out concept, but... we might be able to deal with this. And the next sentence is possibly excellent:

All string literals in Java programs, such as "abc", are implemented as instances of this class.

Though, again, there are some large concepts that you need to understand before it makes sense. But it reads a bit like "abc" is an example of String, and that's a comprehensible idea.

Then, immediately:

Strings are constant; their values cannot be changed after they are created.

Wait, what?

String buffers support mutable strings.

We're very lost now. Is a string buffer a string? Is that a reference to something else? What is mutable, exactly?

Because String objects are immutable they can be shared. For example:

    String str = "abc";

is equivalent to:

    char data[] = {'a', 'b', 'c'};
    String str = new String(data);

Well, that's confusing, not least because the example doesn't appear to match with the sentence leading up to it. But, for the beginner, we are now looking at a series of glyphs. Just the line char data[] = {'a', 'b', 'c'}; has many, many concepts locked into it. There is precious little chance of someone just learning about strings making heads or tails of that code.

It honestly doesn't get much better as you head into the rest of the document. For the beginner, the documentation is not at all helpful without a teacher to walk through it with them.

I assume that most people had a similar experience to me with man pages, and couldn't make sense of them until someone broke it down first. Maybe other people didn't have those sorts of troubles, but I'm pretty sure that the norm is for people to find them very confusing at first.

Anyway, consider, then, what is likely the early experience that learners will have with documentation. It wasn't useful! The early lesson, it seems, is that documentation is dense, full of needless foreign concepts, and not that helpful.

The students will nevertheless go off and find other information that is more helpful. After all, they still need to do whatever it is that they need to do. Where do they go? A friend, a YouTube tutorial, an article, whatever.

It is perfectly natural, then, that as they develop, they continue to (a) go to the kinds of resources that they have found helpful in the past, and (b) therefore don't develop their skills in documentation comprehension.

I think, then, that something like what you propose, with explicit guidance about how to deal with documentation for people at a more professional stage, could be very useful indeed, since at that point, they are more ready to deal with what they'll find there.

Answer 3

In some cases, I suspect the problem may be that the student doesn't really want to understand, they want to do (e.g. complete the lab exercise or coursework). Unfortunately these days there is an awful lot of code examples easily available on-line in tutorials and e.g. stack exchange, and it is often easier to search for existing code that is similar to your problem and cut-and-paste to get your program working. Understanding the programming language and the documentation is often hard work as a learner and there isn't an immediate benefit from making that effort (but very large benefits much later on). I suspect that I wasn't very good with deferred rewards when I was a student, but I had the advantage of learning to program before the WWW existed, so I didn't have the option. When I am teaching I try to emphasise the benefits of understanding what you are doing, rather than just trying to get it to work, but to some extent I think this is a life-skill that can only really be learned by experience.

To answer the question more directly, the thing that may not be obvious to the student is why being able read the documentation is an important skill worth learning, despite not being the fastest solution to the problem immediately facing them.

I dread to think what effect Codex and CoPilot will have on learning to program. I'm not against the use of tools in programming, far from it, but tools for experts can get in the way of learning the fundamentals. I'd be glad to hear of any work that has been done on this!

Answer 4

I don't believe there is anything special to reading documentation. As most things programming related, it's a bit of basic knowledge and a lot of experience. You obviously need a basic understanding of the language (or whatever the docu is about) and the "grammar" which is used for syntax, optional command line or function arguments and the like. But at my Uni this (along with more extensive formal grammar) was covered in the first semester.

To put it another way: If I completed my 5 year CS degree without ever looking at documentation I wouldn't suddenly go "I'm not prepared for this" when handed my first manual.

If I had to point out something it would probably be

a) Documentation (apart from the most common languages) in my experiance is rated on a scale ranging from "patchy" to "non-existant".
The people who wrote it are usually experts on the subject and rather spend their time being experts than write documentation for other people. So a lot of things get left out because either they are obvious to them (and not you) or they just don't have the time or feel like it. So reading documentation sometimes can be a bit of a guessing game. With more experience it get's easier to fill in the gaps.

b) Even if patchy, documentation sometimes can be massive and feel intimidating. E.g. the reference manual for one of the smaller microcontrollers I'm using, has about 1600 pages. And that's only hardware. The programming manual is another 2000 pages. So you have to know exactly what you problem is and where to look.

So if people are "lost" when reading documentation, I don't think it's because the are missing the vital "magic documentation reading skill" but it's rather a symptom of deeper issues with the problem or programming in general.

Answer 5

Documentations are written in a very specific format that follows certain rules. When were those rules explained to a person who starts learning programming?

People look for patterns in order to manage in situations not entirely familiar to them. What patterns do documentations follow that new programmers are familiar with?

I don't think there are many.

The closest you can get is reading an encyclopedia or a dictionary, but not many people do that. What happens then if a person tries to find a definiton for a term they are unfamiliar with? They most probably simply google it. But what are they looking for and what is the best search result for them? One that uses that term in a familiar constructs. One that shows examples. One that consists of multiple explanations trying to present different points of view, so the person reading it can graps the topic as best and as easily as possible.

Where do we find those elements in documentations?

Now, think about what makes a given documentation great. I am sure you have seen better and worse documentations. Try to think what disinguished those bad documentations from the good documentations.

Sometimes it's about examples. Sometimes it's about very specific terminology used. Many documentations fail to acknowledge the fact that they need to be both accurate enough for an expert and written in a language that is simple enough that an inexperienced user should be able to exctract information from it. This is extremely hard to do.

So, to summarize, I believe that the initial problems with extracting information from a given documentation is a combination of unfamiliarity and complexity of the tool. Documentation seems straightforward, but in reality it's really hard to create one that is both accurate and easy to understand.

That's why I believe teaching how to read documentations and getting familiar with them should be a thing. It will just save so much time for everyone.

Answer 6

Documentation is to (e.g.) a stack overflow answer as a dictionary is to a textbook of english. (Or, put another way - by the logic of this question, english language SE would/should have very few questions.) Documentation has certain rules, yes, and these are mainly necessary because it is so compressed from a standard explanation. Instead of saying "The function takes in x, which is an integer which tells the function what power to raise the number to, as well as y, which is an optional boolean (i.e. true/false value) that ..." this all boils down to five or six words of documentation.

Worse, this compression is accentuated by the use of jargon that a newer user likely isn't familiar with (e.g., 'bool' short for 'boolean' short for 'a value that can be true or false' - and this is all very abstract! if you haven't ever seen a use for a boolean value in programming, you'd sure have some questions at this). Also, much like a dictionary, the examples are brief, often unexplained, and at the bottom of the page.

And further, using documentation requires that you know what to search in the documentation in the first place! If you don't know what you don't know, documentation is a useless tool (see also the oft-mentioned problem on stack overflow of over-constraining what the answer must look like, as in, 'how do I sort this list in [xyz] way' instead of 'how can I best store this data'). For instance - I took Spanish in high school, and while I did own a Spanish-English dictionary, my use of it was sketchy at best because I did not know enough Spanish to tell whether I was conjugating the translated words properly, appropriately accounting for connotation, and so forth. (Spoiler: I almost certainly was not.)

Together, all of this combines to create a tool that is useful if you already know what you're doing and need a reference, but almost actively detrimental if you do not know what you're doing.

So, in terms of approaching how to use documentation with students, I think the answer is a combination of two things. First, don't advise defaulting to using documentation to solve your problems. I do think it's a bit amusing you brought up the Stack Overflow 'check the documentation!' thing, because while I do agree a decent percentage of the time that a google search would have helped everyone involved, there's also a decent percentage of the time that the OP was looking for the function that the documentation page linked references - that is, saying 'well just look at the documentation page for [function]!' is equivalent to saying 'well just check the oscilloscope manual!' when someone asks what tool they should be using.

To again overuse the dictionary-documentation analogy, when writing, I rarely check the dictionary if I'm looking for a word - I'll check thesaurus.com to find a word that's closer to the meaning and connotation I'm looking for, maybe check articles about similar topics to see what their word choice is like, and so forth. I rarely crack open a dictionary, because searching for a word that 'fits' would mean going from a to z, and I fall down enough rabbit holes as is. The first place I check when trying to solve a programming problem is ~80% of the time 'stack overflow how to [task] python.' If I have a question about a specific function, then I check the documentation.

In terms of understanding documentation, I'd say your best strategy is likely to be looking at documentation pages for functions that the students are familiar with, in order to explore documentation layout and have them find the pieces they know should be there. Alternatively, for a less-familiar function, cross referencing the documentation with Stack Overflow q&a for more contextualized examples, and also attempting running the function and messing around with the inputs is probably a good strategy. But also, a lot of the functions these students would likely be using might be better explained on YouTube or Stack Overflow! By the time they need documentation, there's a not insignificant chance they'll have gained the knowledge to understand it via programmer osmosis (a dangerous thing to assume, I'm aware).

Answer 7

As I've heard it put: "Unix documentation is written in such a way that, when you finally figure it out for yourself, you can never say that it didn't say so in the first pace".

This points to the fact that reading documentation is a different process from reading ordinary prose. For documentation you have to engage with the prose:

1. you read it

2. you think "that can never be true"

3. you try to think of a counter example

4. you read the documentation again to see why your example is wrong,

5. 6 7 8 rinse and repeat.

This "struggle against the documentation" is frustrating and tiresome. Many people give up pretty early in the game.