You ask an excellent question. That means it's a hard to answer one! The example happens to be an especially difficult little program to write because its operation is so entwined with the way Linux approaches process management. On Windows, for example, one uses the CreateProcess function, which avoids a lot of the oddness of the steps you refer to. However, I recognize you intended your example to just be an example, so I'll try to use it to point out how other software might be written.
I'm a big fan of serendipity. Last night I was watching a bunch of Russian physicists discussing consciousness with the Dalai Lama. In it, one of the physicists brought up a model Einstein used to depict his notion of modern physics. In a letter to Maurice Solovine, Einstein depicted how modern physics works like this:
As he described it, the process for modern physics operated as a loop. On the bottom, we have the experiences we have in life. The arrow sweeping off to the left is intuition. From these experiences in life we distill highly abstract concepts of what the "laws of physics" might be, which are the axioms of science. From those, we deduce very specific testable statements which then encourage us to go out and actually run the tests and make the results part of our experience. The cycle then continues.
I think this is very similar to how a programmer builds up a program. They start from the sum of their experience as a developer, and use that to fuel an intuitive leap to a set of axioms that one thinks can solve the problem at hand. One then builds more concrete statements which suggest ways you can test these axioms to see if they're right. When we actually test it, we experience what really happens, and that fuels the next intuitive leap.
Of course, the challenge as a young developer such as yourself is that you haven't developed that intuition to make the first leap. This is not unusual. Most students get through college without developing this intuition because it's not what school teaches. Instead, you develop that intuition very rapidly once you get out in the field. Don't worry, you're not alone!
To jump start the process, we in the field of computer programming rely on something called "requirements." These are axioms which someone else gives you and says "your project must fit to these axioms." If you're lucky, these requirements were developed by someone who knows something about the problem at hand, so they're pretty close to the correct ones. You then simply have to develop the deduced statements and test them out, developing your own experience.
If you have a very good team lead, they'll give you requirements that are just right enough to keep you productive, but have just enough flaws that you can start developing your own intuition. Often there is something wrong with the requirements, and when you realize this, you are often the only person with the experience required to make the intuitive leap to the new axioms. To give a concrete example, if your team lead has only ever worked on Windows, they may have the Windows axioms in their head, which assume there is a CreateProcess function. If your product needs to work on Linux, their requirements may end up being wrong.
So what happens at this point? Well, you have requirements to find the CreateProcess function on Linux. You deduce that such a function should appear in the Linux API documentation, so you go test that theory. You look through the documentation, and don't find it. Now you have an experience that your team lead didn't have -- your team lead hasn't gone through the documentation like you just did!
Now you know you can reject one of the axioms as "not correct." So what was it's purpose? This is where your intuition comes in. You read the documentation on CreateProcess on Windows, you study what your product (the shell) is supposed to do, and make the intuitive leap that what your team lead really wanted was process management. Now you have an axiom that you've invented. Nobody told you to do "process management." You were told to use CreateProcess. You intuitively theorized the reasons for why they wanted you to do that.
Next comes the development of deduced statements. You have an axiom stating that your job as a developer is to figure out how to do process management. You need to develop a few specific statements which can test this axiom. This might mean making a few statements which you may ask your team lead about. You might tell your team lead that there is no CreateProcess function on Linux, and ask for clarification to see if what he's really looking for is general-purpose process management. Or you might deduce "this sounds like something people have done before, I bet there's an answer on StackExchange." (If it's not on stack exchange, it's probably not worth doing!). Or you might have to take the long route. You might have to say "I understand that Linux has functions to create processes. Which functions are capable of creating new processes?" This would then lead you to go look through the documentation again, and find fork.
At this point, you can fork your process, but obviously that isn't exactly what you want. You want to execute a new program. Another intuitive leap: "given that people execute programs on Linux, there's probably a way to do it." Combining "fork is the way we spawn new processes" with "there's a way to spawn other programs on Linux," you can deduce the idea that there's probably a function which turns one program into another. This will lead you to find exec.
Now all of this is dependent on a balance. Your team lead has to give you enough requirements that your fresh-out-of-school intuition is capable of filling in the gaps, but they also need to be able to listen to you when you come up with new axioms which might change the requirements. Team leads generally have experience. They have the skill to do this (most of them, at lest).
And what if you're on your own? What if you're just sitting at home honing your skills without a team lead to direct you? Well then, you are your own team lead! I recommend starting small. As a general principle, we learn the most when we pick a program that is within our reach, but just outside of our grasp. I wouldn't pick a shell as my first test program, because they're rather hard. However, if you just finished a course on how processes are managed by the operating system, that task may be right up your alley!
And remember to always have fun! It's so much harder to develop intuition when you're not having fun!
