How does the ability to solve a new problem come to a person?

Question

How do you learn problem solving. Programming is all about problem solving. Whenever I try any new problems, I can't solve them. How does this ability to tackle a new problem by yourself come? How do you practice it? How to be creative problem solver? I am wording my question in different words, but they all mean the same thing.

After how much time do you give up on a problem and look at its solution when learning problem solving as a beginner? What is the guide to "Learning how to learn problem solving"?

Most of the guidance online seem to be about roadmaps like "Learn Data structures and algorithms first, then learn object oriented programming, practice hackerrank problems" etc. I want something different. How do you keep belief that if you keep practising one day you'll be able to solve any new problems?

Answer 1

Basic problem solving can be learned in a straightforward way. Advanced skills come with experience, and of course, there are a few people gifted with a good intuition.

Most of the time, problem solving boils down to:

Find a solution that fulfills some condition.

E.g. to find prime numbers, that means:

Find integers that are divisible only by itself or by 1.

This is the analysis phase. It "translates" from the terms asked for ("prime number") to some more detailed technical description, and it can be a challenge on its own. But it's crucial.

Before trying to solve a problem, make sure you have a thorough understanding, in our case understanding what it means to be a prime number. To practice that, explain the term to someone else. Your fellow will hopefully ask questions if your explanation isn't clear. And you will quite often find that, having to explain something to someone else, makes you look at the topic in a new way, so you yourself detect gaps in your understanding.

When the problem is understood, first aim for a simple-and-stupid approach:

• What type of "things" can become a solution? Integers. So, enumerate all integers, e.g. in a loop.
• What is the condition that they have to fulfill? Divisibility by nothing else than by itself or by 1. So, write a function isPrime(number) that checks for all relevant divisors between 2 and number-1, that there never is a zero division remainder.

In at least 90 percent of all real-world programming, this simple-and-stupid approach is the way to go. The main exception is job interviews done by misguided recruiters, and of course coding challenges like hackerrank.

Of course, there are as well real-world situations where the simple approach is not good enough, e.g. in terms of efficiency. Then start to think about optimizations or different ways to do it:

• Keeping the simple approach, are there cases that can be excluded early from the testing? E.g. we know that (apart from 2) no even number is a prime, so our integer-enumeration can omit them.
• Also keeping the simple approach, you might observe that trying to divide e.g. 51 by numbers above 7 is useless: If it had a divisor above 7, then it also needed another one below 7, which the testing had already found earlier. With 51, there is a divisor 17, but that one needs another divisor being 3, and this one has already been checked before encountering the 17. So, the divisor check only needs to loop up to the square root of the number to be tested.
• Instead of checking every single number for being prime, we can turn the question upside down: From a list of integers, eliminate all the non-prime ones. What is left over, must be the primes. This is done in the famous Sieve of Eratosthenes. And you see, finding such an approach can promote you to the "hall of fame" of computer science, depending on the creativity of the algorithm. So, don't be too disappointed if you don't find such an algorithm on your own.

When writing production-quality code, the most important quality is readability, making it easy for your fellows to understand what you are doing. Here, the simple approach shines, as the code you write very closely follows the problem description. Typically, the more you optimize, the less the result still matches the original problem structure, often making it hard to understand what's going on under the hood.

Answer 2

The problem with being a beginner (in any subject) is that you lack the vocabulary or experience to know how a problem can be decomposed into smaller parts. Or your understanding of the problem comes with faulty assumptions because you don't know what you don't know.

There are several aspects of problem solving. Some of these you can work on directly, but some portion of "programmer problem solving" only comes from experience and knowing what tools are available.

---

Knowledge areas of problem solving:

• Scope - Understanding what qualifies as "solving" the problem. Restrictions on solutions.
• Composition - Knowing how to break down the problem.
• Experience - Knowing what tools, "tricks," and algorithms are available and/or relevant.

Scope

If you don't know where you're going, how will you know when you have arrived? For any problem, it's a good idea to formulate (i.e., write down) what will qualify as a solution. Does your solution need to work for all positive integers? Does it need to work for strings shorter than a certain length? Does it matter how long it takes to find a solution? Does it matter if there is "invalid" input to your algorithm?

There are plenty other kinds of questions you can ask about what your solution should look like. In general, you should look at the problem and think if there are any ways you can be "lazy". If the problem statement only asks about positive integers, maybe you can avoid a complication with negative numbers by simply not allowing those in your algorithm. You should make a list (at least mentally) of what options you have for "widening" or "narrowing" your solution, as long as it still meets the acceptance criteria.

Composition: Decomposition

For any problem (in general) that you don't have an immediate answer for, it's a good idea to write down a list of steps that need to happen. This doesn't need to be in code, just an outline of how to work through the problem. Sometimes it's helpful to start at "the solution" and work backwards to "the beginning." Sometimes you know where you start, but are missing some steps along the way. Just list as much as you can and leave some question marks "??" in the spots you aren't sure about. I'll have some more to say in the "Experience," section but pencil in your best guess about what solution would look like in this step.

At this step, you need to continue to break down each individual step in your solution until it's described in a way that you can implement. For example, what does "iterate the primes less than one million" mean? I can immediately picture a sieve solution and know how I would write that. If you don't have a similar mental picture, then this needs to be decomposed further.

• iterate the primes less than one million
  • build a list of integers less than one million
  • iterate the list and cross out every second number
  • find the first number after 2 (which is 3), and iterate the list, crossing out every 3rd item.
  • find the first number after 3 (which is 5), and iterate the list, crossing out every 5th item.
  • find the first number after 5 (which is 7), and iterate the list, crossing out every 7th item.
  • ...

If you think very long about the above example, you will notice it is not very efficient. It's often best not to worry too much about efficiency until after you have a working solution. But do keep in mind the scope of the problem, sometimes this is an important consideration.

Composition: Resolution

One of the marvels of the human brains is the ability to recognize patterns. If you can find a "template" or a "recipe" to apply in more than one situation, then you can be a programmer (more in the "Experience" section).

As far as programming puzzles, it's often helpful to solve a single case. It's a good idea to start with the simplest possible scenario: zero, NULL, "", etc. What happens using your solution with that input? What happens if you use the "next" input? It really depends on the problem for how you work this step. If your problem is about positive integers, then putting inputs and outputs into a spreadsheet might be helpful. You can try plotting different sets of input and output data.

If your proposed solution can be succinctly described in the same way you would an algorithm, then maybe you can use induction to prove your solution is valid for all (within range) inputs. x => x + 1, etc.

More formally, you want to propose a hypothesis, test it, and then prove it. Of course "prove it" can mean different things, but as a programmer, "proof" often means meeting acceptance criteria.

Experience

Can you tell what comes next: 517, 518, 519, 520, 521, ...? Like I mentioned above, pattern recognition is essential to being a programmer. You can take an educated guess that 522 should come next in the sequence because you are familiar with "numbers" and use them extensively (maybe more than you would like). But more abstractly, what happened:

• You saw a sequence of numbers
• problem: continue the sequence
• You drew on your experience with "numbers" that they are nicely "lined up" in a sequence
• You adjusted your general knowledge of numbers and sequence and realized it fits this pattern: [x], one after [x], one after that, and one after that, and so on
• You hypothesize, therefore, the solution is 522, 523, 524, ..

(Probably -- my hope, for this example -- you don't stare at sequences of 517, 518, 519, 520, 521, 522, 523, 524, ... all day, otherwise you would just recite a fact rather than step through the above problem solving steps)

That was a rather contrived example, but the purpose was to draw attention to the above steps. This only works because you have some "experience" to draw on. You have, in your "experience belt", a relevant tool to use: knowledge that integers continue in sequence. Programming challenges are no different.

This is the reason to understand data structures and algorithms. Both of these are essential tools that you need to be familiar with in order to solve programming problems. Some data structures are essential to solve certain types of problems, especially when considered under specific constraints. If you have a programming problem, and you have a bunch of data in a collection, and you need to access an arbitrary item in a few steps without iterating the entire collection, you need to know about hashing and related concepts (another contrived example, there's obviously more than one solution). This isn't something you would happen to know before hand, either you are taught about this, or learn it on your own, but you can't solve this problem without having some kind of experience to draw on.

How you gain experience is up to you (university, code academy, hobby projects, internship, etc). Learning by doing will be the most effective. Knowing how much experience you need is up to you. The depth of computer science is absolutely endless, no one will ever know everything relevant to programming (in practice, or academically), but the more experience you have, and the more tools are available to use, the better equipped you will be to solve problems in the future.

Answer 3

What I would tell my students, at least at the beginning, is that "seeing" how to solve novel problems really just means "seeing" how to (1) separate a problem into its component parts, and (2) learning a programming vocabulary that allows you solve very small problems.

Later, as you mature, you will better distinguish good programming from bad, but at the start, those are the two skills you need.

I suspect that there are already classes of problems that you can solve. Can you make a function that takes a number and doubles it? Can you make a function that takes two numbers and subtracts them? Can you make a function that takes 5 numbers and gives you back a + b - c * d + e? If so, then you are well on your way. Making functions is one of the main techniques we use to turn problems into smaller subcomponents.

Let's say you wanted to create a game without a game engine (NOT a good idea for a beginner!). You want your character to create a bullet that will fly at an enemy. It will be a tracking bullet, no aiming needed. What are the sub-problems?

Perhaps you thought of something like:

a. create a bullet by the location of the player b. figure out which direction to send the bullet c. move the bullet a little bit towards that direction d. check whether it has hit the enemy e. if it has, do damage to the enemy f. if it hasn't, go back to b again

If you see that, then the next step is to take any one of those sub-problems and break IT into its sub-problems. And so on and so forth. At some point, most of those sub-problems get small enough that they become pretty trivial.

You see?

Answer 4

First, there are different sorts of programming problems. Many require some domain knowledge to solve them. And if you work within a domain, then practice solving problems in that domain will serve you well.

Some "elementary" programming problems, such as determining whether a number is prime, require a bit of math knowledge. Some others, such as building web sites require a lot of knowledge of various kinds, some technical and some in the realm of human computer interaction (interfaces, say).

Some problems are just plain tricky and are intended to be such. But those aren't the sorts of things that a working computer scientist will find the most valuable in a career. Fun, though. But, again, practice with those leads to some insight, perhaps into the way that such problem creators think about things.

A database theory course, for example, will require some deep thought into complex data structures (b-trees and such) and also into language design and implementation. Complex stuff that requires domain knowledge background.

But, in general, the two main elements are sufficient domain knowledge and enough practice to apply it effectively. Practice with simpler problems may supply sufficient insight to give you a hook to solve more complex problems, but you need that underlying knowledge first.

Answer 5

Your examples (Roman to integer, update snake) are problems that may not need as much creativity as you seem to think (and they are not really "new" ;-) ).

My strategy would be to approach them incrementally. If we take the Roman number conversion as an example: I wouldn't exactly know a general algorithm off the cuff, if that's any consolation. I would approach the problem incrementally. Start with single letter numbers, continue with monotonously falling sequences, finally consider the difficult case with non-monotonous sequences. At each step, observe how you, as a human, solve it. Try to make the strategy you follow unconsciously explicit. Play through edge cases first in Gedankenexperiments, and then in tests.

The latter step — non-monotonous sequences where sub-numbers must be subtracted — is the difficult one. I'd tinker around with special cases until I notice a pattern, either in the data or the program. This pattern will lead to a generalization, a true algorithm: Roman numbers are additions and subtractions of sub-numbers: They are nested. Therefore, parsing can probably be recursive.

It is not uncommon to see the best (perhaps "true") solution only after many iterations, perhaps when revisiting code after while. But working iteratively will get you there, eventually, even if the best algorithm is different from your original approach.

Answer 6

At first writing programs to solve new problems seems like an impossible task.

This is in part because everything is new at once.

Experienced programmers tend to forget the large amount of knowledge we need to just get started. Each little bit of knowledge is trivial, but there just so many little bits. I'm not just talking about the programming language, but basic things like "What is a file?" and "What do you mean, 'æ' is not a letter?"

Sure, you can look things up, but then you lose track of your thoughts.

And when things go wrong, as they always do, you don't understand what because there so many things you are uncertain about.

The second main stumbling block is that the most powerful problem solving technique is recognition. You remember that you have solved a similar problem before. But you haven't solved any problems before, this is impossible.

The way past both of these problems is practice. And this is hard work.

When reading a tutorial, type in the examples given. Even the most trivial ones. Typing things in gives you better memory than just reading them.

This will teach you some of those trivial bits of knowledge you need. And every example is a solution to a problem, which adds to your store of problems you have solved before.

And then there is the exercises. At first, you can't solve them. So, you look at the solution. And then? Move on to the next problem?

WRONG You job has just started.

First, type in the solution. Muscle memory is good memory.

Second, read the solution until you understand it.

Third, clear the screen and try to solve the problem again. The solution doesn't have to identical, it just has to work. In fact, it is better if you don't get an identical answer.

Fourth, find out why that didn't work. Give it a good try before giving up and looking at the solution again.

Fifth, every time you look at the solution, you must clear the screen and start solving from scratch again.

Sixth, when you have finally solved it, take a short break before moving on to the next exercise. Hurray!

Seventh, the next day go back to the first exercise again and solve it from scratch.

Eighth, the next week go back to the first exercise again and solve it from scratch.

Ninth, the next month go back to the first exercise again and solve it from scratch.

Tenth, maybe, the next year go back to the first exercise again and solve it from scratch.

At this point this problem should be firmed lodged in your library of problems you know the solutions to.

Answer 7

A few things you can do:

• Practice (starting on simple problems)
• Use test-driven-development.

Test-driven-development

Do tiny steps

Solve a simpler problem first. E.g for Roman to Arabic conversion. Solve

• i → 1
• ii → 2
• iii → 3
• iiii → 4
• iiiii → 5
• v → 5
• vi → 6
• ⋮
• iv → 4

However, to do it properly, one would not decide the next step before completing the current step.

Automated testing

Write an automated test for the next step, before trying to solve it.

Answer 8

You are in a forest, with no belongings other than the shirt and shorts you're wearing. You want to be sheltered from the rain because you don't want your hair to get wet. Start.

Genuinely try to come up with a realistic rough description of what you would do before reading on.

---

After how much time do you give up on a problem and look at its solution

"Its solution" implies that there is One True Solution, which is not the case and likely the source of why you think you can't find "the" solution.

While building a 12 bedroom mansion would definitely be a solution to the problem, you probably don't know how to build one from scratch, and it's significantly disproportionate to the scope of the problem.

But I didn't ask you to build a mansion. There are countless ways to solve it. Some are obvious, others require some lateral thinking.

• Build a 12 bedroom mansion
• Build a tent
• Build a canopy

More importantly, I didn't even ask you to build a shelter either. The only thing that you have been given is a requirement: don't get your hair wet.

• Use one of your clothes as a hat.
• Craft a makeshift umbrella
• Find a tree you can hide under when it rains
• Wander around and look for a cave
• Shave off your hair. Can't get wet what you don't have.
• Stop caring about having wet hair

All of these are solutions in a way. Maybe some of these introduce secondary problems and therefore invalidate them as a good solution, but that depends on a lot of context.

Most of the guidance online seem to be about roadmaps like "Learn Data structures and algorithms first, then learn object oriented programming, practice hackerrank problems"

Advice like this is often not helpful in the way that the advice giver thinks. They're listing necessary pieces to the puzzle, but this is not always something you can just learn one after the other without any context until you can put it all to good use at the end. People usually don't learn this way.

A much more intuitive way to learn is to try. Achieve at least something, even if it only works in a subset of cases, or achieves only part of the goal.

Once you've reached that point, take a step back and play around with what you've got. Do you see any small issues that you know how to solve/improve? Can you find some edge cases that you've forgotten?

Only when it feels impossible to improve on it further, that is when you check others' work and see what they could come up with. Is it notably better than yours? What did they do differently?

It might be easier to learn how others' solutions differ from yours, rather than learning how to build others' solutions from the get go. In the former scenario, you've already built up an understanding of the problem domain and are therefore more receptive to alternate solutions and improvements.