Problem-Solving

Note: you can listen to an audio recording of this material on our supplemental materials page.

The title of this course is "Computer Programming and Problem Solving," and in large part we are teaching you to solve problems by experience. Practical experience writing code will sharpen your instincts and help you solve all kinds of problems, but even when solving problems from experience, there is a specific set of steps to follow, which we've outlined below. Also, there are a few other general problem-solving skills that we'll talk about, like incremental problem solving and the divide-solve-combine approach.

Problem-solving Steps

  1. The first step is to understand the problem. Read the problem statement carefully, whether you're working on a quiz question or a problem set task. Then, reformulate the problem in your own words. If you're working on a quiz, you can just imagine these words; when working on a problem set you should write down your understanding of the problem in your code file, either as comments, or if you are writing a function, as the documentation string for that function. Research shows that skipping this step is one of the biggest causes of problems, but those problems don't show up until later in the problem-solving process, so it also can cost a huge amount of time. Note: your description of the problem should include at least one and ideally several concrete examples of what the program is supposed to do in a particular situation.

  2. The second step is to identify similar problems that you already know how to solve. These can be problems from lab, or lecture notebooks, where you've gone over the solution and you know how it works. These won't be exactly identical, and the more dissimilar they are, the more work you will have to do to adapt a solution in the later steps. However, this is the key step that allows you to solve the problem: by understanding how it's like another problem you've already solved, the main thrust of the answer will become clear to you.

  3. The third step is to plan out a solution, based on the similar problem you identified in step 2, but adapting that solution to meet the requirements of this problem that you understood in step 1. Whether you're working on a quiz or on a problem set, you should write down your plan as a series of comments in your answer. These comments (in English, not as code) will form the outline of your solution. It will take some practice to understand the right level of comments to write at this stage, but think of it like outlining an essay: you want to hit the big points, but leave the details for later. One way to think about this step: imagine you are asked to solve the problem as a human, rather than trying to program a computer to do it. What steps would you take? How could you write down those steps as instructions so that another person could do the job based on those instructions? By the end of step 3, you should have both a description of the problem and an outline of a solution written down, but you haven't written any code yet. Again, skipping this step may seem like it will save time, but on average it will cost you a huge amount of time. Also, on a quiz you can often get partial credit if you have included comments describing a correct strategy, even if your implementation of that strategy has errors in it.

  4. Step four is where we start to write code. Based on your outline, you are writing a first draft of your solution. It's just a draft, so it's okay if it's messy and incomplete. The point is to put down some code that follows your outline from step 3. During this step, you will often realize that you forgot something in step 3, in which case you should add a comment explaining the extra step that's necessary as you write the code. This step is really hard, because you're suddenly being asked to deploy all of your memorized knowledge of a programming language, including vocabulary (which functions and operators to use and what do they do) and grammar/syntax (where do you put the parentheses or colons, etc.)! This is something that you'll naturally get better at with practice.

  5. Step five is to test your program. You've finished a draft, but of course you still need to proofread it and polish it up! Your first draft probably has many errors in it, but one skill that you're building is how to quickly spot and fix those errors based on the feedback that your programming language gives you. Being a good programmer is not about never making mistakes, just like being a good writer is not about never making a tpyo! Instead, you want to become skilled at using the tools available to you to quickly find and fix the mistakes that you will inevitably make, and the first part of that process is to find your mistakes. At first, simply running the program will probably produce an error message, but as you weed out some of the earlier bugs, you'll run out of error messages. At that point, use the concrete examples that you developed in step 1 as points of comparison: run your program using the same inputs, and see if you get the result that you said you should expect. In some cases, you might have made a mistake in part 1, and you can revise your examples (it's hard to see exactly how the program will work before you've started writing it). In other cases, you'll find that your draft program doesn't do the right thing, even though there's no error messages any more. If you test your program and it works for all of the examples you can think of, then you're probably done (but try to think of some extra complicated examples first).

  6. Step six is to fix your program, now that you've tested it. You will first need to deploy debugging skills during this step to refine your understanding of why your program doesn't work, so that you can figure out how to modify it. If you're getting an error message, you may still need more information about what's going wrong. If you don't have an error message but your program's behavior isn't correct, it's even more important to understand why so that you can apply the correct fix, instead of making a guess and potentially making things worse! You should use at least the most basic technique during this step: add print function calls to your code so that it reports key extra information as it runs, and look at what it spits out to figure out what's going on. Add as many prints as you need to get a good picture of what's happening. You could also use the debugger to run your code step-by-step. In any case, when you've identified what's causing the problem, modify your code to fix it. Sometimes at this point, you'll have to go back a bit in the process, even as far back as step 1. Maybe it becomes clear to you that you misunderstood the problem description, for example, or maybe you realize that in step 2, you misidentified which other problems you had already solved were similar. Back up as far as you need to and proceed again, at the very least repeating step 5. But don't feel too bad about that: your final program will be better as a result of your revisions, and you're also learning from your mistakes (you'll learn more from making a mistake and fixing it than you will from getting something right the first try). Even if you have to throw away some work, the knowledge you gained from that attempt will help you in your next attempt and when solving future problems!

If you have tested your code in step five and found that it finally works, congratulations, you're done! One wrinkle is that on a quiz, you won't be able to do steps 5 or 6 for real, because you can't run your code. However, you should be learning how to simulate code and imagine what it will do over the course of this class, so you can do steps 5 and 6 that way. We also usually try to keep the quiz questions simpler so that there's not too much to manage.

Incremental Problem Solving

You might have noticed that the problem solving steps above are a bit unwieldy to apply to an entire problem set task! Although writing down an outline of the steps you want to take is helpful for keeping yourself on track, there's so much to do that when it comes to steps 5 and 6, things can get a little bit overwhelming. One problem solving approach to apply in these cases is called incremental problem solving, and it basically says that you should only solve a little bit of a problem at a time. You'll still start with steps 1–3 above, but in step 4, instead of trying to write code to complete your entire outline, just write code to do one or two of the steps from your outline.

At this point, you'll need to look at your examples from part 1, and come up with intermediate expected results. What should be true after the first few steps of your outline, for the specific inputs or examples that you had in mind? Write these things down as comments, and then proceed to steps 5 and 6 for your partial program, testing the code that you've got so far to make sure it works correctly, even though it's not really solving the whole problem. Once you've finished the revision process for the first few steps you took, return to step 4 for the next few steps from your outline, and repeat this process as many times as it takes to complete your program. At every step of the way, you should be able to test things and build confidence that the code you've written so far works correctly, so that when you're writing code for the next step, you don't have to second-guess yourself. Occasionally you'll find you have to revisit some of that earlier code and revise it further, but this shouldn't be that common.

It definitely takes skill and practice to use this incremental technique, but it can help to break a too-big problem down into more manageable chunks, although see the next section for another technique that can help in a similar way.

Divide-Solve-Combine

This approach is similar to the incremental approach, but a bit more deliberate, and it involves changing the nature of your solution instead of just the process taken to construct that solution. Of course, by doing so, it often results in more modular and reusable code, which is a good thing. The divide-solve-combine approach, like the incremental approach, is useful when faced with a problem that feels too big or complicated to solve all at once, and it can also be useful when you get stuck on a single step of a larger problem that might have seemed simple when you were building your outline, but that turns out to be tricky when it's time to write code.

The essence of this technique is to divide your problem into two or more separate parts, such that solving those parts individually will solve the whole problem with just a little bit of glue to combine the individual solutions. Like the incremental approach, it takes skill and practice to learn how to apply it well, but it's a quite useful and general technique. The steps are as follows:

  1. Identify that your problem is too big to solve all at once, or that you've reached a complex sub-problem of a larger problem you were trying to solve.

  2. Decide how to cut up the larger problem: which parts are mostly independent of each other, and if they were solved individually, how could their solutions be combined to solve the larger problem? In some cases, this step is quite hard. Write down your plan for dividing up the larger problem as comments in your code.

  3. Reformulate your smaller parts as individual problems, and write down how you plan to combine their solutions. Then, apply the full set of 6 problem-solving steps to each sub-problem one at a time, starting from describing what the problem is and providing examples of how it should work, and finishing up once you've gone through several rounds of testing and revision and you're confident your solution to the sub-problem works. Repeat this for each of your sub-problems.

  4. Based on your ideas from step 2 about how to combine the sub-problem solutions, write the code necessary to do that, and follow steps 5 and 6 of the general problem solving approach to test and debug your combined solution. It's possible that at this point you may have to revise one of your sub-problems a bit, but you can go back and forth between the larger problem and the sub-problems until things come together.

Although this approach might sound like extra work (I had one problem to solve and you're telling me to create two!?!), it comes with two important benefits. First, it reduces your cognitive load: by dividing the original problem into smaller, simpler problems, you'll reduce the amount of information you need to hold in your brain at once, and especially while you're still exercising to improve the memory structures you'll need when programming, this is a huge benefit. Second, the code you end up with will be easier to understand, easier to debug, and more modular, meaning that it's more likely to be re-usable in the future.