The ‘5 Whys’ technique to debugging an issue

Debugging an issue in code can be a time consuming and difficult process. This can be made even more true when debugging a multi-threaded application that has a bug caused by a race condition, or being thrown in to an unfamiliar code base. However, there is a possible methodology that can help traverse this madness, and that’s the 5 Whys technique.

The 5 Whys technique was used by Toyota as a way of diagnosing issues on their vehicles. Below you can find the same example that was taken from the Wikipedia link on how the method would play out amongst the technicians at Toyota when a car comes in with an issue.

Issue: Car has entered the shop and does not want to start.

  • Why is the car not starting?
    • Reason: The battery is dead.
  • Why is the battery dead?
    • Reason: The alternator was no longer charging the battery
  • Why was the alternator not functioning properly?
    • Reason: The belt providing power to the alternator was broken
  • Why was the belt broken
    • Reason: The belt had been worn to the point of needing to be replaced
  • Why was the belt not replaced
    • Reason: Car was not properly maintained

The above scenario helps the team focus on trying to diagnose and understand the problem. It gives a clear question of what needs to be answered before we can move along and drill down into the issue further. Let’s now apply the above example into a bug that I am currently working on. I’ll also break down the steps a bit more as to how I came up with the answer to the question

Issue: Application hangs when zooming out.

Why is the application hanging

The first why can be the most difficult to answer as there is just a generic broad question that requires a bit of information. Luckily the hang was caused by a unit test that timed out after an hour of running. Because the unit test was nicely laid out into separate steps, it was easy to start adding some logging as to which part of the test was not being executed. If this was a statement coming from a user, I would then have to ask the user about what they were doing to get the hang to occur and try to get as much information from them as possible to try and recreate the issue locally.

Once the section with the last executed code was found, we can then start going and trying to identify what was going on. In my case the application was found stuck in the gesture handler. With that we had an answer to the first why.

Why was the gesture handler stuck

Now we can move onto the second question. The gesture handler was being used to set the set a scaling factor to be applied to the world and the objects within it so that more objects can be visible within the camera. It was also used to try and center the camera relative to the center of the 2 fingers performing the gesture. However, the gesture handler was returning an error while trying to re-project the center coordinate into screen space if the gesture was being done too fast.

Why was the coordinate failing when projecting to screen space

Looks like we’re now getting a better idea of what the issue can be. Some projection is failing to get converted into screen space. With that I started digging down into the camera and seeing if for some reason the camera viewport was not updated correctly as this was a multi-threaded application and indeed that was the case

Why was the camera not being updated

We’re almost at the home stretch at question 4. Here I know that the camera and rendering logic is happening on a separate thread. After investigating and finding the correct thread I can see that we are indeed inside a method that is updating the camera. After further investigation we see that we have reached a deadlock between an object’s resources in 2 threads.

Why is an object attempting to be accessed across multiple threads

Without getting too much into the details of the fifth why and the simple resolution of refactoring the logic a bit so that the locks can be freed earlier, we can see how this technique can be used to help focus on an issue. While this might not speed up the debugging process and there may need to be some questions to be answered by other members on your team, as they may be a bit more knowledgeable in certain parts of the code, it will help you in trying to understand the problem better and will make you better at debugging issues in the long run.

2019 Retrospective

2019 was a year with many things happening at once, with the major change being that I had left my job of 7 years. This brought along fears of not being up-to-date with the current programming skill set. Up until my departure I have been working with C++03 and boost 1.61. While this allowed me to stay somewhat current with shared_ptr, recursive_mutex, and for_each loops. With my time freeing up and taking on new jobs, I was able to play a bit of catch-up with unique_ptr, move semantics, and lambdas with C++14/17.

In my previous post https://slewicki.com/wordpress/index.php/2018/03/05/does-having-a-senior-title-actually-mean-that-youre-a-senior/ I mentioned my worry about if my years as a developer actually meant anything since interviews would shut their doors at me. Some would state that I was behind in my programming knowledge, while others would ask complicated logic questions to be solved within a certain time frame. This led my morale hitting an all time low, but luckily I was able to get a chance and show what kind of an asset I can be.

If you find yourself in a similar situation where interviewers keep closing the door at your face, work making you feel stale, or just feel like you need a change but don’t know where to start. Try and find a few minutes a week to brush up on something to help advance your career. If a significant other or kids are involved then chances are that you will most likely be foregoing some sleep in order to push yourself, but there is always a way to help improve. Try a side project that helps solve an issue you have using a language or framework that is relevant to your career or interests and just keep pushing.

It’s rough out there, but there is always a little glimmer to help show that it can be better.

A New Adventure Begins

Many of you may be surprised to see a new posting on this site after a huge hiatus. You may be asking yourself some of the following:

  • Is the site now under new management
  • Is this another promise of trying to do something only to get nothing in the end
  • Will there ever be any useful topics on here worthy of my time

These are all valid questions to ask and if you’re here, you either stumbled upon this site looking for programming topics from a search engine or were subscribed to this blog from long ago… in a galaxy far far away (sorry I couldn’t resist). If you’re still with me and willing to learn why I’ve decided to attempt this for the thousandth time, feel free to continue. If you are looking for some new and exciting content, I’m going to try small and say that I’ll attempt something new each month.

What does all of this mean then

The past 2 years have been a rough obstacle course between unfortunate circumstances in my personal life, huge life changes, and meeting with esteemed individuals in the software development field (friends and family included). The above took a huge toll on me personally adding stress and causing some health issues to arise a little too early in my life. All this came to me coming to terms with me needing to reevaluate what it is that I need to do with my life. And it all started with me leaving my previous job.

Come again? You did what?!

It had to be done. Yes that job is currently the only thing I know at the moment, but I can’t let that be the case. I’ve nearly worked myself to death constantly pushing myself and always worrying about how the last release went, or how the next day will come and go. I need a break from it and that lifestyle honestly.

So what’s next in that case?

Honestly, at this very moment, I’m not entirely sure. I’m going to take a small break and recharge. Maybe go out on a drive, meet-up with some friends of old, or try my luck at programming again to see if I can find some joy in it like I used to. If anyone has any suggestions feel free to comment. Willing to give something new a try at this time. Until next time.

Does having a ‘Senior’ title actually mean that you’re a senior

Image courtesy of istockphoto

I want to start by prefacing this article that it is more of a rant then anything else. Maybe it applies to you or maybe it fits someone you know very well.

 

I Have Experience, I Swear

For the past few months now I’ve been looking for a new job. Something that would be different then my current routine and something that could help my career by thinking differently or trying out a new stack. Figure that I have 7 years professional experience; that I shouldn’t have trouble looking for something new. That’s when I started to question my abilities.

When I started I wasn’t getting many calls. Figured it was due to being late December and people are out on vacation so I wasn’t expecting much. Starting from January the phone calls started coming in. Calls from the Big 4, financial firms, companies in the public sector, and some fun and interesting startups. First phone screens seemed to go well. My nervousness quickly subsided once we started talking and discussing what my role was, my greatest accomplishments, and what obstacles I had to overcome. Then they wanted to schedule a more technical interview, either through HackerRank or their own in house interview software. This is where I was getting kicked down over and over again.

Things like Data Structures and their time and space complexity, being able to design systems from scratch with APIs that I’ve never even worked with, false descriptions, or losing track of time and not finishing the coding exercise.

 

Data Structures (and the lack thereof)

Over the course of my career I’ve used third party libraries like boost and Qt for my development needs as we would be using versions of C++ that were missing features (shared_ptr I’m looking at you). A lot of features and functionality that had to be designed had to work around limitations of our framework as the framework was designed to handle quick data messaging between modules running on different systems. Unfortunately, we had to scrap our current product and create something new. Problem is that the frameworks short-comings were now showing with the amount of data getting loaded and processed and we just had to keep pushing as there was no time to revisit the issues and that we need sales…. but I digress. We used the frameworks functionality and tried our best to work around it. Because of that my exposure to other data structures were limited and to keep up with them was getting rough with the team constantly working to push out new features or fix bugs that were cropping up.

Without keeping up-to-date with the latest or even keeping with the existing data structures, I would go into interviews completely unprepared or even stepping over myself. I would be asked to implement a breadth first graph traversal and instead find myself implementing a depth first graph traversal. You would see me over-complicate implementing an insert function for a circular linked list with all possible edge cases. Sure I understand the concept, but when there is someone watching me while I’m implementing the code I freeze up and confuse myself. To which I started going into interviews stating that I would be implementing a brute force approach and if time allotted that I would optimize it. After the coding portion I would get asked how efficient in time my data structure would take and how much space it would use up and unless it’s O(1) or O(n) I would stumble with the O(2n) or the O(2 log(n^n)).

I Can’t Be That Incompetent…Can I?

After almost 3 months of this I felt like I had hit my lowest point in my career. At this point I’ve been questioning myself if I was even fit to be a programmer, but then a good friend of mine spoke to me and told me to ignore those nay-sayers. Sure my Data Structures were weak, sure I’ve been working in the same framework that prevented me from being exposed to something new and different, but I was still kept at my job for the past 7 years and no matter what was thrown at me I was able to tackle it. Now I must take upon myself to improve my Data Structure knowledge. In the mean time I need to remember that because I still have a job I must be doing something right.

Taming the snake: Learning Python and putting it to use

For the duration of my programming life I’ve been working with C, C++, and C#. If you ask anyone in the programming space you’ll see a lot of articles about how they know a multi-verse of languages and how they won’t even consider anyone a true programmer unless they have 25 different languages under their belt. So I figure that I venture down that trail and start with Python.

Going on an adventure with Python

I started taking some online classes from codecademy to see what this is all about. What I’ve noticed off the bat is that I’m still in my C-style ways of semi-colons and tab styles that would cause scripts to break. I have an old project that was setup in C#, but seeing as I have moved on to a Linux machine for work I figured I try working in a language that is a bit more cross-platform friendly right out of the box (maybe I’ll revisit having a C# application on Linux when Microsoft releases .NET Core Final Alpha Beta Release Candidate). I’m going to start porting over various scripts to my git hub as the bits and pieces come through. Hopefully the next update will arrive before the end of the 3rd quarter.

Motivation vs Discipline: Having an idea and executing it

I had a thought of creating this website where I can show off some of my work in progress websites and to help remind me to push myself towards learning and getting some of my work out there. Throughout that time neither seemed to have happened and this site was just out there collecting Internet dust. If you asked anyone what I was doing they would each say that I was working on a game, or some side project, or even learning a new language. If you’re like me then your initial reaction would be “Awesome, when can I see it in action!” and that’s where it would end. I would have tech demos sent to my buddies, or have a small little gameplay video to show but after that it would be silence. Life would intervene and push my side projects on the back-burner. When I found myself with extra time I always had 2 options open: Either get some sleep and prepare for my next day or attempt to go back to the project but completely forget where I left off and step through my logic to figure everything out. You could say that I had a big picture in my mind for where the project was supposed to go but never had the small puzzle pieces lined up as to what to tackle next.

Having a big picture of what to do is great and it gets you fired up to do something. You start to plan for the project by compiling a list of ideas and making a plan of attack before programming. I started to fail when the pieces of the big picture were still too big to put together within my short span of free time. Between working 60-80 hours of my professional job and spending the rest of the time keeping up with family and friends left me with a couple of hours a week for myself, which would mostly go to sleep.

With the limited time left I am going to try a different approach to my side projects to help not only motivate me, but to help with execution. I am going to see if I can split m tasks down into short hour or 2 bursts that can be found here and there. This way tasks can be simple and independent of each other without having to go back to another part of code and trying to see what had been done. I’m hoping that this will help break down my giant puzzle pieces into smaller ones.

If this little experiment works then hopefully there will be content here to show for it. Otherwise, we’ll be back to collecting dust and figuring out another way to approach this.

Wise words from an old mentor

I had many mentors as a Padawan in the world of programming. These mentors varied from helping me learn the first steps as a programmer from the famous “Hello World” program in C, C++, and C#. To the more advanced techniques and theories of data structures and various design patterns. I also got thrown into a world of 2D and 3D graphics programming with OpenGL and DirectX. But the mentor that stood out the most out of them all taught me about Linux, embedded architecture, and the famous x86 assembler. He was very knowledgeable and gave me the following wise words to always remember as a programmer. While he added his own spin on these rules, they always stuck by me while I was designing new functionality or going back and improving my existing code. Below are his words that were modified from Code Complete and from NASA’s 10 coding commandments that stuck with me:

  1. Comment your code with valuable comment not useless ones
  2. Don’t forget about error checking since anything can fail and it will
  3. Use less code (DRY don’t repeat yourself)
  4. Write code that can be easily modified but don’t make it unreadable. Code can always be thrown away.
  5. Write code that can be easily tested. If you did not test it then consider it broken.
  6. Fix the problem, not the symptoms. The symptoms are the obvious artifacts of the problem, you need to find the problem.

NASA’s 10 coding commandments

  1. Restrict all code to very simple control flow constructs – do not use goto statements, setjmp or longjmp constructs, and direct or indirect recursion. (well all of these can be used in moderation if need be but there must be a justification)
  2. All loops must have a fixed upper-bound. It must be trivially possible for a checking tool to prove statically that a preset upper-bound on the number of iterations of a loop cannot be exceeded. If the loop-bound cannot be proven statically, the rule is considered violated. (Good idea but not always possible but you must justify the exception)
  3. Do not use dynamic memory allocation after initialization. (of course this assumes that during initialization you know exactly what memory you need, which might be difficult to determine)
  4. No function should be longer than what can be printed on a single sheet of paper in a standard reference format with one line per statement and one line per declaration. Typically, this means no more than about 60 lines of code per function. (Good idea but limits be arguable)
  5. The assertion density of the code should average to a minimum of two assertions per function. Assertions are used to check for anomalous conditions that should never happen in real-life executions. Assertions must always be side-effect free and should be defined as Boolean tests. When an assertion fails, an explicit recovery action must be taken, e.g., by returning an error condition to the caller of the function that executes the failing assertion. Any assertion for which a static checking tool can prove that it can never fail or never hold violates this rule (I.e., it is not possible to satisfy the rule by adding unhelpful “assert(true)” statements).
  6. Data objects must be declared at the smallest possible level of scope.
  7. The return value of non-void functions must be checked by each calling function, and the validity of parameters must be checked inside each function.
  8. The use of the pre-processor must be limited to the inclusion of header files and simple macro definitions. Token pasting, variable argument lists (ellipses), and recursive macro calls are not allowed. All macros must expand into complete syntactic units. The use of conditional compilation directives is often also dubious, but cannot always be avoided. This means that there should rarely be justification for more than one or two conditional compilation directives even in large software development efforts, beyond the standard boilerplate that avoids multiple inclusion of the same header file. Each such use should be flagged by a tool-based checker and justified in the code. (could be a debate here but since macro are not well checked this is a good idea)
  9. The use of pointers should be restricted. Specifically, no more than one level of dereferencing is allowed. Pointer dereference operations may not be hidden in macro definitions or inside typedef declarations. Function pointers are not permitted. (No function pointers? Seems like the skill level expected here is suspect, but it does prevent static analysis tools from working well)
  10. All code must be compiled, from the first day of development, with all compiler warnings enabled at the compiler’s most pedantic setting. All code must compile with these setting without any warnings. All code must be checked daily with at least one, but preferably more than one, state-of-the-art static source code analyzer and should pass the analyses with zero warnings. (in some ways I agree, but suppressing the errors and warning can cover-up other problems)

Do you agree with these rules or are they outdated? Do you have other rules that you abide by that help make you a better programmer? Leave your comments below.

 

The above statements came from a post from my mentor Gary Miller. May he rest in peace.