Monday 10 September 2012

The Risk of non-reproducible bug

The Risk of non-reproducible bugs:

                 I have come across with few non reproducible bugs. When I found for the first time,  I was not aware of the criticality of that bug. Because I felt it happens in rare cases. Once I was going through the blog of a famous tester, in which he has mentioned about the risk of such bugs and also I was guided by my coach to investigate this issue. So I realized this may not occur today, but might occur in future. Because the issue is not resolved yet. As a tester, I am responsible for the quality. So again I started my investigation. When I was looking into this issue I found another issue which was more critical than the one which I was looking for. So I was thinking if such issue still exist, then what are the possible things would happen.

1. The user might lose the trust.
2. There is a chance for the user to move to the competitor’s product.
3. This could cause a big loss for an organization.

 Okay. What was the bug?
        When I was testing a site using my credentials, unfortunately someone else account has opened. I found that issue in a e-commerce site. As I was aware of the risk, I made some investigation on that issue. My first approach was :
          Changed the content in the URL and looked if  any change has occurred. I found that the page was loading very slowly. So I tried the same investigation on a different day at the same time, it happened again. I noted the speed and I found that the page was quite down between that time. So I realized that the server was down. Since there is a huge traffic to the site, the server was not able to respond to which the request was send at that time. So whenever the server was down I found some more issues. So I felt it’s just because of the server performance the issue has occurred. I could say that the non-reproducible bug is more dangerous than the reproducible. Because the reproducible bugs can be reported and fixed. But in the case of non-reproducible bug it’s quite hard. So more investigation is needed in the case of non-reproducible bug.

How do I approach:

 If any bug was found and not able to reproduce then I consider the following,

1. Nature of the bug.
2. Risk, if it appeared for the next time
3. Note down the time
4. Environment
5. Under what circumstances it appeared
6. Will it affect other functionalities.
7. What was my previous action.
8. Systems behavior (before and after the bug occurred).

Other possibilities:

In order to reproduce the bug, I look into some other possibilities like

1. Changing the content of the URL.
2. Opening some more tab and performing the different task for the same application.
3. Opening the different browser and  performing the different task for the same application.
4. The same operation under different machines.
5. Perform the same step and check whether the issue occurs again.

If it happens again, we might not be able to provide a proof for its occurrence. So it is always better to record a video while testing. Because that gives a strong proof for its occurrence.

2 comments:

  1. Pradeep,

    Share your email-id on blog.
    That helps you lot.

    ReplyDelete
  2. Pradeep, good to see your posts. It is no doubt that non-reproducible bugs are the scariest. It is like a crime scene investigation where most likely you can't reproduce the crime but have to investigate just based on the clues/open ended questions that comes with it.

    It may be an overkill to record video all the time. It is better to maintain a log on the following and in most cases they are available by default.

    1. Test log (including start, end, parameters, dependencies, type etc)
    2. Test environment log (including data deployment, code deployment etc)
    3. Relevant server (like Web, DB, App server) log to see the health of the servers
    4. Network log to see the complete health of it
    5. Tests, processes etc that runs in parallel with your test.

    The above holds good wherever you run the test.

    ReplyDelete