Saturday, August 27, 2011

Should QA learn programming concepts?



Recently one of the QA engineers in Eurocenter started an interesting discussion on the skills required from a current day QA engineer. The discussion was opened in the linkedIn group of Eurocenter which we call "InsideOut". It is an open group and you can join from: http://alturl.com/qtron
One of the questions raised was whether the QA engineers should learn programming concepts in order to dig into the code written by developers to review the quality. Though this idea was favored by some other colleagues, my opinion was that this is not required and there are lot more valuable areas a modern QA engineer can look into, for improving the quality of a product. Following is my elaboration on this. I directly copied the following part from my comment in the group.


It is agreeable that the industry demands more from QA than doing conventional testing. Major reason behind this demand is that many quality aspects required from a software product are not covered in conventional testing which includes functional testing and traditional non-functional testing like load testing. In native world, for example, memory corruptions might not surface during normal testing. However, they can cause disasters in production environment. This puts the need for a QA engineer to have more insight into the operation of a software application than he observes in the application interface. There are two main avenues in achieving this.
                       1. Code level analysis
                       2. Operation level analysis
The former can be called white box testing and the later, black box testing. The important thing to note here is that peeping into the code or writing supplementary code is not the only way of getting more insight into an application.

Operation level analysis deals with analyzing the operation of a software in the context of a system (operating system for example) where the software is evaluated with respect to the changes it cause on the system, how it is affected by changes in the system and how it performs under various conditions in a system. The good thing about this domain is that many sophisticated tools for doing these started appearing with latest versions of operating systems. Microsoft Application Compatibility Toolkit, for example, can monitor the operation of a software and provide a detailed analysis into security problems, user access control problems, memory problems, etc. One other good example is Microsoft Application Verifier which can detect memory corruptions, memory leaks, low level operating system call failures, I/O overhead, resource allocation problems, improper locks, unsafe parallel access of data, dangerous API calls, etc. These are vital information that help in deciding the quality of software without looking at a single line of code. Being a native developer for few years I still cannot detect most problems revealed by Application verifier by examining code. There are a bunch of other useful tools of this kind that are bundled with the operating system itself. These tools are little known and are not given the attention they deserve. Even when they are used it is done by developers. However, I think QA engineers are the best people to use them to evaluate software.

Even if we think about tools like Sonar they provide a lot of metrics that manifest the quality of software without going into code level. Once we go into the code level there are hell of a lot of peculiarities like design patterns, compiler optimizations, API tweeks, hacks, etc. Since software product is more important than the code I think it will be more productive to analyze the quality of the software itself utilizing modern tools. Having said that I repeat that it's vital for a QA engineer to have knowledge to automate things using simple programming.