Smart Software Testing - My Testing Blog

Software Quality Transcription

With today's fast paced production and testing cycles, there is certainly a shifting trend in the technology hierarchy (listed below).

1. Developers
2. Testers
3. Tech Writers

And by hierarchy I don't mean relative importance but the actual level of involvement with technology. It is quite obvious that things are changing. Testers are working more closely with developers and they don't report just defects anymore but also the fault that caused it along with suggestions for a probable fix. All this has been made possible nowadays by the high technical skill requirements and expectations for a tester. It is interesting to see what this might lead to.

Developers love coding, not testing.
Testers love testing not documenting (writing).
Tech writers love writing not coding.

See a cycle here?

If everyone were allowed to do just what they liked, I'm sure the team will be a lot more productive. Its already happening to an extent. Developers don't test anymore (except for Unit testing) and you couldn't see a happier bunch at the workplace today :). Their creativity and time is invested 100% in coding.

Now, how about we do the same with Testers. They could devote all their creative energy towards thinking up new and devious test cases. Documenting test cases might hamper the thought process! Well, ever heard of a tester who says he/she loves documentation? I already know the answer to that one! The point is, why not let the Tech Writers work closely with the testers and do some Software Quality Transcription. This is analogous to Medical Transcription. If it can work for them, it sure can work for us!

Just think about it. Testers can go thinking up test cases in a verbose manner and Tech Writers can document it on the fly. It can also be recorded and documented later on. This would require Tech Writers to be little more technically inclined, just as the testers had to step up to the developers. It would certainly result in an efficient and close knit team.

As a Tester myself, I'm for it. What do you think?

Based on my experience, the most important factor makes an inspection successful is human psychology. Well, lets face it. Who would like their brainchild (code) to be put in the line of fire of a bunch of people whose sole intention is to find defects? Of course, this is not the best of attitudes and the inspection team is really there to help you but why not avoid certain delicate situations when you really can?

Its a well know practice that upper management is usually not involved in inspections and mostly peers are made a part of the team. This reduces a lot of tension.

However, analogous to the Hawthorne effect, the mere fact that the entire inspection process is being recorded and is being reported to upper management, keeps the tension level high for the code author.

I would like to propose a change in this reporting structure that would make inspections less formal but still retain their value. Let the recorder record all the details as usual, but the final report to the upper management at the end of the inspection process should be made by the author himself (or herself). The author can obtain the details from the recorder and prepare his/her own report. This report should also include how the author proposes to rectify the defects and the proposed timeline. This way you also get a written commitment from the author. The tension levels are reduces as the author is sending in he report himself and feels more confident and in control. At the same time details will not be left out of the report as the author knows that a copy exists with the recorder (based on Hawthorne effect again).

Based on market trends, I have always believed that to be successful in sales you need to focus on size and quantity. The success of these factors is in their inverse relationship.

Case 1: Mass produce a simple scaled down version of a product and sell it cheap. Generic use, universal compatibility and ease of use is the key here.

Case 2: On the other hand, produce a limited amount of high end, specialized version of the same product with all the bells and whistles (Also includes custom made orders).

Consider selling 10,000 pieces of product 1 at $100 a piece:

10,000 x $100 = $1,000,000

Now, consider selling 100 pieces of product 2 at $10,000 a piece:

100 x $10,000 = $1,000,000

Either way, you make good money. The point is to stay at either end of the market and not in between. Mathematically, it can make sense to remain in mid-range where you still get a million dollars by selling a 1000 pieces of a product priced at a 1000 dollars, but in reality, you never get so many customers or be able to sell at such a good price.

The reason, is the way people think. It's ok to try out a cheap product just because it is cheap. You don't lose much even if your decision was wrong. This mindset actually attracts a large number of customers. What the seller loses in the profit margin can be gained via number of sales.

On the other hand, there is always a small market for specialized goods. People who don't mind spending a big amount for something they really need or cannot avoid. One time investments, Custom services and goods, etc. This gets you the large profit per sale. What the seller loses in sales quantity can be gained in the per sale profit margin.

All right, now does this have anything to do with software testing? It definitely does! We all know by now that adopting test automation is a slow and expensive process. Replacing manual testing completely with test automation for a new project will only spell disaster. Instead, the above marketing trend can be used successfully in software testing. Here's, how...

Let test automation evolve in your organization along with manual testing. Service small automation requests across different projects in your company at first. Create test automation snippets that are generic and can be used across all projects (For example a script to test the login screen). Gradually setup the automation framework and build a library for various test utilities (Useful later on for component based testing). At this point, though automation is not fully developed, it is still generating a good ROI by servicing several projects at the same (Though in small ratios). It even gives time for the test team to train and be useful simultaneously (Indirectly reducing training period)

Once the independent testing team within the organization feels it is ready enough to take on bigger tasks, it may take up the entire testing responsibility of a single project. It would now have to churn out custom testing scripts and test data, however this dedicated and extra effort for a single project (customer) is more than justified by the increased ROI obtained by testing the entire project.

Staying at either end of the market helps! Going through the intermediate stages should be for transition purposes only.

Blogged on: 20 Jan, 2005

How would you like to be in control? I would love it! In fact everybody likes it. It gives you a sense of security and well being. This is exactly the feeling the marketing people cash in on. Right from the auto industry to the food retailer, you as the customer are given 'choices'. Accessories for your car, flavors of ice cream, all these give you a sense of control. This feeling in turn encourages you to make an investment.

Now look at the software testing industry. I wonder if the test automation tools became popular because of their practical need or if they were a welcome relief to the only option - manual testing. As a beginner in this field, I often used to come up a wall whenever I started a new testing project. The fact was, with manual testing, you seldom know where to start! (Don't even mention knowing when to stop testing...). Until I learned boundary value, equivalence class and other techniques, clicking away randomly was the default approach.

Probably that's why test automation tools were adapted so rapidly as it gave you a well defined approach to testing. This sudden interest in test automation tools has unfortunately given rise to a myth that these tools can replace manual testing. Not quite. Actually tools were an outgrowth of automation code used to repeat certain processes and artificially induce system load. Even today their most obvious use is in regression testing and stress/load testing.

Don't get me wrong. Test automation tools today aid us a lot in testing projects. However, I just cant help wonder if their mere presence as an option ensured their survivability.

Besides the obvious reason 'Quality', I feel there are several other vital tasks performed by the quality department that no other team can perform quite as efficiently.

- The quality team functions as the 'internal customer' for the organization. The issues identified and resolved by this group saves the company's image and reputation which otherwise would have ugly consequences had the real customers discovered the issues. Having an internal requirements to confirm to really helps.

- Reporting! This is something that the upper management cant do without. Converting all that data into information is on of the other important responsibilities of the QA team. Preparing reports with emphasis on the relevant data for the right people is an art in itself. These reports are so powerful that they even determine the future funding for the development teams.

Prototype development model would be more productive if the prototype components may be reused. However most prototypes are made with only one thing in mind, "Its just a prototype". They are not made with the same care and effort that would be given for the actual product. Hence most components cannot even be reused. So, in short try reusing components, provided the prototype has been made keeping the 'software reuse' factor in mind. Otherwise don't even think of reusing the components. You will only end up introducing defects into the system from the sloppily coded prototype (Prototypes usually have a lot of hard coding in them).

If I were to answer that question, continued research in my area of interest is my long term goal. What ever 'post' I may be at an organization, I would like to have the enough authority and autonomy to implement, execute and try out all the concepts I've either come up with or learned over the years for the benefit of the company I work for. It doesn't matter if I am known as the "Director of QA" or as the "Lead automation engineer".

The past few years have been a wonderful learning experience for me. However, it was also the period where I did what I was told and not what I felt. They were probably right, since I lacked the experience. I should soon be able to change that (Working on it :-) )

The creation and execution of test cases have been automated. Where do we go from here? Automating the next important task of the software tester, the defect report, of course! Most test case management systems nowadays incorporate a detailed test case format. The test cases may be ranked by category, priority, severity and many more relevant criteria. This makes it all the more easier to automate. If a test case being executed by automated system fails, the defect reporting system may auto-generate a report using the title, description, priority, severity, category and failure details (say a log dump etc.). It can also forward the report to the tester concerned who may add to/modify the report if required and log it into the defect tracking system.

As a Teaching Assistant for the Software Testing course at USF, the biggest cause for dissatisfaction I find among students is the documentation and report writing process. Most students are excited about finding bugs but are not so enthusiastic when they have to type out pages to report the same!

The point I am getting at is that all the required information is readily available. Its up to us to tap the potential and make the testing process more productive.

I'm just back from successfully completing a simple project but it had a lot of complex settings. I also had to integrate several readily available utilities into this application that I was building. I had a real tough time putting them all together and getting them to work as a single piece! Now that I am done with it, I have to acknowledge the help I got from one friend of mine. He's a file called "The Log".

Never underestimate the usefulness of the log file. It may be simple text file thrown in some corner of your hard disk, but it contains a wealth of debugging information. Basically it lets you see what the application is doing internally. This information is priceless, especially when you have a .exe in hand and you cannot see the code. (Remember, how we would debug code by placing convenient I/O or print statements between lines of code?)

As a developer, always try to put in a procedure to create a log file with all of your applications. You will be doing yourself and the tester a big favor. (It reduces the chances of the tester filing a bug report with insufficient information)

As a tester, always look for the log file if something goes wrong. Keep in mind that there may be several log files generated in a multi component system and not all log files will be placed on the same disk location. Another point to remember is that the log file generation is not enabled by default. You might have to check the application settings for the option.

An often overlooked point during testing, especially involving automation is testing beyond the GUI. Most automation programmers write scripts that click buttons, pull down menus, fill forms, navigate web pages etc. However what is being overlooked is the fact the data at the back end isn't being validated. What you see via the front end GUI may not be what is actually being stored. In a more complicated scenario, the data you see may match the one being stored in the back end, but it may not be the same during the intermediate stages. This is important, especially when you use the data even before final storage for use in expressions and formulas (formulae).

In any automation project, it is extremely important to go the full circle. What I mean is that, when you write a script to perform a set of operations in sequence, make sure you restore the system under test to its original state using the same script. This needs to be done as automated testing generally involves running the script several times on the same machine (Running a script several times may not be necessary always, but in some instances, I've seen it being done just because it can be done! :) ) This system restoration is necessary for the script to execute successfully the next time it is run. During this restoration, not only the application needs to be restored but you also need to clear the buffers, data repositories and other test run byproducts.

Simple. Why do we want humans to test something that was created by humans. Testers are equally likely to make mistakes. If we humans were so perfect, why did we make those coding errors in the first place? This is where automation steps in. Agreed, computers don't add any creative value to the testing process. That's our job. Testing? That's the computers job!

Quality Realization

"Quality is free" - Isn't that a well know quote? Then why don't all organizations have it? The answer lies in the way quality is acquired. Quality is free but not easy. It takes a lot of time and commitment. Though it doesn't take long to setup quality processes, it does take time to reap its benefits. This may even take years. That is why it is very important for the management to stay committed to the quality cause.

A bigger danger lies in becoming complacent. Just as it takes time to achieve quality, it also takes time to deteriorate. If one employee decides to ignore a part of a process, there may not be much of an effect on the overall system productivity - at least not immediately. The flaw is hardly noticed. However, over time this flaw seeps in and affects other parts of the project. Eventually, quality decreases and by the time this is realized, it is just too late. You would then need a couple of years to get back in to shape.

I would think of attaining quality as a broad curve (Where the dip of the curve depicts quality deterioration and vice versa). The point here is that it is gradual. You cannot expect to see a spike. Keeping this in mind helps on the long run.

Quality is a long term commitment but its well worth the wait.

I'm sure you must have come across the popular joke that goes like this, "If cars crashed as often as software, there's a good chance that I wont be around to complete this sentence". Funny yet true. This got me thinking. What is the underlying difference in the processes that are followed to make these two products? Both follow stringent industry standards. Both have authorized bodies governing them. Then why this stellar difference in output?

Yes, its the processes. What else could cause such a difference? Look at how cars are made. Every car in the assembly line goes through the same treatment. All operations performed on them are identical to the millimeter. The end result? A set of identical cars.

Now, imagine the same with software. To produce identical software, we simply create a copy! (A pity that the same can't be done with cars :) We don't need to code from scratch in order to make a copy. If that were to be done, then to create enough copies of the Windows operating system for the whole world, it would take hundreds of years!

See the difference? The processes we have for software are meant to create 'similar' software and not 'identical'. Now this is a huge difference. A difference that clearly allows for more defects to creep in as you custom produce software for a client while at the same time staying within the process limits. With this perspective, I'm sure you can now see why cars don't crash as often as software!