Table of Contents
Software testing is one of the hardest tasks in any IT department. It’s also where projects fail most often. As an IT veteran, the lack of thorough testing in this industry still astounds me. I’ve seen “finished” programs from developers that wouldn’t even run. The code hadn’t been tested at all lessons from 40 years online.
Programs and systems must be checked against both a specification and a set of standards. This can’t be done randomly. You need a solid grasp of what your systems are supposed to do. Your specification might say “a standard URL will be accepted in the address field.” Your standards might require validation of all buffers for overrun conditions, URLs in valid format, and so on. Standards apply to all testing. Specifications are tailored to the specific program or system.
The True Nature of Software Testing
Here’s a hard truth: the marketing department is not in charge of testing. Testing requires dedicated, trained, and motivated people.
A fatal mistake many companies make is shipping product copies to thousands of beta testers without clear goals, expert supervision, or consistent management, hoping to get useful feedback. Beta testing matters, but it can’t replace a professional testing team. Another truth that escapes many people: beta testing is for testing, not marketing. Marketing is part of the product plan. It has no place in the testing plan.
The Common Pitfalls in Software Testing
The pitfalls in software testing are numerous, but a few themes keep showing up. Here are the 10 most common mistakes:
- Testing to prove a program works – It’s natural to want your programs to work. But the purpose of testing is to test, not to validate the programmer’s ego. Testing should be thorough and ruthless. Find the deficiencies. Record them.
- Attempting to prove a program doesn’t work – The purpose of testing is to test, not to prove anything. Follow a well-defined testing plan.
- Using testing to prototype a product – Prototyping belongs in the analysis and design phases. It gives users a glimpse of the end product. Once design is complete, discard the prototypes.
- Using testing to design performance – Performance goals must be defined before a project leaves the design phase. Testing validates performance against the specifications.
- Testing without a test plan – No plan means inefficient and ineffective testing.
- Testing without a specification – Testing ensures a system meets the specifications. Without a specification, you can’t test anything meaningful.
- Asking developers to test their own programs – Developers make poor testers. They lack testing training and have a conflict of interest. Testers need an unbiased mind.
- Testing without a goal – Without a clear goal, testing becomes aimless. You’ll never know when you’re done.
- Relying solely on unsupervised beta testing – Beta testers need defined goals, supervision, and leadership to provide useful feedback.
- Making design decisions during testing – Design decisions belong in the analysis and design phases, not after implementation.
The Role of Analysis and Design
Before any real testing can start, you need thorough analysis and design. When these are done right before implementation and testing, your chances of success go way up.
I once worked for a boss named Gary who ignored this rule. Gary wanted to implement a warehousing system for a client without a thorough specification. I objected. He didn’t care. His design process was to spend a couple of hours asking the client what they needed, then start coding. This led to a cycle: code something, show the client, make changes, repeat until the client gave up and called it acceptable. The project took far longer than necessary. It didn’t fully meet the client’s needs. It was riddled with bugs. It demanded enormous support during the first couple of years.
Marketing’s Role
Software testing shouldn’t be marketing’s job. But marketing should be involved in the analysis phase. A skilled analyst knows that marketing is a customer whose input during analysis is valuable. Any involvement beyond that stage can push the product in a different direction during testing and invalidate the test.
Specifications as Contracts
A specification is not just a document. It’s a contract. The goal is to build something that meets the specification. This is the most effective way to deliver software that meets customer expectations, assuming solid analysis and design.
Say you’re hired to create a new warehouse system. You do your analysis, design the system, and the customer approves it. Then the customer decides to add bar coding mid-project. You must stop, assess how it affects the project (at the customer’s cost), and submit a new cost estimate and delivery date.
Maintaining Standards
Standards matter. Testing measures the implementation against both specifications and standards. Standards might cover field validation methods, buffer overflow prevention, consistent screen design, and similar concerns.
Testing confirms that the implementation matches everything in the specifications and standards. It does not measure the product against customer expectations. That’s marketing’s job, and it should have been settled during analysis and design. If the specification meets customer expectations before implementation, the final product will meet those expectations. The specification represents the expectation.
Conclusion
Software testing is one of the most misunderstood parts of the development process. Many projects fail or underdeliver because of inadequate testing, poor analysis, and weak design. Avoid these pitfalls. Align your project with a clear specification. Stick to your standards. Your software project will meet customer expectations. A specification is not just a document. It’s a contract.
Takeaways: Software testing requires a plan, clear specifications, and trained professionals. Treat specifications as contracts. Marketing belongs in the analysis phase, not the testing phase. The goal of testing is to ensure the product matches the specifications and standards. Avoid the common pitfalls and maintain your standards throughout.