If you don’t yet know what an SQA job is, please read the part 1.
Features.
There are always some important aspects of a feature that MUST be tested and some that are optional or informal (e.g. “The feature can also work in that way but it’s not required by design”). Few thing must or can be known for a feature:
- Sotware requirements.
- Hardware requirements.
- Customer requirements
- Limitations.
Optionally for testing recommendations can also be taken:
- Development documents.
- Informal documents such as forum talks, e-mail listings, foreign experience and other sources not in the design notes.
- Improving previous tests not originally designed for this specific feature.
Ways to test a feature.
- Parameter testing, we already mentioned that can be completely automated.
- Negative parameter testing. Also completely automatable.
- White box testing. We know how the feature is supposed to work and its internals by observing the code itself. This testing method is suitable for QA engineers that can analyze the code of the developers and provide some possible break points.
- Black box testing. We don’t know what exactly the features is supposed to have as input and what output it provides. This method is suitable for proven software or device hackers that can find a way for penetrating the service or the system tested. Sometimes a small script sending random parameters to the feature can find a major bug or critical memory leak.
Implementation of the feature. (unit testing)
Every feature is a part of a system. Having the feature working in the system can sometimes unravel even more bugs. An Inter Process Communication bug or even wrong parameters between 2 features in a system can lead to another major bug missed while testing the feature alone. While the feature is not fully integrated in the whole system, we can’t be sure it works @ 100% so we make another tests. An example of this can be a system driver implementing the VLAN support. We know we can assing this feature per device physical port but never tried to set it on a virtual port, because virtual ports are different feature designed for routing protocols. If we only tested the feature on physical ports, we can’t be sure it works on virtual ports etc.
Dialog between systems.
The next step in feature testing is when we are happy how the system implements this new feature.
In most features, an interactive dialog between two systems is what actually the feature does. In the VLAN example I make often in this article, this would be the dialog between 2 ports on neighbouring systems:
2 systems
So, there will also be the additional tests in a system setup between 2 or more systems. Those tests will include the protocol acceptance between them, the right encapsulation of the packets, control sums and their decoding, the timings, delays and many others that can’t be tested if one system or the feature alone is under test.
Some black box testing can be done also. We can inject wrong dialog between the 2 systems and see how they behave. What time it takes to resume normal operations and possible security breaches if a dialog is broken and spoofed dialog is injected instead of the expected one.
Dialog with higher level features.
The same example as before. Imagine the newly implemented VLAN feature has to be tested to build an MSTP ring in a larger scale of 4 core switch units with one dozen more border/demarcation switch units outside the main ring. This example is one of the projects I work on often. The features we test are not as simple as the VLAN, but it takes big networks built in a lab and often new bugs pop up, so It really pays having a big system to test a basic feature. Test cases such as this last one takes lot of time to develop and the actual test can take also lot of time.
It is really important though.
Dialog with foreign or competitor systems. (compatibility)
One also very important test case, that also includes having much more expensive lab equipment. Imagine a ring with Cisco, 3Com, HP, Extreme networks and Telco systems core switches. All of them have implementation of Virtual LAN per port. You can make a simple test case for multi Vendor testing or even an MSTP ring, R-APS ring or MPLS mesh. It takes time to develop and additional knowledge of the competition’s equipment.
MOST if not ALL of the QA labs have such test cases. Your management team should be aware of the trends in the other brands and will probably have some units in the lab for compatibility tests.
