tag:blogger.com,1999:blog-6040134032896431563.post8785774555361474133..comments2012-12-13T21:36:19.190+00:00Comments on ObServant Tester: The need for confirmation – why checking against requirements is not going awayUnknownnoreply@blogger.comBlogger5125tag:blogger.com,1999:blog-6040134032896431563.post-10269967127812359992010-08-02T15:47:14.477+01:002010-08-02T15:47:14.477+01:00Thomas,
Absolutely. Tracing back to requirements...Thomas,<br /><br />Absolutely. Tracing back to requirements is only one way of demonstrating that the SUT is fit for purpose.<br /><br />StephenStephen Hillhttp://pedantictester.wordpress.comnoreply@blogger.comtag:blogger.com,1999:blog-6040134032896431563.post-28078707576343702302010-08-02T15:40:41.243+01:002010-08-02T15:40:41.243+01:00All, thanks for the comments.
Simon, the need to ...All, thanks for the comments.<br /><br />Simon, the need to test because of regulatory reasons or because the need to comply with certain standards is nothing that we can do anything against. <br />We can ask if it adds any value apart from helping marketing (which is important enough).<br /><br />James,<br />I'm in two minds about traceability and it usually comes down to the project. I've seen projects where developers simply forgot to add a function and only found out when someone asked about it. For this reason alone having a rudimentary check at the function level might be a good idea.<br />My case against traceability is that it is sold as something that adds value to the customer. I'd argue that, for small to medium projects, we deliver less value. The time and effort that we spend in traceability could've been spent in testing the application and finding out more information about it.<br /><br />I take your point about people not taking requirements seriously if we don't check against them. If, however the customer is involved in the development of the product/solution, what need is there for tracking requirements? Some food for thought.<br /><br />Stephen,<br />I'd argue that I can show good coverage of the system without tracing back to requirements. I'd even argue that I'd provide better value to the customer (internal to the PM, external to the paying client) if showing what the application really is about. Wouldn't that add more value than showing that we adhered to requirements that the customer might now feel weren't so good in the first place?Anonymoushttps://www.blogger.com/profile/03762884988997347021noreply@blogger.comtag:blogger.com,1999:blog-6040134032896431563.post-16900711694206252562010-08-02T15:10:52.556+01:002010-08-02T15:10:52.556+01:00Thomas,
Great article. I like to be able to trac...Thomas,<br /><br />Great article. I like to be able to trace my testing back to the original requirements in broad areas - for example I want to see which functional areas of the application have been exercised by particular tests.<br /><br />For me, the important thing is being realistic about what the audit trail is telling you - and what it is not telling you.<br /><br />The concept that you have highlighted where a single test condition is used as a basis for ticking a box against a 'multi-user' requirement, continues to concern me as I see evidence of it so often.<br /><br />As you and James have said, the fact you have provided evidence of traceability of coverage, for example, does not mean that you have done good testing.<br /><br />StephenAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-6040134032896431563.post-12159782695285872762010-08-02T14:48:20.497+01:002010-08-02T14:48:20.497+01:00I'd pretty much agree with you Thomas. Traceab...I'd pretty much agree with you Thomas. Traceability is important, but only in a limited sense, and its importance varies depending on the nature of the development and contract.<br /><br />As far as I'm concerned traceability qualifies as being necessary, but definitely not sufficient, for good testing. But that's based on the assumption that the requirements are reasonably good quality, and it doesn't take account of all the testing that's not based on the requirements. If the requirements are rubbish then who cares about traceability? There are bigger problems, and a neat trail from the requirements through to the testing won't help much.<br /><br />There are all sorts of weaselly subjective words in that paragraph, but that's because it all depends on the particular context of the company, product and project.<br /><br />As a test manager I like to have traceability just to provide an initial structure to the testing; so I can see that at least I'm covering the whole application, its functionality and its environment. I also want to be able to know what testing is relevant to which requirements. That's about the management though, not the real testing. As you say, Thomas, it doesn't mean you've covered the testing for a requirement effectively.<br /><br />Traceability is more about a basic level of accountability, but providing it shouldn't buy you a huge amount of credibility.<br /><br />It disappoints me when the debate about traceability becomes polarised. Some places take "testing against requirements" to ludicrous extremes. I've seen a "standard" that mandated that all testing should be traceable back to a requirement. That gave bad testers the perfect excuse to skimp testing when there were missing or poorly stated requirements.<br /><br />If you warn against the dangers of obsessing about traceability people can start to assume you're dismissing the whole concept and don't take requirements seriously. It's not a case of either "100% 2-way traceability" or "forget about traceability". Presenting the debate as a dichotomy is extremely unhelpful. It doesn't acknowledge the the interesting subtleties of requirements and testing.James Christiehttp://www.clarotesting.comnoreply@blogger.comtag:blogger.com,1999:blog-6040134032896431563.post-17669316511686467882010-08-02T13:59:04.764+01:002010-08-02T13:59:04.764+01:00GCF conformance testing of mobile handsets is anot...GCF conformance testing of mobile handsets is another that comes to mind - the intention with these is that manufacturers can say that handsets "behave" in a certain way under certain conditions. <br /><br /><i>It's not, of course, saying that the handsets are fault-free - but then I don't think they'll claim that - and is probably only a subset of the product manager's requirements.</i><br /><br />So, for potentially legal requirements a form of traceability (against a subset of the testing) is required in these cases.<br /><br />I thought I'd written previously about conformance testing problems - but see it's in my draft folder - time to get busy on that..Simon Morleyhttps://www.blogger.com/profile/10629592766073538811noreply@blogger.com