Lets start out by looking at a most extreme example of discovering context.  Wilson Greatbatch was a scientist investigating a device to record heart rhythms, who accidentally used a massively overpowered resistor. Instead of recording, it pulsed with a slow regular rhythm. The invention became the pacemaker, replacing a giant fixed machine.  Harry Coover was a military research scientist, who tried using a new mystery compound in gun sights without success then later in aircraft cockpit canopies without success, because the substance quickly became a gooey mess.  It took a while before he also realised it bonded things together with an incredible strength. Of course, we now all know it as superglue!

At least with software and projects our contexts are more easily discoverable (but not always!). Can anyone think of other extreme examples of discovering context?

Hmh... wrong password, so I submit it again. And my password is stolen. Why?

I'm calling myself evil tester. For many people it sounds negative term but I think it describes my testing style very well. Evil tester is not evil for people but for software.He also knows how criminals could take benefit from software problems.

For example I've found lack of validation at redirection URL. Such issue is quite often at login because it is good idea to redirect user to previous page after succesful login. But if the url is not validated, evil person can redirect the user to some other page. That doesn't sound bad - right?

What if the destination page looks just like normal login page with text about unsuccesful login like above? How many noticed that URL had wrong domain? Would you have noticed it during normal login? Evil tester knows how to take benefit from the weakness. And tells that.

It doesn't have to be like that at all. Collaboration is matter of give and take. Showing an interest in each other's work is a necessary first step. Second is the realization that openness in a team is a must. Too often I hear that testers have limited access to unit tests. At the very least, one should be able to read all test code, to minimize overlap in test effort.

Code should be readable, even for a tester without a programming background, given a short walkthrough by the programmer. (If not, then it's probably not going to be easy to maintain the code at a later date.) Any tester worth his or her salt should be able to review whether the tests are adequate in terms of covering decision paths and boundary values. Quite likely, the tester can add value by suggesting improvements and additional test cases. If those were adopted into the project, to be run at every build of the system, it would strengthen the base of the so-called testing pyramid, and improve testing overall in the project.

In an ideal world, testers and developers make a habit of designing tests together, possibly pair programming from time to time. In my experience, testers often have to prove themselves to the team, before they are consulted early enough in the design of tests at the unit and integration levels.

Especially the integration (unit integration and system integration) levels are interesting for testers. Most business logic will be checkable at this level. It would be a shame if it were excluded from automated testing in the style of unit testing, just because it involves more than a single unit. Unit testing is just a label; you can use the tools for it at higher levels as well. Experience has shown that numerous defects are inserted precisely at this integration level, and that it is hard and time consuming to expose them by end-to-end testing or through a GUI. Even worse, such tests tend to be very brittle, and hard to maintain.

Now suppose you are a tester in a team that does not automate many tests at the integration level. My best advice is to discuss this in the team, and not to accept quick fixes. It's very frustrating for a tester to hear that a bug was already fixed, and no test was added to the project. That means that you would need an old version of the product to reproduce the defect, and that manual testing of the fixed application only proves that you cannot easily reproduce the aberrant behavior. It's much, much better for everyone's confidence in the solution if your can agree on something that is called 'defect TDD':
1. When a defect is found, tester and developer discuss what the most appropriate spot is for a test to expose the defect. Quite often, the defect can and should be caught at the unit / unit integration level.

2. The new test is written, and it is run to see that it fails.

3. The code is fixed and the test passes.

4. Refactor if possible, and re-run the tests.

Unlike regular TDD, this approach for defects comes with very little overhead. (Because the code is already complete, there is no need to write mocks for classes that are about to be written. As noted above, it's okay if the test involves more than one class.)

Once a team is accustomed these practices (sharing information about unit and integration tests, securing that a defect stays fixed, placing more and more tests at the integration level, etc), testers will find that they have more control over their situation. There should be no debug cycle near the end. In fact, it ought to become harder to find defects.

Surely, a situation may still arise where a large part of the functionality is declared "ready for test" in the last week of the iteration. The point is that such functionality will already have undergone a lot of testing. Also, the team will be more quality-aware, and may see testing more as a team responsibility. With the provision that no-one tests their own work, developers can certainly chip in to reduce the testing workload. Developers will be able to do some non-confirmatory testing. Especially if their quality awareness has reached a point where they take pride not just in their code but also in the unit and integration tests that come with that code.

As I said, collaboration is a matter of give and take. In the next part of this series, I will discuss ways in which we as testers can make life easier for developers and designers.

But most important thing is to find out if I can write blogs without JavaScript. I used Web Developer plugin for Firefox at testing. I had JavaScript enabled when I logged in, then I came to posting page, disabled JavaScript and reloaded whole page again. It gave me the text area for blog writing. I am adding manually tags to see if the text gets formatted. If this works, I can then write plain html to text editor, add links and image tags, then post the form.

After reading Vesna Leonard's blog post titled - 'You are NOT like any other QA or tester I've ever met', one statement stood out:

'You are NOT developing software…a mindless tool….you are developing an EXPERIENCE that lives and breathes with its users.'

The statement sums up the essence of the post and how as a tester, she cares for the stakeholders more than the process.

Unfortunately this forces me to put images to outside Eurostar Conference's site. I hope server won't die to the load.

Oh and about screenshot. Facebook is trying to fool me to submit Skype password to it. That is horrible requirement from security point of view. I've taught to my kids that they don't submit passwords to any external place. (And if the image does not appear below this, you can see it at https://kiva-mesta.net/~teme/2010-08-17-mit-vit.jpg .)

So the image should be above this. And I didn't try any security violation. ;) No onmouseover-tags, no broken images with error handling. If I were doing security testing to this site, I'd try those.

The Test Team and I recently headed out for a night on the town. As we haven't been out on a night out in a while, I thought I would do the honours of organising a well deserved night out and the make the most of the weather we were having at the time. As it happens, the place in mind that would have been perfect got called off due to bad weather. Not a total surprise if you're English or follow cricket!

Planning:

One of the main problems I had found with organising a night out is that some of the testers have more of a disposable income than others. Some people want to go to a fairly nice restaurant with pricey beers, whereas others are more reluctant and want to go to a cheap and cheerful pub with proper pub grub and reasonably prices Ales. Some people won't come if we go to the pricey place, and some people won't come if we go to the cheap place.

Equivalence Partitioning:

The difficulty in this was trying to strike a balance and ensure that enough people are pleased with the decision without annoying others too much. One method I used to try and find a happy medium was using Equivalence Partitioning. I placed the number of testers who wouldn't go to the really fancy or really cheap place and placed those in invalid partitions. I then placed everyone else into two valid partitions, about 50% towards the fancy end, and 50% in the cheaper end. I then made a scale of pubs from cheap to fancy and placed those underneath the partitions and picked a pub that was roughly in the middle. The majority of people were happy with the decision; a pub chain selling fancy burgers with both reasonably priced and expensive Ales!

On the night out/Testing in the Wild

A number of issues can crop up during the course of the evening, however.

Requirements issues:

The food or drink does not meet the specification, or does not match the user's expectations. For example, the menu states that the salad comes with olives, where in reality the salad only contains one olive. A regular diner would probably have not even noticed the amount of olives buried within the salad, and have probably already forgotten what the menu said as soon as they put it back into the menu holder.

Performance Issues:

There is usually a gripe about how the waiter/waitress isn't performing as expected. These never get logged or raised with management, so they're never aware of them, however testers readily express their discontent with other testers.
As the evening progresses, the restaurant becomes more and more busy, and the once attentive waiter/waitress is now rushed off their feet serving other customers.

Load Issues:

Someone always eats or drinks way too much *Ahem*

Poor management and leadership:

Unfortunately in this instance, as I was the organiser, I was also the leader. You can see above that I had done my best to decide on a place that suits most people, however, once you decide to leave that place it's a free for all. I don't think you can plan a night out too much and it should be up to the group to decide where to go. As it happens though, there's always someone who looks at you and wants you to make a decision.

Moans, gripes and jokes aside, it was actually a good night and everyone seemed to have a good time. We have a few more nights out planned in the coming weeks and months which should be a good laugh.

Usually we know what bugs should be fixed. But that's not always clear to other people at project. This problem appears usually with security issues. Project and product managers don't usually see how some security issue or risk can be so destructive, that they should be high priority issues. Most proof of concepts from security issues are not real exploits, but rather "Hello world"-types because creating good proof of concept just costs too much to be justified.  Instead of writing POC, one should know how to write story about problem. And that's what I'm now days usually doing when I submit security bug.

Every story has some common components. So must have every bug report which is telling the story. They should be:

• Technical details at the beginning. The problem must be found easily by developer.
• Actors. I'm always giving some kind of description who they are. Almost always there is one villain and victim, but villain can be also some innocent person, who just tries to get her work done more quickly.
• Motivation. The villain has to have good motivation to exploit the bug.
• Result. It should hurt someone, kill the business or reveal sensitive data.

For example:

Many PHP-applications has weak session handling. If the attacker can give own value for PHPSESSID, it is accepted and used by the application. Those can happen with cross site scripting bug or local access to the computer and tempering directly the cookie storage of Firefox. The tested application - women's social site about home violence WSSAHV.COM - has weak session handling, but the tester didn't find XSS. So report with story would be:

(Technical details before the story. Also describing how the session should work.)

There is family and members are Hilma (wife) and Onni (husband). Onni is jealous. At some point he finds out that Hilma is using site WSSAHV.COM. Of course he suspects that she's cheating or telling lies about him there. He directly doesn't want to attack against her and force to give the password so he starts hacking and notices the weak session handling.

He has the administrator rights to their shared Windows desktop, he has also access to Hilma's personal files. He goes to Firefox profile files, opens cookies.sqllite with sqllite. There he inserts new cookie for WSSAHV.COM. Domain is 'WSSAHV.COM', path '/', name is 'PHPSESSID ' and value is 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', expiration is set to July 15th, 2020.

Next time when Hilma logs in to WSSAHV.COM, Onni has access to her profile from different computer. He just sets PHPSESSID at his machine to 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'. Onni can alter the profile, read the private message, see the hidden areas where Hilma might have access and so on. This violates the privacy and meaning of the site. It also places the users to danger.

1. Virtualization. It is already there, but I think it can be used to much greater effect, now that the number-crunching power of computers does exceed what is needed in many cases, and extra gigabytes are dirt cheap. Soon, testers will use virtual machines that emulate the target platform in such a fashion, that you can also take a step back after a defect is encountered.
Then, you just hand over the virtual machine to a colleague, who can reproduce the defect instantly.  

This is not science fiction, it is already there. Virtualization and emulation tools already exist, they just need to be perfected a bit.

   Image size should be reduced: right now, an image of a virtual machine can easily be 4+ GB.

   High-performance corporate networks and Internet connection would make a big difference. People will want to send their virtual machines over a network.

2. Soft skills / consultancy skills. Demand for solitary minds in testing is dwindling. Today's and tomorrow's projects require testers to communicate well with colleagues and business people alike. Recruitment of testers (and selection of who stays on companies that reduce their staff) will focus on soft skills that are hard to train, rather than experience and knowledge that can be taught.

3. Career paths. Employers of testers must provide career opportunities and growth options that fit the new scheme. This includes differentiation for affinity towards business or technology. There must be room for individuals with aptitude towards consultancy and leadership. Also, technical excellence (e.g. tool champions) must be rewarded appropriately.
For some really technical testing jobs, it will effectively be a requirement that the 'tester' has experience as a developer. Thus, it must be a step up - not down - to choose such a career path.

4. Solid developer testing (as in the Agile testing pyramid) will gradually become the norm in the industry, allowing testers to do what they are good at: non-confirmatory testing.
Again, this is nothing new, and it does not require any new technology. However, for the standards to be raised across the field (and for agile development and testing to become everyone's frame of reference), that does take time. In some countries, universities have been teaching students XP practices for years.

5. Test frameworks become integrated with middleware solutions, to the point that they become the deciding feature for choosing such software. For example, imagine a middleware suite for modeling a business process flow and a big information system around it. All the big names in the industry have their own suite, and they all have loads of features. It may be deceptively easy to develop a large workflow application with those packages, and very hard to test the result. Tool suites that really give good support for testing the application, will have a decisive competitive advantage, as those would reduce the total development cost.

6. In-memory databases are gradually becoming accepted as a stable solution that can yield great cost savings. Back in 2008, I encountered no real differences (other than the speed) when I tested an application built using an in-memory database system. The main difference for testing was that we could use loads of DbUnit tests in our CI build, without any need to place the Db tests in a nightly build scheme.
Here, it is just the human factor that slows down adoption of a technologically sound solution. If the main performance bottleneck in many of our information systems is the response time of a costly database system (or cluster of database servers), then it stands to reason that more companies will take the plunge and replace them with a lean and mean in-memory system.

Of course, this is just my own opinion. Please feel free to comment on the trends I outlined above, and add your own.

In many cases, you have to be aware that you are not just testing the application, but by default also the process that led to it. I have been in projects where I started testing in late stages of development, simply because I wasn't called in earlier. Even in the presence of a fair number of unit tests, I would find many defects if there was something amiss with the process. For example, if there was little or no communication with the customer that had commissioned the product, then I would find many defects that were due to assumptions made by the development team. If requirements had undergone many changes, then it is likely that those changes caused many tests that were built early on, to fail. I have been on a team where I saw that the total number of unit tests in the project had suddenly gone down instead of up. When I politely asked my colleague about it, he informed me that he had been pressed for time to incorporate the scope changes, and that he had outcommented the old tests. He wasn't slacking off; in fact I knew him to be a really hard-working individual who put in a lot of unpaid overtime. He had simply been overruled, and pressed into releasing the software with a mediocre code coverage. What was defective about the process, is that the team was not allowed time to do the rework in the next iteration. Until we managed to convince our customer of the importance, that is.

I want to encourage any testers to analyze the causes behind the defects that they encounter, and look for a pattern. If you then discuss the causes of the problems with your colleagues, then as a team you should be able to improve your process over time. In so doing, it should become harder to find defects. Agile testing is like that: more focused on preventing rather than detecting defects.

Ard Kramer

Test consultant EclipseIT

PS Luckily I found out that you can find out what it is on the English Wikipedia (http://en.wikipedia.org/wiki/Egg_of_columbus ) and I am convinced that testers have a lot of those moments so I propose to make the expression one of the standard expressions of a tester, "the egg of C". almost as good as "42" or  " RTFM

Every great dream begins with a dreamer. Always remember, you have within you the strength, the patience, and the passion to reach for the stars to change the world. -Harriet Tubman

Around January, I saw the advertisement for 'Call for papers for EuroSTAR 2010'. The theme 'Sharing the Passion' goes so well with 'Weekend Testing' . I submitted my abstract and my topic was - 'Weekend Testing - Skilled Software Testing Unleashed' [http://bit.ly/bypNlO]. My dream of presenting in a conference outside India got a big boost when my abstract was accepted.

Fully prepared to attend the conference and enjoy the wonderful city of Copenhagen, I was very excited. Though attending a conference outside India proves to cost more on the monetary aspect, I was fully aware of the 'Cost Vs Value'. I'm sure that the value I would get out of the conference and the experience would definitely outweigh the costs of attending the conference.

As Lisa Nichols mentions in the chapter - 'The Secret To The World' in 'The Secret' book, "Everything that you want-all the joy, love, abundance, prosperity, bliss-it's there, ready for you to grab a hold of it. And you've got to get hungry for it. You've got to be intentional. And when you become intentional and on fire for what you want, the Universe will deliver every single thing that you've been wanting."

Within few weeks, my prayers were answered. EuroSTAR announced about the BlogSTAR competition and here I am blogging as one of the BlogSTAR contestants.

I'll be posting my thoughts, experiences and learning here. Please feel free to comment, argue, question, discuss, agree, disagree, correct me.

You can email me at AJAY184F[AT]GMAIL[DOT]COM

Skype: ajay184f

Twitter: ajay184f

Click here to vote for your favourite TeamSTAR entry

Copy+paste from other application - I tried it already. Image seems to be broken. That is not surprising me.

Second attempt would be to hijack the traffic and add the images to HTTP-request. I would first try with Tamper Data and if that doesn't work, I'd hack small perl script which would send plain html to the server.

Third attempt would be with FireBug. It seems to have good possibilities to add elements to this iframe. I wonder if it would be submitted to server. At the same time I'd wonder if I could also violate the security of blog-system.

Forth would be feature request which also sounds good idea (and it'd be even legal one).

How would you add the image to your post?

This is the first time I've tested something of this size and I would say I've tested 25% of the application so far, and found 35 defects. I know that it all depends on the context of the application, and some would argue that because this is a high profile application used by external clients, that this is probably not enough. I've found a couple of hum-dingers which I'm actually quite proud of, but the majority of my findings are small little niggily things, like basic layout problems and missing resources.

A question I asked a colleague when I was starting to feel a bit overwhelmed was "Would you expect to find this many bugs in a feature complete build?" The answer I got back was that it depends entirely on the developer. Some developers do unit test their code quite thoroughly and wouldn't give you a build unless they were absolutely sure that you weren't going to find anything wrong with it. Others will give you a build because they've put all the necessary code in the software for it to work, and, as far as they can tell, it should work.

What would you expect from a feature complete build? Not a final build, but a feature complete build. To me, a feature complete build should be treated exactly the same by the developer as a final build but should come with an expectation that, yes, there maybe one or two little niggles here and there. If a developer has given you a feature complete build, you would still have expected them to have carried out unit tests on the majority of the functionality. When a developer deploys a feature complete or final build, to me, they are saying "This has gotten everything the users asked for and I'm happy enough with it that, if they want it deployed now, it'd work within minimal findings"

One would argue that my assumption is wrong, and that a feature complete build is a build that has everything the users asked for, but has not been tested, either unitly (that's a word I use all the time starting just now), or by a tester. This is compared to a final build which has all the required functionality, has been unit tested, and some testing by the tester has been performed. How do you see it? Let me know in a comment below.

I have about 12 years of experience in software testing and I have also about a decade of programming experience, part of those being concurrent. Agile at heart, I see the two disciplines as different but non-exclusive. I have contributed to several conferences, written a few articles, and co-authored a book. Passionate about sharing my experiences, I really want to be at EuroSTAR this year.

At EuroSTAR in 2008, I was selected as a speaker. During that event, my colleague Anko Tijman and myself also launched our book about agile testing (Dutch title: 'Testen2.0 - de praktijk van agile testen'). Work is still underway to produce an updated version in English. Of course, I will blog about my progress there, and give a few sneak previews.

Shortly after EuroSTAR2008, I made a blog post based on an insight that was part of my presentation. (It can still be viewed at http://testing2point0.blogspot.com ) The point that I tried to make and clarify, is that projects using modern architecture or business process middleware naturally gravitate towards an agile style of project management. Also, 'agile testing' may very well require two very different approaches. At one level, testing is highly business oriented, whereas on another level, there is also testing that has to be performed in a rather technical fashion. In my experience, this is best implemented by having more than one tester on a project team, as the tasks require vastly different skill sets.

In the next few days I hope to write about various topics, all relating to (agile) testing in one way or another. I will post a list of some of those topics (my product backlog) as well, and I am open for comments on what questions you would like me to answer. I have plenty of work ahead of me that I can write about. I'm preparing for several conferences, and next Thursday I'm attending an open space event. I will definitely write about my experiences before and during the Agile Testing Days in Berlin. Over there, I have a presentation and a full day tutorial together with my colleague. The presentation materials for those two are still far from being finished, and it's only 5 weeks till the event.

Read More

So you might have heard about the new ISO/IEC 29119 Software Testing standard that is currently under development? You may have even seen a presentation on it at EuroSTAR, or read about it in a previous EuroSTAR newsletter. In this article, we give you an in-depth look at this new and exciting standard, and keep you up to date on the very latest developments coming out of the working group responsible for developing the standard.

What is ISO/IEC 29119?

ISO/IEC 29119 Software Testing is a new international standard that is aimed at supporting all elements of the software testing lifecycle, throughout the development and maintenance of any software intensive system. This standard will essentially provide a body of knowledge on software testing, that has been developed by software testers, for software testers.

The standard is under development by ISO/IEC JTC1/SC7 Working Group 26, Software Testing. Eighteen different nations are represented on the working group, and more are joining each year., The national that are currently represented are Australia, Canada, China, Columbia, Denmark, Finland, France, Germany, Hong Kong, India, Japan, Poland, Russian Federation, South Africa, South Korea, Spain, United Kingdom and the USA. Thus, when it is released in 2012, ISO/IEC 29119 will be the truly international standard for software testing.

What does the standard cover?

We are pleased to announce that ISO/IEC 29119 is now comprised of five parts, as follows:

• Part 1: Concepts & Definitions 
• Part 2: Test Process
• Part 3: Test Documentation
• Part 4: Test Techniques
• Part 5: Software Testing Process Assessment

Stop press! If you had previously heard that the standard was comprised of four parts, you were absolutely correct! Part 5 was approved as a new work item for our working group just days ago, so keep an eye out for information on this exciting new addition to our standard!

More information on each of these parts can be found in the section below. One of the most important elements of this standard is that it prescribes a risk-based approach to testing, from risk-based test planning and the definition of risk-based test strategies, to risk-based prioritisation during test case design and execution.

There are three base standards that will be subsumed by this standard. They are IEEE 829 (a well-known test documentation standard), BS 7925-1/2 (a well-known component testing standard and vocabulary that you may have learnt about in your ISEB or ISTQB courses) and IEEE 1008 (a much lesser known unit testing standard). The following diagram illustrates the relationship between the first four parts of ISO/IEC 29119 and their relationship to the standards they will be subsuming.

What's within the five parts of ISO/IEC 29119?

The planned contents of each of the five parts are as follows.

Part 1: Concepts and Definitions

• Introduction to testing
• Software testing within an organisational and project context
• Risk-based testing
• Test phases (i.e. unit, integration, system, acceptance) and types of testing (e.g. static, dynamic, non-functional)
• Testing within various software development lifecycle models (e.g. agile, evolutionary, sequential)
• Roles and responsibilities in testing
• Metrics and measurement

Part 2: Test Process (illustrated below)

• Organisational test process
• Test management processes
• Static testing processes
• Dynamic testing processes

Each of these processes are described using detailed text-based process, activity and task definitions, and these are supported by diagrams that illustrate each process (for example, the following diagram illustrates the activities that exist within the dynamic test process).  

Part 3: Test Documentation

• Organisational
  • Test policy
  • Test strategy
  • Project
    • Project test plan
    • Test completion report
    • Dynamic Testing
      • Test specification
      • Test results
      • Incident reports
      • Test environment report
      • Test status report
      • Test completion report
      • Static Testing
        • Review Checklist
        • Static Analysis Rules
        • Action Item List
        • Static Test Report
        • Appendices, including examples of all documents

All test documents that are described in part 3 are produced directly from the test process that is defined in part 2.

Part 4: Test Techniques

• Static testing techniques
  • Inspections, walkthroughs, etc.
  • Dynamic testing techniques
    • Black-box, white-box, etc.
    • Non-functional testing techniques
      • Security, performance, usability, etc.
      • Test measurement approaches (e.g. coverage)
      • Appendices, including examples of all techniques

Part 5: Software Testing Process Assessment (new!)

• This (very recent!) addition to ISO/IEC 29119 will provide a framework for guiding organisations and individuals in the assessment of their testing process against ISO/IEC 29119-2 Test Process, which will allow them to identify if and how improvements to their processes can be made.

What's different about this standard?

There are a number of important things missing from existing software testing standards that will be covered by ISO/IEC 29119, including:

• risk-based software testing
• organisational processes and templates (IEEE 829 only covers project test management and the dynamic testing process)
• common system testing and acceptance testing techniques, such as Use Case Testing (BS 7925 only covers unit testing)
• non-functional techniques (e.g. Security Testing, Performance Testing)
• static testing techniques such as reviews and inspections (these are not covered by any existing ISO/IEC standards)
• vocabulary for the software testing life cycle (BS 7925-1 only covers unit testing)

Who does the standard apply to?

This standard is aimed at providing guidance on software testing for any software development or maintenance project. It is aimed at supporting all industry domains, including regulated (e.g. financial, safety critical) and unregulated, all project life cycles including agile, iterative and waterfall, and all system types from information to embedded systems. Perhaps the more important questions to consider are "when wouldn't you select and use testing techniques" or "when wouldn't you plan your testing?".

That being said, as testers we are well aware that this standard will require testing (i.e. trialling) on a variety of different types of projects all around the world. Trials are currently underway, and they are proving to be valuable feedback mechanisms for our team.

When will the standard be available?

Parts 1 to 4 of the standard are due for release in 2012, as shown in the timeline below. Since part 5 was approved only days ago, it will be ready for release at around 2014 to 2015 (part 5 is not shown below). However, drafts of parts 1, 2 and 3 are already available and are currently being reviewed by experts worldwide. A draft of part 4 should be ready for review by the end of this year. 

 Can I get involved?

If you are an expert in software testing and you wish to provide your feedback on ISO/IEC 29119 to our team (including trialling the standard in your workplace), please contact the national standards body in your country to request involvement in their software testing standardisation activities (visit http://www.iso.org/iso/about/iso_members.htm to find out the contact details of your national standards body).

You might also like to consider joining ISO/IEC JTC1/SC7 Working Group 26, the working group responsible for developing the standard. This requires a lot of commitment, from attendance to two separate weeks of WG26 meetings per year at various locations around the world, to reviewing drafts and producing materials for the standard throughout the year. ISO do not provide funding for working group members, although some national standards bodies do provide nominal amounts of funding for attendance (contact your national standards body for more details).

Where do I go for more information?

For more information, please feel free to contact either myself (taflinem@kjross.com.au) or the convenor of our working group, Dr. Stuart Reid (sreid@testing-solutions.com). For all the latest news, visit the website of our standard at http://softwaretestingstandard.org.

Most projects apply some kind of (implicit) risk-based testing. We all have to balance between product quality and tight deadlines. Risk-based testing is the basis of almost every testing activity. Of course risk-based testing should be driven by business objectives. Testing is not the risk owner, but the products' stakeholders are. It is our job to inform the stakeholders about risk-based decisions and provide visibility on product risk status. Risk-based testing starts by doing risk identification and analysis in close co-operation with stakeholders. It also addressed the mitigation approach regarding the identified product risks.

From many practical experiences in various domains, Erik shares 10 essential lessons learned regarding risk-based testing; 10 things to remember.

1. Start risk-analysis by doing a proper stakeholder analysis. Since stakeholders provide the essential information for the identification and analysis of risks, having the right set of stakeholders is essential. In Utopia a thorough stakeholder analysis has already taken place during requirements phase. Both stakeholders from a business perspective and from a technical perspective (e.g. architect, lead engineer) are required. Remember, a forgotten stakeholder implies forgotten risks.
2. State the product risks in a business language. Communication is vital to a successful project. Product risks should be stated in such a way that they are understood by the business. It should be clear to them what it means if a risk becomes apparent. Only product risks where all understand what the impact is, in case of a failure, will get focus in communication
3. Recognize that impact and likelihood are different. Some product risks analysis techniques calculate the level of risk by multiplying impact by likelihood and from then on just the resulting calculated risk level is used. An extremely high impact risk (e.g. safety) with a low likelihood may then not get any or too little attention. Impact usually relates to business factors and business risks, likelihood relates to technical factors and technical risks. These types of risks are by nature very different and should also have different mitigation approach.
4. Visualize the results of the product risk analysis. A picture is worth more than a thousand words. Presenting the results in a diagram is usually much more clear to all stakeholders than a table (excel sheet) with many numbers.  The latter becomes unreadable very easily and some loose themselves in a number based discussion.
5. Consider both functional and non-functional risks.  Like with requirements specification some "forget" the non-functional product risks. However, in practice the non-functional quality attributes, such as performance, reliability and usability, often make the difference. Beware not to go overboard and lose yourself in long and detailed non-functional list of attributes that nobody really understands. Only discuss a limited set of non-functional attributes that could be off importance, and that you are capable of testing using test design techniques.
6. Define a differentiated risk-based test approach. Product risks that are more critical than others should be tested differently, with more coverage, stricter exit criteria etc. A tester testing an item related to critical product risk should act differently than testing a less critical item. This differentiated risk-based test approach should be clearly defined upfront to allow for effective usage of test resources.
7. Report against the identified product risks.  Many do a product risk analysis but test reports are again defect based. Stakeholders tell us which product risks are important and should be mitigated before being to release the system. A test report should provide this information and support the release decision. In practice, defect based reports are often not the most usable for business stakeholders. It is recommended to define product risk coverage and product risks mitigated as completion criteria.
8. Choose the product risk analysis method that meets your needs. Many methods on product risk analyses are not light weight and extremely thorough. This may fit when doing testing in a V-model environment for a safety critical system. When doing testing in an agile context it is still important to make choices. However, the product-risks analysis should be light weight and very focused. A simple brainstorm may suffice at the beginning of an increment. In general, don't make it more difficult than necessary.
9. Revisit the product risks analysis on a regular basis. The product risk identification and analysis are based on stakeholders' perceptions and expectations. These will change over time. Early testing will reveal some new risks while mitigating others. Changing requirements usually means changing product risks. It pays to revisit the product risk analysis on a periodic basis, at least at every project milestone.
10. Establish clear risk ownership and responsibilities. In many organizations testers' identify and analyze the risks. This is wrong; testers are not the risk owners!! Our responsibility is 'only' to facilitate the risk analysis process and inform our stakeholders on the status of the product risks. When stakeholders are asked to identify product risks and thereby indicate what to test and what no to test, they suddenly become aware that they are the deciding factor. If they do it wrong (e.g. miss a product risk), they are to blame and not the tester. This often leads to stakeholders' resistance: "It was so easy when the tester took the decision for us."