Per Erik Strandberg /cv /kurser /blog

Eller: Sessionsbaserad utforskande testning

Jag försöker få den manuella testningen på mitt nuvarande uppdrag att bli bättre. Jag samlar inspiration från rapid software testing, exploratory testing och session-based testing. Jag införde för ett halvår sedan mer uppstyrd rapportering och började föra statistik på hur mycket olika funktionella områden och vilka fysiska enheter som testats. Men det börjar bli dags att uppdatera metodiken.

Jag tänker mig att man inte bara samlar in en slags total-tid, utan försöker bryta ner den i tre bitar:

Utöver att man får in tiden bättre skulle jag vilja få en bättre effekt av själva testningen. Det tror jag man får av mer uppstyrd förberedelse. Jag skulle vilja att testarna (och utvecklarna som får ta på sig test-hatt med jämna mellanrum) går igenom tre faser i varje testsession:

Men produkten är ganska komplex och kräver att en bas för testningen finns med på något sätt. Jag tror på att låna idéer från Scenario based Exploratory Testing för att få in den testdokumentation vi redan har, men att uppmuntra testarna att inte bara följa den stig som specifikationen anger.

Jag vill även försöka samla in en känsla för hur det går. Jag hoppas kunna se trender som att det initialt är mer förvirring än senare i releasefasen och att det blir fler glada ansikten över tid. Jag lånade helt enkelt en massa ansikten till rapporten från Unicodes Emoticon block (se [1]) som ska väljas utifrån hur testningen kändes.

Referenser

Rapid Software Testing Class Materials

Here are some more pretty unstructured notes from Rapid Software Testing Class Materials, written by James Bach, Michael Bolton and Paul Holland. Available at satisfice: [2]

They say once that...

The Rapid Software Testing define coverage as "...how thoroughly you have examined the product with respect to some model...", where you cover the famous "San Francisco Depot, or SFDIPOT": Structure, Function, Data, Interfaces, Platform, Operations and/on Time. It is, in addition, also important to investigate Development aspects of the system: Supportability, Testability, Maintainability, Portability and Localizability.

Good products that are testable have: Controllability, Observability, Availability, Simplicity, Stability and information.

Quality Criterias are diverse (CRUCSS CPID) Capability, Reliability, Usability, Charisma, Security, Scalability, Compatibility, Performance, Installability, Development.

Exploratory Testing is an approach to testing that emphasizes the personal freedom and responsibility of each tester to continually optimize the value of his work by treating learning, test design, test execution as mutually supportive activities that run in parallel throughout the project.

"How do you invent the right tests, at the right time? Evolve them with an exploratory strategy".

The Rapid Software Testing approach seems to prefer testing where a tester follows ideas: "The skilled tester remains in control of the process" whereas test specifications are not well liked "Scripted procedures give the illusion of control over unskilled testers".

Two concepts I like are

If you like rapid software testing and exploratory testing you have to like wierd acronyms. The one for General Test Techniques is FDSFSCURA:

"Instead of thinking pass vs fail, consider thinking problem vs no problem"

Testing Patterns that are mentioned as examples of quick or careful tests:

Slide 108 lists "Common Problems with Test Documentation" - Haleluja moment: "What does Rapid Testing Look Like? Concise Documentation Minmizes Waste"

Consider Automatic Logging: How can you automate logging in your project?

To increase Accountability

"How do you effectively report your work? Learn to tell a compelling story that provokes the right questions."

Bach'arnas Session-Based Test Management

One of the first references in the wikipedia page on Session-based Testing (see [3]) is a link to Satisfice (see [4]) and Jonathan Bach and James Bach's Session-Based Test Management (see [5]), written by Jonathan Bach.

Bach first write a little about the background for the session based approach:

Testing in Sessions

Debriefing and Planning

Test Session Sheet

The test session reports are written with a semi-formal text format that makes me think of pubmed, gene and protein sequence formats. The reason for Bach to do this is not that you have to do it - it just facilitates automatic reporting. With some scripts they can get statistics easier.

The Test Session

One of my favourite quotes in his text is: "From a distance, exloratory testing can look like one big amorphous task...", so we'd like some reports from it, but reporting takes energy away from the actual testing, so it has to be lightweight. They use a break down into three areas. TBS:

Some of the interesting sections are:

They wrote a script to analyse the reports to create something that can be exported into excel with lots of traceability. This would also measure coverage and uncomplete testing.

Metrics

Some of the metrics mentioned are focused around predicting work effort. An example: Assume we have 4 testers that do 3 test sessions per day, 2 more release candidates to test in 3 weeks, and a product with 20 major areas:

BugFinders Training Video: Exploratory Testing Types

BugFinders Training Video: Exploratory Testing Types, Details of various types of Exploratory Testing, See youtube at [6], by Martin Mudge

A nice short video with the objectives to...

They explain that exploratory testing is a skill and mindset of tester and the "Concurrent test execution & design". Exploratory Testing is not: "Following your nose".

Their list of Varieties of Exploratory Testing contain 5 versions:

They give good examples of the above in the video, and a nice example where they compare testing to taking a trip to Australia. Should you follow the guide or randomly explore the first beach you see?

BugFinders Training Video: Session Based Exploratory Testing

BugFinders Training Video: Session Based Exploratory Testing, An Overview of Exploratory testing Techniques, See youtube at [7], by Martin Mudge

Another nice short video with the objectives to:

They define Session based exploratory testing as "define what to test, give it a time scale and test based on that."

This type of testing requires some session preparations. And testing should ideally follow the plan. (Test Objective, Duration, what to test, type of testing how much of the time should be spend exploring and how much on the charter, evidence recording, expected Risk Areas, and so on).

They have a nice example with that includes testing of a pogo-stick. Context is important. They use mand mapping as a tool for mapping the area to test, and they use the mind map to map the testing performed. Their tips are:

Tillhör Kategori Test
Se även Beräkna Testinsats och Riskbaserad Testning
Se också James Bach Open Lecture On Software Testing