The report below is from Zeger van Hese who attended
and spoke at EuroSTAR 2009 in Stockholm. His blog can be viewed at
http://testsidestory.wordpress.com/
I absolutely *love* Stockholm in wintertime.
Pepparkakor, glögg, gravad lax... and Eurostar
too. People keep telling me that I would probably love
it even more in summertime, but I'll always associate
those dark days with Eurostar. I presented my first Eurostar
track there in 2007 - nothing but good memories - and I was
selected this year as well. The Eurostar
line-up is always pretty impressive, so it can
be both intimidating and exciting to be a part of that.
It's just a matter of keeping the intimidation level below the
excitement level, I guess. As a boyscout, good old Baden Powell
always told me to "be prepared". Now sometimes I wouldn't
recognize a life lesson if it punched me in the face, but here's
one that I did remember. So I found myself writing a paper and
assembling a presentation during those hot holiday nights in
Southwestern France. You just gotta love those early deadlines!
November 29
After an uneventful flight from Brussels to Arlanda, set foot on
Swedish soil. Met up with fellow Belgian Mieke Gevers, a member
of this year's program
committee and in charge of the track chairs as
well. I helped her carry some excess bagagge that turned
out to contain presents for the trackchairs - you can't go
wrong with Belgian chocolates and "jenever". We took the Arlanda express (easy and
quick) to Stockholm C and a cab to the Rica Talk
hotel.
November 30 - Tutorial day
On monday I attended a full-day tutorial by Michael
Bolton called "Exploratory Testing Masterclass" (slides
available here). Two
years ago I attended his tutorial on Rapid Software Testing, which
I found very valuable. Michael Bolton is an engaging
speaker and teacher who invites you to think, rather than just sit
and absorb theoretical matter. There were lots of exercises,
including one on factoring (identifying dimensions of interest in a
product). We were asked to identify all dimensions of a wineglass
that may be relevant to testing it, using the "San Francisco
Depot" - heuristic (Structure, Functions, Data, Platform,
Operations, Time) - not new to me but always worth repeating. A lot
of mnemonic wizardry to be found here. What about that handy
mnemonic for oracles - HICCUPP/F
(History, Image, Comparable product, Claims, User expectation,
Product, Purpose, Familiar problems) - never again say that
you don't know why something should be considered a bug.
Care to take a ride on that test reporting heuristic called MCOASTER? Well I'll see
your CRUSSPIC STMPL, and raise it with a FCC CUTS
VIDS (Mike Kelly's application touring heuristic).
Mnemomania!
Of course, there were plenty of other impressions that kept
lingering for a while.
- A quote by Jerry Weinberg: "A tester is someone who knows
things can be different" - true.
- "If it ain't exploratory, it's avoidatory" - made me
laugh.
- "A good tester doesn't just ask "Pass or Fail?". A good
tester asks "Is there a problem here?".
- CHECks are CHange detECtors, testing is exploring.
- A complete debunking of some boundary value analysis
truisms: it is generally accepted that the behaviour at
boundaries is more likely to show erratic behaviour, but how do we
know these boundaries? The actual boundaries in a system may not be
the ones we are told about. That's why we must explore.
- Testing is "storytelling" - I liked that take on
testing:
"You must tell a story about the product, about how it
failed, and how it might fail - in ways that matter to your
various clients. But you must also also tell a story about
testing, how you configured, operated and observed it
- about what you haven't tested, yet... or won't test, at all
- and about why what you did was good enough."
The end of the session was foreseen at 5 PM. The discussions
kept going on until 5.45 PM. I think that says it
all. Later that evening, an international amalgam of
testers set out to explore the possibilities of finding food in
Gamla Stan. Eventually we found an Indian
restaurant using that good old I.NEWTON heuristic (Indian,
Nearby, Edible, Welcoming, Tasty, Open, Not-too-expensive).
The end of a great day. Had some nice conversations with
Rikard Edgren, Tone Molyneux, Ray Arell and John
Watkins (my trackchair) as well.
December 1
The second day started with a tutorial as well, be it a half-day
one: Managing Exploratory Testing by
Jonathan Kohl. Of course there were a lot of
similarities with the first tutorial, but this was more
of a hands-on session, where we could put Michael
Bolton's concepts from the day before into practise. There was some
theory about coverage models - SF Depot anyone? We ended up
describing a whole bunch of characteristics of a table that we had
never associated with an ordinary table before. Practical and
fun. Certainly an eye-opener.
At that point I was still trying to get a hold of the person I
was supposed to trackchair on wednesday. Originally I would be
trackchairing my colleague Wim De Mey's track about regression
testing in a migration project, but Wim had to cancel his
presentation at the very last moment because of unfortunate
familial circumstances. A replacement was found in the
person of Mika Katara, from Finland - but no sign of him,
yet. Oh well, time for a quick lunch, a tour of the expo and the
actual kick-off of the conference. Dorothy
Graham opened the 17th Eurostar conference in style. She
introduced the program committee (Tone and Mieke made
sure Isabel Evans was also
represented by carrying an air-filled balloon with a face
drawn on it - I'm not sure if Isabel would be too happy with
the analogy 
) and set the scene for the first keynote
speaker.
Lee Copeland started this very first talk of
the conference about nine of the most important innovations in
software testing: the context-driven school, test-first
development, really good books, open source tools, session-based
test management, testing workshops, freedom of the press,
virtualization and "testing in the cloud". Strange that he sees the
context-driven school as an innovation - as far as I know it was
founded in 1999; the first book that explicitly named
it was already published in 2001. I agree with the
freedom of the press thing. Testing blogs are appearing everywhere
(guilty, your honour), twitter is on the rise. Lee is
apparently not a fan of twitter. Neither was I - I always
thought of it as encouraging the spreading of triviality,
but I'm actually starting to come back from that.
I noticed that a lot of people within the testing community
are using it to share their ideas, give advice or call for help.
And it gives a great deal of extra coverage to an event like this
(see twitter.com/esconfs), so maybe I'll give it
a try. Later.
The rest of the afternoon consisted of a series of short
20-minute tracks, which is mostly just enough to launch some
provoking ideas, but not really ideal for a lot of content.
Johan Jonasson talked about how he managed to save
a project with the introduction of a structured exploratory
testing approach. This track would have benefited from a 45 minute
timeslot - there was no time to go into detail, which I found
a pity. Next up was Julian
Harty, who explained the concept of
"trinity testing": short session of around 90 minutes per
feature, where the feature owner, the developer and the test
engineer work interactively through the software to share knowledge
and ideas. Pretty interesting, since I also found out later
that "the trinity test" was also the
name of the very first nuclear test ever conducted,
marking the very start of the nuclear
age. Julian is probably aware of this - I didn't hear him
mentioning it, though.
Geoff Thompson then talked about reporting
- "If only we could make them listen!". Well actually, it's
more the communicator's job to make sure he gets heard. It was a
great talk - he was able to slip in the Challenger disaster and the
Heathrow terminal 5 debacle as examples of how important messages
were apparently not deemed important enough, with horrendous
results. Knowing your recipients is key, and knowing what
information they want as well. Noteworthy: a lot of people are
color-blind. If you absolutely want to make sure that everyone
understands your reports, shouldn't you avoid the reds and
greens?
Besides being a sapient testing
evangelist, Michael Bolton is also a human
quote machine. He did this cross between stand-up routine
and political televangelism called "Burning Issues of the
Day" (available here). A lot of
wisecracks and eye-openers, the funniest moment at Eurostar
for me. He was even able to win a bet by slipping in a quote
about agilists and sex:
"The agilistas did not discover pairing or test-first
programming. They're like teenagers who've just discovered sex. It
IS great, but calm down".
The last speaker of the day was the same as the first one.
Jonathan Kohl talked about how our urge to be
"Agile" can distract us from our mission to deliver software
that our customers value, while supporting our team. Agile can
distract from skill development too. The term "Agile" has
become big business, and lost a great deal of it's significance. So
let's stop worrying about whether what we do is "agile" or not, and
go back to calling it "software development". As far as I'm
concerned, he hit the nail on head. I wouldn't have minded him
talking about this a little longer.
The day ended with drinks in the expo and my attempt at playing
a memory game at one of the stands. I kept failing epically. While
I was trying to get asleep I found the ideal excuse: my head was
already full of things to remember - no room for these trivial
button sequences.
December 2
Right before the first keynote of the day I finally met Mika,
whom I was supposed to be trackchairing in the afternoon. He was
invited as a backup speaker on friday to speak on
wednesday, was able to make it, but had to leave
immediately after his talk. A true case of hit-and-run
guerilla presenting at Eurostar! Naomi Karten then
delivered an interesting keynote about "changing how you manage and
communicate change". Her talk was built around the Satir change model. There's an initial
status quo, then a foreign change-inducing element causing a 'POW',
then chaos, after that an adjustment and in
the end a new status quo. When people are confronted with
change, they are experiencing a loss of control, and they
often react to that in an emotional way. Important:
listen, be empathic, regularly communicate the status of the
change, even when there is nothing to report. She also used a quote
that I well certainly use myself when feeling
cornered:
"Hofstadter's Law: It always take longer than
you expect, even when you take into account Hofstadter's
Law"
By then it was time to pay a visit to the Test Lab that was set up by
James Lyndsay
and Bart Knaack. It was a Eurostar first, and
I am actually wondering now why it took so long to have some
actual "testing" going on at a testing conference. The software
they were running was Open EMR, an open source patient management
and appointment book system. What made it even more interesting for
me is that I have been testing and working with a similar
(not open source, though) system for a long time, so I more or less
know what to expect (or what actual users of the software would
expect). I paired up with Rikard for a while and found a whole
bunch of issues by merely touring the application
- we noted them for later reference. It is always nice to pair
with fellow testers to see what they focus on, and what their
reasoning is. The state of the software under test was
something else. It showed some pretty alarming behaviour, and it
was far from intuitive or user-friendly.
By then it was time for Eurostar veteran Erik
Boelen, speaking at Eurostar for the fifth time
already. I've known Erik for some time now, and his talks are
always entertaining and relaxing in a way. "The power of risk"
was his view on how to use a risk-based test strategy that
"makes people talk", like Läkerol. His main message was (apart
from the implicit one that testing can be fun *and* will rule the
world) that they defined all the risks and used them
as entry paths for exploratory testing. For the highest
and medium risks they documented their test cases, and for low
risks they just reported the results.
After lunch I introduced Mika Katanen (from the
university of Tampere in Finland) and his talk about Automatic GUI
test generation for smartphone applications. I am totally new
to model-based testing and I was impressed with the brief demo he
showed. His track went well, and there were a lot of people
approaching him for a chat at the end. I do hope that he was
able to catch his plane on time. Parallel with this track, Shrini Kulkarni
held his talk about software metrics which I was unable to
attend. People said it was good - I hope I will be
able to see him speak some place else in the future.
Remembering the memory game disaster from the day before, I
decided to unfocus for a while - my mind was getting stuck again. I
teamed up with some CTG colleagues plus a wildcard named
Tom and enrolled ourselves for the quiz that was supposed
to take place in the evening. We aptly named ourselves
"The Handsome Oracles", but it wasn't meant to be. The quiz
was canceled later on, so we weren't able to put the money where
our mouth was. We also worked out some testing limericks for
the limerick competition - we didn't win. I thought they
were good, but that's probably just another example of parents
not recognizing the ugliness of their own babies. There's a good
joke and an interesting analogy about that here. Gitte
Ottosen ended the day with a talk about combining
agile and maturity models which was chosen best presentation
last year in The Hague. I had the impression she was a little
nervous - which is completely understandable. I was telling to
myself that delivering a keynote for a full auditorium like
that sure looked like a daunting task - until I suddenly
realised that I would be standing in that same room
tomorrow. My unfocused mind started wandering off.
While the temperatures were taking a dive, the Handsome
Oracles went into town for dinner. I returned a bit earlier
than the rest to rehearse my talk and to get a good
night's sleep while the (by then just plain) Oracles went
barhopping. Haha! Life's good, but not fair at all.
December 3
The last day of the conference, and people started looking
weary. Ray Arell gave us a good wake-up
call with his keynote on moving to an agile environment, based on
his experiences at Intel. Ray's a great speaker (and a fun guy
too - I might add). He described his hits and
misses; the 'misses' are often the most
interesting parts of experience reports. Lot's of good advice
and some nice puns (Wagile, FRagile, Scrumfalls).
I stayed in the agile track in the big auditorium where
John Watkins presented some material from his book
on agile testing, aptly named "Agile Testing". John had gathered
case study material from twenty agile projects
and proposed agile methods for small, medium, large,
off-site, and even off-shore projects. Intriguing,
but upon hearing the idea of "agile best
practises", my context-driven genes started to play up.
John was also my great trackchair and introduced me as
"Filmstar, Rockstar, Tester!" At least, that was his own juicy
summary after I mentioned to him that I had worked as a movie
distributor before and had also played in a rock band.
Granted, I also admitted playing a zombie once - a serious
case of method acting. Anyway, his introduction loosened the
audience a bit and I was able to present my track "A
lucky shot at agile?" without any problems. I wanted to tell a
testing story and I think it went well. I felt at
ease (those wireless microphones are really great) and
there were many questions afterwards. During the rest of the day
people I didn't know came up to me to congratulate me with the
presentation, which was nice. I took a long lunch and had a walk
around the expo. I went back to the Test Lab to
report the bugs that we found earlier. I didn't
succeed in entering them all, which made me feel kind of
guilty - I wished that I would have spent more time
there. But I had a hard time choosing. It's a pity that test
labbing also meant skipping tracks as well.
The last regular talk of the conference was held by Rikard Edgren, who
is also a Eurostar regular. I had seen his presentation on
testing creativity ("Where testing creativity grows") in 2007
and I liked it a lot, since it is also a subject
that is dear to me. There's far too many people that think that
testing is not a creative or challenging
activity. This time he talked about "More and better test ideas". He promoted the
use of oneliners as test ideas - a brief statement of something
that should be tested. These test ideas can then be used
as a basis for test cases, or as a guideline for other types of
testing, or even discarded when there irrelevant or when there is
simply not enough time. I think Rikard's subjects will always
be a bit polarizing due to their innovative nature - you
either like them or you don't. I am a believer and it was a good
way for me to finish the conference.
I missed the first part of the Test Lab result
presentation since they changed the timing and I totally
forgot about that. But I got the most important statistics. Over
two and a half days, more than 50 bugs were found. My first
reaction was: "Only 56? Man, there's hundreds of them hiding in
there", but then I realised that people had been testing in the lab
only for short periods, in between tracks, just as I did. I wonder
what would have happened if hundreds of testers had a go at it, all
at the same time. Bugfest!
After a short panel discussion with John Fodeh (next year's
programme chair), Geoff thompson, Tobias Fors and Nathalie
Van Delft it was time for the award ceremony. Naomi
Karten received the Best Tutorial Award and the European
Testing Excellence Award went to Anne Mette Hass.
In the meanwhile I was dozing off in my not-so-comfy chair - these
4 days of conferencing were finally getting to me. A friendly
woman on the stage was mentioning someting about a longlist of
papers, and a shortlist, and a final selection of three,
containing two Dutch and one Belgian paper. Now wait a
minute... how many Belgians sent in a paper? 1...2... before I
could make the math, my name was announced as winner of the ENEA
Best Paper Award. Two talks at Eurostar, two papers, two
awards... what are the odds of that? I was absolutely
flabbergasted. That's actually three in a row for my company CTG, since Bert Jagers won the award last year
in The Hague. The pressure is on for next year .
I spent the rest of the evening in the hotel bar, where all the
testers with an early flight on friday morning were flocking. We
ended the day singing an eclectic mix of Irish traditionals,
Dylan, early Springsteen and - of course! - Abba, accompanied
by a non-certified tester, who plays a mean mandolin. I love
Stockholm in wintertime. It was a good Eurostar. Yes
sirree.