As a prequel to Agile Testing Days where I will do my presentation, I am trying to interview the other speakers. Today we have  Matt  Heusser , a passionate tester that, among others, does "This Week in Software Testing" podcast, he is a member and part of Board of Directors at Association for Software Testing , a Contributing Editor at STPCollaborative and he also gave a talk at Google Test Automation Conference in 2007.

Eusebiu Blindu: How would Matt Heusser describe himself in the context of the testing community and as a person?

Matt Heusser:  Wow, Eusebiu, that's a tough one.  Within the test community, I'm sort of a holistic quality person — I have done stints as a programmer, project manager, and in test/QA.  As a consultant, I tend to focus on the whole delivery process.  Within the test community itself I tend to advocate for the tester as a specialized role, with a skills set that is different than the programmer or business analyst, not that they can't overlap a bit.  As a person, I am a writer, husband and father; a great deal of my success in the software community comes from sharing ideas through writing.

Eusebiu Blindu: Since the conference is called "Agile Testing Days", can you tell us how would you shortly define "Agile Testing"?

Matt Heusser:  I am not going to get any soft-ball qustions today am I, Eusebiu?  (Laughs).  Look, the question of What is Agile Testing is challenging on multiple levels.  Most importantly, if we write it down, people will tend to canonize the definition, to hold it up as the one right/true process to do testing, and that runs contrary to the spirit of the Agile Manifesto.  Of course, that presupposes that there is one right/true definition, and that we can all talk about it without inserting our own experiences and watering down the definition a bit. So it's a challenge.  I mean, how deep do you go?  Your one-sentence description is going to have information loss, and your five page description isn't going to be read by anyone.

But to answer your question, personally, if I had to give a couple-sentence description of "Agile Testing", I would say it is a test process designed to plug into an Agile Process as defined by the Agile Manifesto.  That means we want to be able to deploy to production much more often than in traditional models and we want to respond to change — so we have to get very good at risk analysis, at getting a great deal of confidence in the system in a compressed period of time, and we probably want to focus on actually getting things done instead of heavily, over-specialized role definitions.  Will that suffice as an answer for now?

Eusebiu Blindu: Do you apply "Agile Testing" currently and if so, how exactly?

Matt Heusser:  As I write this I am currently engaged on-site with a 'classic' agile team.  The entire team fits in one room, the code is written with 100% pair programming and Test Driven Development, stories are defined up front in a kickoff meeting and generally have an executable acceptance test element, and so on.  What my clients might be like in November, of course, is anybody's guess! 

To answer your question specifically, a lot of my advice and consulting is in the area of dealing with risk, with shrinking the time it takes to get an assessment of the software, with shrinking the "regression test window" (for lack of a better term), with helping teams to adapt to change.  So yes, I like to think that what I do is basically Agile-Testing, in the best sense of the term, sure.   

Eusebiu Blindu: Ok, so you say you can't really put a definition on "Agile" and that has be avoided. But can you make a parallel between "Context Driven" and "Agile"? Its the same, similar or different?

Matt Heusser:  I hope I've given you a working definition that is "good enough", at least.  Another definition might be preference for a certain style of working environment, combined with the belief that that preference leads to better outcomes.  So when "Agile"-infested people come into a new environment, they have a whole set of prescriptions: Pair Programming!  Test Driven Development!  Automate the Acceptance Tests – up front!  Story Driven Development!  Tear down the cube walls!  And so on.

I identify myself publicly with the Context-Driven School of Software Testing, which as a philosophy, is a little different.  I would go so far as to say that the first sin of methodology is make a claim like "This worked for me once!  Therefore, it should always work for everyone, all the time!" — that offering a prescription without first examining the patient is not best practice, but instead malpractice.

So while Agile Testing is a tool in my toolkit, one I am personally fond of, and one use often, there are some circumstances where it may not be appropriate.  Before I offer suggestions to an organization, I want to know a great deal about them.

As you can probably guess, offering training in context-driven testing is a real challenge, because you can't just tell people what to do. There are several ways around this, including workshops, simulations, and exercises designed to provide people with examples and skills training.  With enough data points under your belt, you start of say "oh, this reminds me of project foo.  How did that turn out?  Would that be acceptable to me on this project?" – you start to model the dynamics on the project and plan, instead of pulling out a plan from someone else.

Because it allows different approaches to be valid in different environments, context-driven is sort of a meta-school — it can swallow up specific methods, as long as they have a "place."  So, for example, I prefer Agile methods for projects done under changing conditions and time pressure, which happens to be the majority of my clients.  So I need to know a thing or two about Agile Testing, and Agile Software Development, in order to be effective.

Eusebiu Blindu: Liked your answer and the "meta-school" label. You have a tutorial with Pete Wallen and a closing keynote, both with very interesting titles. What should the audience expect from these? Can you give a preview?

Matt Heusser: The tutorial is going to be about testing.  Not management, metrics, templates or automation:  Actual testing.  That is to say, given a new build, which tests should you run?  What are the most powerful tests — the tests that tell you about the system?  Once you have run them, what can you say about the software?  What tests should you run next?  If you learn something about the software that might impact which tests to run next, how do you adjust?  How can you learn those things earlier?

In order to get the most benefit from our tutorial, we won't just lecture.  Instead, we'll air-drop the students into a software test environement, actually doing testing, sharing what we learned, covering the next topic, and testing again.

Yes, you heard right:  A test conference session where you actually get out a keyboard and do testing.  I know.  Who ever heard of a CRAZY idea like that? 

As for the keynote, that one will be about the dynamics on software projects — how we can structure them to encourage a successful outcome.  Or we could structure them so that everyone is competing and encourage to hide information.  I'll be covering this from within the general framework of Game Theory, that mostly came out of mathematics, where I did my undergraduate work.

Eusebiu Blindu: What is your current focus related to testing besides work? What is your plan for the next period?

Matt Heusser: My main focus is on how organizations deliver software, and how to do it well – how to make better decisions in the moment, throw away the stuff that slows us down, and do more of the stuff that speeds us up.  Along those lines, I have a preference towards exploratory methods, so I'm curious who is using how much exploratory testing to do what.  By exploratory, I mean a lot more than exploratory testing; I'm talking about how the teams are organized, how the work flows, how people are spending their time.

Toward the test side, I've spent about nine months working with a group where the whole team creates examples, that the  programmers automate as part of the development effort, using a tool like Fitnesse or RSpec.  Once those examples are automated, you have a very powerful domain specific language (DSL).  In some cases, I have seen testers reproduce defects from the field by modifying those old executable specifications — when this happens, they can turn a failing automated check over to the programmers to fix.  I think that's a good thing.  I am interested in what kinds of envrionments are fertile for that kind of approach, and the idea often falls down as it is transferred from person to person.

I am also using more tools to blend human, exploratory approaches with more traditional automation approaches — both things like the kind of model-driven approach that Harry Robinson recommends, and building better stubs, drives, and hooks into the software so testers can explore (or randomize input).  Testing complex GUIs, and just plain complex and high-volume systems, continues to be on my radar.

As I implied before, my focus seems to be shifting toward the entire delivery process and how to do it better.  Some folks call that work "Organizational Transformation", and I haven't found a better term yet.  Basically, I think most software teams can get better performance by restructuring the way they take in and do work.  My test work is probably the most visible, but you can apply the same thing to all aspects of software delivery, including the results, the real thing the customers are buying.  To put that another way, if the customer could get the same results without buying your software, of course they would.  So I care about the value delivery process, not just software.

Eusebiu Blindu: By the way, how did you get into testing?

Matt Heusser:  That's a long story, Eusebiu.  I'll try to answer it quickly.  When I started my career, I was a programmer, at a company with thirty employees; perhaps twenty-five of us were programmers.  We had no testers, no business analysts, just a bunch of programmers, a couple managers, two guys in sales, and a receptionist.  In other words, I had no safety net.  I had to learn about testing, to deliver high-quality code to the customer.  I was scared.

So I learned.

I went to school at night for my masters, and studied testing.  I read books, I went to conferences.  Half-way through my career, I moved to a larger company with a few specialities, but still no testers. People noticed that I had an interest in testing, and I started to help out on projects that were not my own.  My business card still said "programmer" (and later "project manager"), but, by 2005 or so, I self-identified myself as a tester.

When I left to go work for Socialtext, they had an HR change during the interview process.  The CEO told me that they no longer had the test manager position open, but would I like to be a programmer?  I remember replying "Why would I want that?  I can be a programmer right now."

They ended up making an opening for me in test anyway.  The rest, as they say, is history.

That's the short version.  I'll be happy to tell a longer story at the conference.

Eusebiu Blindu: Oh, thanks for the interview. I can't wait to meet you at the conference and get that story!

I am trying to find some use of the Benford’s law in testing.

“Benford’s law, also called the first-digit law, states that in lists of numbers from many (but not all) real-life sources of data, the leading digit is distributed in a specific, non-uniform way. According to this law, the first digit is 1 about 30% of the time, and larger digits occur as the leading digit with lower and lower frequency, to the point where 9 as a first digit occurs less than 5% of the time. This distribution of first digits is the same as the widths of gridlines on the logarithmic scale. Benford’s law also gives the expected distribution for digits beyond the first, which approach a uniform distribution as the digit place goes to the right.” –wikipedia

I thought I found a solution when googling by “1”,”2”,”3” or “one”, “two”, “three” or using even quotes for the same searches to be the exact match. But then I realized that I can hit a search result containing “two hundred and twenty one” when searching for “one” so I am very far from applying the principle in a relevant way.

Well it seems to make some sense though based on the graph, so maybe it’s some sort of applicable variation.

But let’s try something more relevant:

List of countries by number of mobile phones in use

http://en.wikipedia.org/wiki/List_of_countries_by_number_of_mobile_phones_in_use

1	China	1,010,000,000
2	India	911,168,193
3	United States	327,577,529
4	Indonesia	250,100,000
5	Brazil	245,200,000
6	Russia	224,260,000
7	Japan	121,246,700
8	Pakistan	114,610,000
9	Germany	107,000,000
10	Nigeria	90,583,306
11	Mexico	88,797,186
12	Italy	88,580,000
13	Bangladesh	86,550,000
14	Philippines	86,000,000
15	United Kingdom	75,750,000
16	Vietnam	72,300,000
17	Egypt	71,460,000
18	Turkey	66,000,000
19	France	58,730,000
20	Thailand	69,000,000
21	Iran	68,000,000
22	Ukraine	54,377,000
23	South Korea	52,510,000
23	Spain	50,890,000
25	Argentina	50,409,800
26	Poland	47,153,200
27	Colombia	46,147,937
28	South Africa	42,300,000
29	Algeria	33,000,000
30	Venezuela	27,400,000
31	Peru	27,100,000
32	Taiwan	25,412,000
33	Romania	22,800,000
34	Canada	25,543,862
35	Morocco	27,050,000
36	Netherlands	20,000,000
37	Australia	21,260,000
38	Saudi Arabia	46,000,000
39	Malaysia	30,379,000
40	Chile	21,000,000
41	Guatemala	17,571,895
42	Sri Lanka	17,359,312
43	Ecuador	15,900,000
44	Portugal	14,500,000
45	Nepal	14,240,670
46	Hong Kong	13,264,896
47	Belgium	11,822,000
48	Hungary	11,833,000
49	United Arab Emirates	11,540,040
50	Bulgaria	10,655,000
51	Israel	9,319,000
52	Denmark	7,000,000
53	Azerbaijan	7,000,000
54	Jordan	6,010,000
55	Singapore	7,289,000
56	New Zealand	4,620,000
57	Estonia	1,982,000
58	Lebanon	2,720,000
59	Lithuania	4,960,000
60	Montenegro	1,294,167
61	North Korea	1,000,000

Starting digits for the number of mobile phones in this list:

Considering that the data sample has only 61 records and for the first digits we have a success, I think its “relevant enough” and the example is “acceptable”.