I want to make a list and explain some of the preconceived ideas about exploratory testing that come mostly from people who heard little or not at all about the term. I used the discussion from a context-driven testing forum, especially James Bach comments.

• Exploratory testing is synonym with manual testing. This is not true because you can use any tool you decide helpful to accomplish your mission, including scripting. Its the same thing as saying that driving is a manual task. You use your head when driving (not all the time maybe or not in all situations but those are exceptions).
• Exploratory testing is called sometimes ad-hoc testing. Ad-hoc is part of ET but not the definition of it. With ET you seek continuously to improve your tests each time. If you go on a trip and visit a lot of stuff, that is close to ET. Having a quick look in a museum is ad-hoc.
• Exploratory testing is a technique. Well ET is not a technique in itself. It might contain a set of different techniques but its more like an approach. Breaking a wall is an approach to make more space in your house for example, but you can use different ways to tear it down.
• Exploratory testing is never planned. This is not necessarily true. You can plan it to some extent. You can plan to split an application to different areas. Then you decide to reorder different because of the new finding done with ET.
• Exploratory testing is not documented. There are examples on how to document ET. One of those is Session-Based Management that can contain data of the tests that have been performed. You can report the path that lead you finding a bug (not referring to steps to reproduce). You can report tools used, knowledge used ( SMPT protocol)
• Exploratory testing is never used in combination with scripting. ET and scripting are opposite approaches but can be mixed. You can create a script to get more data and arrange it so you can find a possible pattern.

I want to make a little intro for scripting that might be useful in testing. Some testers are very scared of it and don’t use scripting at all. Some are scared when see the jobs ads and become obsessed in learning all the tools required.

I will practice on an application used by Markus Gärtner to perform testing but with a different approach. This is Parking Calculator. It is basically calculating the cost of a parking ticket based on the entrance and exit time-stamp.

After you play with it (like 15 min or even less) you can see how it works: gets some input, and when pressing “Calculate” you get a result expressed as an integer (basic case). But what is very important is the URL generated. For example this case:

The url generated is:

http://adam.goucher.ca/parkcalc/index.php?Lot=STP&EntryTime=12%3A51&EntryTimeAMPM=AM&EntryDate=7%2F14%2F2010&ExitTime=12%3A52&ExitTimeAMPM=AM&ExitDate=7%2F22%2F2010&action=calculate&Submit=Calculate

You can see “12:51″ is there in the url as “EntryTime=12%3A51″ for example. “7/14/2010″ is there as “EntryDate=7%2F14%2F2010″. You can change the url itself and see the application appears to fill the data itself. Instead “52″ use “48″. Also important the result of park ticket cost is generated by only loading the link. This means we can use the link directly and read the source code of the webpage to check the result.

With python code we can do this with 3 lines:

import urllib2
f = urllib2.urlopen('http://adam.goucher.ca/parkcalc/index.php?Lot=STP&EntryTime=12%3A00&EntryTimeAMPM=AM&EntryDate=7%2F14%2F2010&ExitTime=12%3A00&ExitTimeAMPM=AM&ExitDate=7%2F22%2F2010&action=calculate&Submit=Calculate')
print f.read()

The first 1000 chars will look like this:

So till this point we have a python script that for a predefined date(fixed not parametrized) is outputting the source code for the page containing the relevant result.
In the webpage source we see result “210″. We can search to find where is located within:

This is a lucky situation because: the result is displayed only once and it has before it a char recognizable for parsing the result “$”. Not to parse the result to get it from the huge page you can do a regular expression extraction or chop out the text. Using regular expression can be more complex.

I will chop it. First start from the dollar sign to the rest of the initial string:

import urllib2
f = urllib2.urlopen('http://adam.goucher.ca/parkcalc/index.php?Lot=STP&EntryTime=12%3A00&EntryTimeAMPM=AM&EntryDate=7%2F14%2F2010&ExitTime=12%3A00&ExitTimeAMPM=AM&ExitDate=7%2F22%2F2010&action=calculate&Submit=Calculate')
text1=f.read()
text2="$"
print text1[text1.find(text2):]

Now we have to truncate the right part starting from firs occurrence of char “<”

import urllib2
f = urllib2.urlopen('http://adam.goucher.ca/parkcalc/index.php?Lot=STP&EntryTime=12%3A00&EntryTimeAMPM=AM&EntryDate=7%2F14%2F2010&ExitTime=12%3A00&ExitTimeAMPM=AM&ExitDate=7%2F22%2F2010&action=calculate&Submit=Calculate')
text1=f.read()
text2="$"
text3 = text1[text1.find(text2):]
text4 = "<"
text5 = text3[:text3.find(text4)]
print text5

If you save this into a file test.py and run “python test.py” you should have “$ 210.00 “.

This is not the way of doing python programming but should help a lot the beginners! We also use the code as secondary need, its not included in some product!

Now let’s parametrize the input so we can use any input data.

We have this parameters:

1. Lot type, “Lot=STP” the default value seen in url
2. Entry time hour
3. Entry time minute
4. Entry time AM/PM option
5. Entry date day
6. Entry date month
7. Entry date year
8. Leaving time hour
9. Leaving time minute
10. Leaving time AM/PM option
11. Leaving date day
12. Leaving date month
13. Leaving date year

We also need to reduce the string used because is very long and hard to work with.

import urllib2

def CalculatePrice(P1,P2,P3,P4,P5,P6,P7,P8,P9,P10,P11,P12,P13):
    urlPage="http://adam.goucher.ca/parkcalc/index.php?"
    urlLot= "Lot=%s&" % (P1)
    urlEntryTime="EntryTime="+P2+"%3A"+P3+"&"
    urlEntryAMPM="EntryTimeAMPM=%s&" % (P4)
    urlEntryDate="EntryDate="+P6+"%2F"+P5+"%2F"+P7+"&"
    urlExitTime="ExitTime="+P8+"%3A"+P9+"&"
    urlExitAMPM="ExitTimeAMPM=%s&" % (P10)
    urlExitDate="ExitDate="+P12+"%2F"+P11+"%2F"+P13+"&"
    urlEnd="action=calculate&Submit=Calculate"
    urlstring=urlPage+urlLot+urlEntryTime+urlEntryAMPM+urlEntryDate+urlExitTime+urlExitAMPM+urlExitDate+urlEnd
    f = urllib2.urlopen(urlstring)
    text1=f.read()
    text2="$"
    text3 = text1[text1.find(text2):]
    text4 = "<"
    text5 = text3[:text3.find(text4)]
    return text5

result=CalculatePrice("STP","12","51","AM","14","7","2010","12","52","AM","22","7","2010")

if result=="$ 210.00":
    print "Test Pass"
else:
    print "Test Fail"

You can see here too many parameters. You can leave like this or group by date and time. You can have 3 parameters by doing that.
You can use the unittest framework, the usual framework for building automated tests in python.
You can parse better the html using BeautifulSoup
You can do better coding.
You can generate a range and expected result for running.
So here it depends on the person doing that. If you do as a task only in work you will care very little for the value of testing and focus on coding. If you want valuable result s from that you will not care too much for coding, unless you have in mind integrating and sharing.
The most problematic issue with automation is that increases thinking but in a direction that doesn’t help too much the testing value, doesn’t protect the application from bugs getting to customers. It only seems more valuable when the other testers not only don’t know scripting but don’t study testing at all.

I saw that is somehow popular to create sometimes a 101 list, so I though I would create one myself.

The list is about the terms encountered, most of them in testing folklore,job ads, some relevant some not, but in general words related to testing.

1. Tester

2. QA

3. Test Case

4. Test Plan

5. Selenium

6. Automated Test

7. Pyhon

8. Programming experience

9. Agile

10. White Box

11. Black Box

12. Grey Box

13. Verification

14. Validation

15. SQA

16. GUI testing

17. Unit Testing

18. Integration Testing

19. System Testing

20. Regression Testing

21. Acceptance Testing

22. Alpha Testing

23. Beta Testing

24. Performance Testing

25. Load Testing

26. Stability Testing

27. Usability Testing

28. Security Testing

29. PEN (Penetration) Testing

30. Internationalization and Localisation

31. Destructive Testing

32. Smoke Testing

33. Ad-hoc Testing

34. Waterfall

35. Test Cycle

36. V-model

37. Tools

38. Exploratory Testing

39. Pass

40. Fail

41. Input

42. Output

43. Testing Team

44. Test Team Lead

45. QA Lead

46. Testing methods

47. Associate tester

48. Junior Tester

49. Senior Tester

50. Specifications

51. Documentation

52. Bug

53. Bug Report

54. Bug Tracker

55. Jira

56. Bugzilla

57. QTP

58. Test Run

59. Coverage

60. ISTQB

61. ISEB

62. Certification

63. Test Specialist

64. Check

65. Values

66. Oracle

67. Algorithm

68. Heuristic

69. Statistic

70. Model

71. State

72. Step-by-step description

72. Expected result

73. Actual Result

74. Test Suite

75. Test Summary

76. Test Scenario

77. Test Procedure

78. SUT (System under test)

79. Test harness

80. executed tests

81. Test execution engine

82. Test script repository

83. Manual Testing

84. Assumption

85. Session based testing

86. Path

87. Test environment

88. User acceptance testing

89. Risk

90. Test estimation

91. Test configuration

92. Test results

93. Investigation

94. Error

96. Windows and Linux environments

97. Test Analist

98. Test Phase

99. Test Design

100. Fix

101. Test Experience

Well I might have duplicated or forget some important ones but this are pretty common terms used when referring to testing. It might not be in the same context but they are there and maybe it shouldn’t be.

I don’t see too many articles or blogging on the other part of a testing project: the client.

We have to bare in mind and be prepared to give the client some things that tends to be everywhere.

Examples:

• Most customers will want a person to be responsible in testing for a project or a team. So here are needs for leadership, taking changes, good organization, reporting, coverage, task split and many others. If you deal with people, even if you have a lot of stuff properly done, there is always some “surprises”. Like someone in the team just hates you.
• Most clients need to have things “already done”. That’s why so much automation even when is not recommended. Most managers think Test Cases-> Scripts and everything there is 1) covered and 2)automated and this is all. Scripts debugging, assertions put to pass just to “get over with this script” are not being taken in consideration. The end goal is usually to have everything automated and if new issues still appear more automation is needed. That’s why some people who do everything asked and make huge amount of scripts are fired because many bugs get away with this methodology.
• Most clients had already something in mind: a structure and an approach that will not be changed. For example when I did scripting in one of my previous companies I was literally sabotaged because no one was able to do any useful scripting and that interfered with the program even if I was the most productive tester. This is the actual cause why most companies use obligatory automation because they know only this option and the option with people who don’t know scripting. And the automated approach seems more successful because in the other case those “testers” don’t try to learn something that will help them a lot but they will try to get away. Besides scripting they will not try to learn and study testing in general and you will have mediocrity there. I think this is one of the major problems in our industry.
• Most clients want testing to be MEASURABLE. In the internal structure there are reports to fill and pressure to evaluate. In normal company management needs to evaluate an employee to know if it keeps him/her or give more/less money, promotion. Same with contractors/consultants they want to know if its useful. And I think measurability was discussed but still with no real indicators.
• Most clients expect from testing to be simple and that’s why they select testers based on Test Cases, Scripts, methodology (“Do you currently do any load testing with that specific tool?”) or on random things. But in order to justify them our approach we really need to be prepared.

I wanted initially to make a simple video on some Skype issues. What I realized is that I don’t do good videos, I don’t speak clear enough, I don’t keep people connected. But I don’t want to spend too much time in creating a video to look good and over thinking. I want to just try to point my idea for now as best I can, post it, see my obvious mistakes and try to improve the parts that don’t work so well in the future. Keep the flow. Step-by-step.
I really need to improve a lot, but till I get better, I can provide usefulness with some other skills that I am already above the average, like creating a nice puzzle http://www.testalways.com/2010/07/05/find-bugs-and-patterns/ .
In testing everyone tends to go on their already defined area of expertise. Either if its team leading and task assignment, programming, good communication skills, logic and problem solving, business model approaches, its always good as long as it doesn’t go too much on just bulshit. But we need to learn and get familiar with the other parts that we need in testing, try to be completely covered.
So this is what I am trying now.