All companies want “test automation”. Managers that never tested think they need “test automation”. Test managers who don’t like to code want someone with “test automation”. Fresh students who were just hired know “test automation”. Because of that recruiters want “test automation” for “QA”. Testers are encouraged to know “test automation”.

But when asked to think about it or describe what is that, no one wants to answer, either because it doesn’t need any explication or its like those concepts like “life”.

When people do “test automation” the following happens:
- time is spend in creating some scripts and maintaining it and eventually is not as “faster” as was ideally though it would be
- people are bored and unmotivated because the job just becomes a sub developer position at every level, especially payment; they eventually might choose to be totally change their lives and become tourist guides
- people quickly want to change their task and want to advance so they won’t do that anymore, but tell others how useful is “test automation” to do all day
- because there is not too much time to investigate the purposes and needs of some functionality, simple checks are done under the cover up of “test automation” or mocking assertions where everything passes.
- fake data is generated for managers who don’t care anyway except their own job
- parts of application code is just copied and adjusted a little bit to work as test automation
- if bugs to customer appear often, of course “we need to hire more test automation guys”

Here is how I see those concepts:

Testing is the activity performed by the person, not by the tool, including decisions, use of skills.
If I do scripts that’s not testing. That’s scripting. Testing is when I make the decision of creating a tool that will help me.
Automation can be the process of using scripts to help with testing. Populating a database with data every week/day for me to test that is automation. A script to install the application every day on my computer, that’s automation.
Occasionally the automation might check an expected result: like when I use selenium to do some web application steps and check if some data is present and that data is as expected. I call that automated checking. Its not testing. Its just while I was doing testing I though of the need of expanding my reach by using a tool.
And because of that the term “test automation” or similar associations between testing and automation doesn’t make sense. It is used improperly most of the time and its referring at some low level development.

I want to recommend this tool http://code.google.com/p/knull-shell/downloads/detail?name=knull-shellv1-beta.php for testing security issues.

With it I was able to hack into one of my client’s web server (literally the server that hosted the web application) and was able to create folders and  delete files.

The script can be used for hacking a web application in places where you can upload images or any other files.

Example of a scenario:

1) Check if the web app has a section to upload an image (usually user profile)

2)Use an image with a specific name that you can recognize and upload it

3) Check after upload and storage where is the image now located (check if the name is intact ; check if the size is the same )

4) If the image is stored with the same name or is not altered, you can try uploading the knull-shell script and the check the url

5) if the url of the stored file ends in …/some_location/knull-shellv1-beta.php and permissions are set badly, you can try to execute that script by opening that url

6) If you are successful you should have something like this:

Username: root

Password  :toor

and you should have something like this

Now you can try to see how far you can go!

Another session was this Saturday, actually sessions (+WT and WTA). Its very good considering there were basically 6h for those willingly to join.

Even that I proposed the mission and got to play a little before the actual meeting, I think I got more knowledge about boundary testing and limits.

Everything will differ for a product and environment, but in this context I saw:

-a chat message goes through many locations. As we assumed (read the transcript) it will be probably be an Xml record at one point, then maybe a db record. Depending how the data stream is sent, we will probably have another set of limits.

-in testing limits we can actually do (without even trying)  security testing or Pen testing. So we should be also careful from the legal points of view.

-the boundary depends on the input. An alpha numeric char differs from another char. An example used were kanji (Japanese) chars (“Petteri Lyytinen: I tried to paste a text string that is 32768 kanji characters” ..”only showed the two dots”)

-anything too big is truncated (or should be truncated in some situations) and the process of truncation can be different in some cases, some unexpected

-some parts cannot be tested easily (or in the required time interval) like the history containing large messages

-tools help – this time we used one available, but in other cases tools need to be built up

Thanks to the participants:

Anna Baik, Ben Simo, David Vydra, Kristjan Uba, Lalitkumar Bhamare, Michael Larsen, Narasimha Reddy, Nitin Purswani, Petteri Lyytinen, Santhosh S Tuppad and Eusebiu Blindu (as host)

In medicine and journalism there are these models, these principles that are a guidance to the practitioner.  For example doctors have the Hippocratic Oath which is an oath historically taken  swearing to practice medicine ethically.

According to The Elements of Journalism, a book by Bill Kovach and Tom Rosenstiel, there are nine elements of journalism.

So I though of proposing myself a similar set for testing:

1. Testing’s first obligation is to show the truth.

This means to take off the veil of fake certainty that everything works. Also to validate any assumptions claiming there is nothing that reduces the value of the product.

2. Its first loyalty is to the client(s).

Here the clients can be users,  stakeholders,  managers. Its important to know who are the clients in the first place.

3. The essence of testing is questioning a product in order to evaluate it.

This is actually inspired from the definition that James Bach gives to testing.

4. The testers must maintain an independence from the people that give them the info.

For example the data gathered from a developer should not be trusted 100 percent. People will give you what they think or what they assume is right.

5. The testers should inform based on their analysis and thought.

This means they should not just take the info they receive and jump to use it to inform others. If initial information is false,  it will result in more disinformation.

6. The information provided by testers should be also reviewed.

This means a bug can be set invalid, or the reports should be reviewed by someone else.

7. The information about the product to test should be significant, interesting and relevant.

Testing should consider risk areas, priority based on circumstances, context and the  information should be useful.

8. The information provided should be comprehensive and proportional.

This means to inform the client right to the point, also in an acceptable format. No one will read a 100 pages report when its not needed.

9. Testers should be allowed to exercise their personal conscience.

This means the approach should be left for testers to decide  and have a relative independence.