A well-defined charter is vital to keeping a group test session on track. Otherwise, the session quickly dissolves into a blizzard of "what if" questions and outlandish suggestions about what the mission "really means." You're going to get some of that anyway, but you can minimize it by providing a concise set of statements to help guide the focus of the session. Let's first understand what a charter is: it's a contract between the testers and a client to perform a small amount of work. It must represent a reasonable common understanding of the problem to be solved, actions to be taken (or avoided), and results to be delivered. In the absence of that understanding, a responsible tester should question and re-negotiate the charter. Whatever the outcome of the test session, we want to be able to say "I accomplished the charter" or "I made good progress toward this charter" or at least "I did good work with respect to this charter" or "I decided to go off charter or ignore this charter because..." Therefore, the first questions you must answer when you get a charter are: - Do I understand this? - Do I know how to do this? - How do I know if I'm doing it well (or badly)? - Can I commit to this? And therefore, if you are providing a charter you should write it to anticipate those questions, and either answer them outright, reduce the likelihood that the tester will make bad assumptions about them, or be prepared to answer them interactively. As you prepare your mission, consider the following factors: - Objective: What is the objective? Is this for a bug bash? Is this a product evaluation? Is this a coaching session with more than one tester who have the same question about how to do something? Is it to practice a certain skill? Use a certain technique? Evaluate a specific tool? Each of these things requires a potentially different set of information presented to the tester. - Focus: In considering your objective, decide what you wish to focus on. Any of the following may be part of your focus: - Client: a specific client the tester must satisfy. (e.g. I am a marketing guy and I need you to test my product.) - Activity/Skill: a specific kind of behavior or skill you want the tester to do or practice. (e.g. practice state-based testing or demonstrate gallumphing) - Problem: a specific problem you want the tester to solve. (e.g. test against a specific risk, or test a specific area of a product) - Deliverable/Outcome: a specific result you want the tester to achieve. (e.g. find five bugs and report them) - Other Context: Any other resource or constraint or consideration that the tester must take into account. (e.g. do not use Google) - Freedom: In considering your objective, think about what freedom you can allow. Testers are clever, and if you don't tell them they can't use Google, for instance, they will probably "cheat" that way. - Setup: Is there a need for setup activity? Signing up for a website, or installing a product takes time. You don't want people burning the precious session time on these things. So you should pre-announce any required setup. (Information gathered about the setup may be relevant to the testing exercise, so encourage note-taking and reporting on it, but keep it separate from the assigned session work and the debrief.) - Feasibility: The charter should be something that could, in principle be accomplished in the time allotted. Otherwise, testers will feel a little depressed at the end. - Relevance: The charter should seem relevant to the tester. Be prepared to explain how it relates to practical industrial software testing. - Fallback: In case the charter cannot be fulfilled for some reason (say, a particular website you need cannot be accessed) have a plan "B" ready. - Outcome Flexibility: Be aware of and respectful of the full range of possible positive outcomes of a testing session. Even if your goal is specific, avoid giving the impression that failure to achieve the tangible goal is equivalent to "producing nothing" or "wasting time." - Outcome Quality: Consider specifying or else allowing the tester to specify the quality level of their answer. For instance, a tester may offer a "quick" answer, which means an answer based on no more than a few minutes consideration; or a "practical" answer, which means it is based on a specific set of context assumptions offered by the tester; or a "deep" answer which is represents the best thinking and research that the test is able to do (often requiring the tester to ask a lot of questions or declare a great many assumptions) and may address a wide range of contexts. Evaluate the tester with respect to the quality level that he targeted. - Outcome Aggregation: Do you require instant delivery of results for aggregation into a group result? If so, specify the method of gathering the results. (Google docs, skype, etc.) - Intentional Vagueness: Is this an exercise about analysis and thinking? If so then you might want to make the mission intentionally vague in some way, in order to get testers to discuss it with you. You may also want to strategically deny testers certain key information. - Trap Avoidance Behavior: Testers who have experience with exercises will be suspecting a trap, and may engage in long series of questions designed to detect your trick. This is normal, but if you want to avoid this, you can tell the testers that it's okay for them to make any assumptions they feel are reasonable, and then reiterate the main goal. General patterns: - IN {w} TIME, PERFORM {x} TEST ACTIVITY ON PRODUCT {y}. PROVIDE RESULTS IN {z} FORM. - EXPLORE {w} FOR THE PURPOSE OF SOLVING PROBLEM {x} FOR CLIENT {y} IN CONTEXT {z} - HERE'S A TESTING SITUATION {x}, ADVISE THE TESTER ABOUT WHAT TO DO NEXT. - REVIEW AND EDIT THIS TESTING ARTIFACT {x}. Example Charters: ------------------------ #1 Product Tested: Virtual Magnifying Glass Charter: Test Virtual Magnifying Glass and state whether we may use it. ------------------------ #2 Product: Freemind Charter: Evaluate Freemind for use of generating test ideas. Mind map your test ideas, and see if Freemind is useful in this. Select an area of focus for your testing, and mind map what ideas you would like to explore. ------------------------ #3 Product: Brain Workshop, a “Dual N-Back” brain training game intended to improve working memory and fluid intelligence. Charter: Explore Brain Workshop using the FCC CUTS VIDS touring heuristic and report on your findings. The goal is to practise using tours to learn interesting quality related information about a new application quickly, and to produce a short report on what you learned. ------------------------ #4 Charter: http://www.allprofitallfree.com/color-wheel3.html I have three colors in my mind. Prepare a list of questions which would help you guess the color in my mind. You can ask as many as 32 questions. You can choose to ask questions one by one or in a list. ------------------------ #5 Charter: Understand 508 compliance and accessibility testing. Todays Charter divided into three different tasks. 1. Find out information about 508 standards (brief info) 2. Download 508 toolbar by searching for it on the net 3. Check the websites using the toolbar Site you can test: http://beanbaglearning.com/ http://www.mindbites.com/ http://www.etoro.com/ http://www.clubcooee.com/ http://www.yoowalk.com/ ------------------------ #6 Charter: A startup has sold their product to a major financial customer that will result in its large scale use (over 15,000 users). The application has never been thoroughly tested. The bank in particular has concerns on the security of the data. ------------------------ #7 Application Tested: Klavaro Charter: Felipo want us to convince them the problem is INDEED a problem. Handy mnemonic for oracles: HICCUPP by James Bach for your reference can be found out here. ------------------------ #8 Charter: To evaluate the use of Ben Simo’s usability heuristic for error reporting. Select at least one error you have encountered recently, and use the FAILURE heuristic to evaluate it. Submit a report evaluating the heuristic. ------------------------ #9 Application Tested: Clicky Gone Charter: To find quality related information about the product. ------------------------ #10 Product Tested: Converber Charter: You are moving from lovely Europe with the measurements based on the metrics system to the US with the imperial units. Test Converber v2.2.1 (http://www.xyntec.com/converber.htm) for usability in all the situations you may face. Report back test scenarios for usability testing until 4.30pm GMT on bug repository. ------------------------ #11 Charter for the Session: Generate test ideas to test Google buzz with performance as the quality criteria. ------------------------ #12 Product Tested: Wiki on a stick Charter: Use the SFDPOT heuristics to test Wiki on a Stick. - SF DePOT (Structure, Function, Data, Platform, Operations, and Time) - http://www.satisfice.com/articles/sfdpo.shtml - http://www.developsense.com/articles/2005-10-ElementalModels.pdf Report your bugs and test reports to bug repository (http://bugrepository.com/) under the Project EWT08 until 16.30 GMT. ------------------------ #13 To find Functional bugs in the Splashup application Session details: Notalon: Download link: http://sourceforge.net/projects/notalon/ Lecture: http://testingeducation.org/k04/video/OverviewPartA.wmv Reference: http://en.wikipedia.org/wiki/Cornell_Notes Charter: LISTEN to the lecture and Prepare the notes from the lecture. Deliverable: The notes must be exported to a pdf file using Notalon application. The best note-taker would be judged on the following parameters: 1. Content – Lecture Content 2. Easy to read notes. 3. Good usage of Notalon features. ------------------------ #14 Website Tested: http://www.cgitoronto.ca/ Charter: To find as many bugs as possible that will hamper the credibility of India and its power as a software house of the world in this application that other country people use before entering India. ------------------------ #15 Context: A client wants Beta Testers for testing Converber. You have to prove them that you are the best Beta Tester for them. How will you be judged? You have to give a list of Test Ideas/Scenarios you would cover if selected as the Beta Tester. Based on the list of Test Ideas/Scenarios, you’d be selected/rejected as the Beta Tester. ------------------------ #16 Application URL: http://www.flasharcade.com/arcade-games/play-3836/reverse-mouse-game.html Charter: Complete the game. ------------------------ #17 Application: http://sourceforge.net/projects/freekeygenera/ A small application: A very quick password generator. Charter: Your Test Manager has asked for a detailed test plan as to how you will test this particular application. ------------------------ #18 Application: http://www.gamesforthebrain.com/game/whatwasthere/ Charter: Score the highest points and capture its screen shot. ------------------------ #19 Website Tested: http://calendar.google.com Charter: You work in a small-medium company, and your manager has been asked to evaluate switching the company over to using Google calendar. He needs a quick assessment from you before his conference call this afternoon. ------------------------ #20 Website Tested: http://www.bing.com/maps Charter: Test Bing Maps and bring up test scenarios. Create a list of scenarios with users of the system and how they interact with the system. Write your scenarios as narratives – mini “soap operas”, as they’re described here: http://bit.ly/3CVd4z . (Note: scroll down to the bottom of the page for some examples, the article is quite long.) Please note down your findings and enter them on http://www.bugrepository.com under EWT02 project ------------------------ #21 Website Tested: http://www.splashup.com Charter: To find a list of bugs in the SplashUp application. Please note down your findings and email it to europetesters@googlemail.com. ------------------------ #22 Charter: Brainstorm test ideas for Installability testing and report few bugs based on these test ideas ------------------------ #23 Charter: To find all the bugs you can find in the product [James Bach - Triangle Program] ------------------------ #24 Charter: You are contracted by three stakeholders to evaluate URL shortening services. The stakeholders are - a heavy twitter user tweeting a lot of internet references over the day - a university student wanting to share stuff from his homework/courses over the internet with his fellow students - an online magazine publisher for referencing further readings on the internet The pre-selection already has taking place and the most promising products are http://tinyurl.com http://bit.ly http://3.ly http://is.gd http://tr.im http://stnx.at Evaluate the products for their applicability for the given users and report any problems the mentioned stakeholders may run into.