Examples of when to say YES:
Please keep in mind that the examples and instructions included here are guidelines. It is impossible for us to include every single case that you’ll come across while working. You should answer Yes in cases like the following:
-
Answer yes if the expected outcome(s) of the question are met, and you see no other issue.
-
Answer yes if the instructions ask you to perform an action and you can confirm an outcome, after taking small, reasonable steps to move forward even if the action isn't fully described. Sometimes test authors may not fully describe the steps needed, if the steps are obvious in the context of the test case. If some small actions leading up to the desired outcome are not fully spelled out but it’s clear what you need to do in order to proceed based on the situation you’re in, then go ahead and take small, reasonable steps to move forward and answer YES if expected outcomes are met. Some cases you may encounter:
-
Go to new tab, open image URL and save the image. The next step does not ask you to return to the original tab, but gives you instructions that you have to perform in original tab. → It is clear that you should continue in the original tab and you must know how to open windows in a new tab in any browser.
-
Navigate to the “Dinner” menu. Were you able to download the “Dinner” menu? → It is clear that you are supposed to find and click a link or button to download the menu.
-
Enter the provided email and password. Are you taken to your dashboard showing recently viewed articles? → It is clear that the next step is to log in using that email and password in order to see the dashboard and articles.
-
Instructions say to enter provided email and password, and the question asks if you’re able to log in. The instructions don’t specifically tell you to click the log in button though. → It is clear that you should log in using the email and password.
-
Instructions tell you to search for a name (e.g. a patient record) in a table, and the table spans a few pages. → It is clear that it’s your responsibility to browse and search the entire table, meaning all the pages it spans. In this case, you may also want to leave an optional SI under the category “The action you want me to do is not fully described” and ask the author to provide a clarification that there are several pages testers shouldn’t miss. Please reference the educational ‘Tester Basics’ article if you’re still unclear.
-
-
Answer yes if you personally understand the instructions, even if you may be worried other testers will be confused. You must click YES if expected outcomes are met (i.e. you can answer the question), but you may click “Suggest Improvement”, select “These instructions are hard to understand” and leave a thoughtful comment explaining that some testers may not understand the instruction, and explain why (or even suggest an alternative). Be sure to be polite and courteous.
Examples of when to say NO:
If you need to answer NO, be sure to leave a helpful, concise and grammatically correct comment explaining why the step failed. Here are some examples of when you’d need to answer NO:
-
Say no if the outcome of following instruction(s) doesn’t match what’s asked in the question. For example:
-
The instructions ask if you see a page with 5 images but there are 4 images. If the test author expects 5 images but only 4 appear, then clearly one is missing and you should fail.
-
You’re instructed to click the continue option, in order to complete a purchase, but nothing happens after clicking the continue button.
-
The page simply does not load.
-
-
Say no if there are multiple questions asked and ANY of them is false, answer NO and explain the issue. We are actively working to remove and discourage use of multiple questions in instructions, but you may still see them from time to time. For example:
-
Navigate to the “Crafts” page. Once loaded, hover over the menu on the left hand side and click “Crochet and Knitting Crafts” option. Once the page is loaded, click on the first product you see (“Squishy” brand yarn). Does a new page load? After the page has loaded, add 3 of the yarn into your cart. Proceed to checkout. Does your cart load and show all 3 balls of yarn?
-
You may also add an optional Suggest Improvement under “This step can be broken into multiple steps” and ask the test author to only include one question for clarity.
-
-
Say no if you must complete one or several extra action(s) that are not common or immediately obvious for the test case you’re in. These are specific examples where the test author requires you to make too big of a leap from where you are to where you need to be. You should explain that you cannot complete complex actions unless guided to do so. For example:
-
You are instructed to navigate to the “Hats” section of the menu. You cannot find the hats menu because it is actually a submenu under “Clothes” that you’ve not been instructed to click or hover over. → It is not obvious where this menu can be located, so you should click NO.
-
-
Say no if you don’t understand the instructions provided, but make sure to explain what is causing the confusion. Remember that all testing happens in English, and you are expected to have a good command of the language. If the instructions are in English, but have unexplained jargon or are written in a very confusing or unnecessarily complicated way, you must be very clear in explaining what’s causing the confusion. Otherwise the test author won’t be able to update the instructions for you and others in the future. If you mistakenly pass or fail the test because of poor English comprehension and a Peer Reviewer decides that the original instructions were clear enough, there is a score penalty. For example:
- The test author used some (internal business) jargon that can’t be found anywhere in the app or page you’re testing, such as “Do you see the DKIM string on the page?” → If DKIM is not mentioned anywhere on the page, you must explain “I don’t know what DKIM means, it’s not explained anywhere”. Please make sure you double check the application before reporting the term definitely isn’t there.
-
Say no if the test author asks you to do something that is not possible for you as a tester. For example:
- You are asked to reference a different tests of theirs. You should report NO and explain that you can’t see other tests, only the one you have in front of you.
- You are asked to wait for something to happen for very long periods of time (e.g. over 10 minutes). For example, with emails, wait 3-4 minutes at most. But if the expectations is clearly to wait more (e.g. run a report and wait 15 minutes for it to generate) then report NO right away and explain that you may lose the terminal you are working on if you follow the provided instructions.
- You are asked to log into any platform using your personal email or login credentials (e.g. email account, social media accounts, sign up pages for any customer app, etc). Report NO and explain this is not allowed as per Rainforest’s security guidelines.
-
Say no if the test author asks you to answer YES even though the answer to the step question should be NO based on what you saw and/or did in the app. Answer No, politely explain this is against Rainforest testing rules, and also send a support ticket via ‘Get Help’ using the category ‘Other’.
- Say no if you are unable to complete non-conditional actions. Answer No, and politely explain the issues with the instructions.
- You are given variables and asked to fill in the fields for the name on card, credit card number, expiration date, and CVV number but the expiration date field doesn't appear at all. Report NO and politely explain that a field is missing.
- In a scenario, where the test authors state that a field is conditional and it's missing, then answer YES. For example, you are asked to fill in the name on card field (if applicable) and the field is missing then you can still proceed.
-
Except for unique jobs clearly marked as ‘Continue Past Failure’ jobs -- say no if the test author asks you to continue even if you experience an error or failure, AND the error you encounter is not in pop-up format. You must ignore the instruction and report NO if you see a non-pop up error or can’t proceed with the job. Leave a comment that explains why you are failing the job. Be sure to be polite and courteous. However, if a TA specifically states or shows a screenshot of an error that you should ignore, then please follow those instructions.
Example of instructions you should report:
- “During the test please pass any issues or failures and just note them in Suggest Improvement” → TAs shouldn't ask testers to use SI to provide annotations. The feature must stay true to its original purpose and both testers and TAs should avoid misusing it.
Note: Our definition of pop-up is “anything that suddenly appears (text and/or window) AND you don’t need to hover over it in order to see it.” For specific advice on what to do if the author tells you to ignore errors and failures, and you encounter an error in the form of a pop-up, please see Rule 3.