Rule 2: Pay Close Attention to Quotes & Placeholders

Quotes are used to indicate specific text or values to look out for, and Placeholders stand in for variable text that will populate in the app you’re testing. This is a very important rule, especially when it comes to exact matches. Some functionality may simply not work if certain criteria aren’t met.

What clients want: With minor exceptions, it means the test author wants you to check that the text or values within quotes appear exactly as such in the app or that a certain value may be present in the app that they can’t specify exactly, so they will use a placeholder.

What this means for you: You must understand that words in quotes need to be matched exactly to the client’s specification, with minor exceptions (e.g. capitalization differences and where automatic number formatting comes in, as described later), and that placeholders can be placed both inside and outside quotes. You should also know that this is one of the hardest rules to enforce, and that human error may be present through typos, unfinished quotes, or other types of placeholders used. For that, we added several examples below.

Important notes

What clients see in Rainforest (that you should know about)

• Rainforest does not currently have a way for clients to highlight text only in order to draw a tester’s attention to it (e.g. without having to match it exactly). You may see some test authors use all caps or asterisks to draw attention to certain buttons or elements. Asterisks and all caps are acceptable usage, fall outside of the quotes rule, and should not be considered quotes. See our explanation for what constitutes a quote below.
• Rainforest formatting turns multiple spaces and line breaks into a single space. If the client asks you to match something with special formatting (e.g. text on multiple lines must match), they are advised to provide a screenshot for guidance as this is not possible from our step instructions. 

How we advise clients to write tests:

• To stay consistent in their use of double quotation marks or underscores as placeholders, but a few exceptions may still come through due to the unpredictable nature of test writing. If the rest of the step instructions give you enough context to understand that the client means a placeholder even though they’re not using the given examples, you should not fail their test.
• To only use quotes if it’s absolutely necessary for their step, and avoid using them if they’re not critical to the outcome of the test.
• To not confuse testers if what they really want is to draw attention to a word or element on the page, rather than get an exact match for it.
• To be careful about the language they use (e.g. “a couple” means two - not several items, even though in informal speech this is often ignored)
• To provide screenshots and call out the need for an exact screenshot match if there’s specific formatting on the page they want testers to verify (that our step instructions can’t render).

How you may see quotes and placeholders formatted:

• The quotes may be formatted with single or double quotation marks (“like this” or ‘like this’) or the character for grave accent (`like this`) - the same rule will apply. Mixed quotes (‘like this”) may be typos, but you should assume the intent was to quote the text and match it. In that particular scenario, you could also use the optional Suggest Improvement under “I see an additional problem that is not asked as part of the question” to ask the test author to clarify their intent.
• Customers can use _ in the instructions as a placeholder for a variable you will see in the app being tested. The _ placeholder can be used inside or outside of quotes, as well as for an unlimited amount of letters, words, numbers, special characters, etc. In addition, the customer may use a single underscore (like _ ) or multiple underscores strung together (like _____ ) to represent one single placeholder, to ensure this is visible.
• We also recommend that customers don't use placeholders other than _ but few other varieties do occasionally escape and are impossible to monitor. If it is clearly mentioned in the instructions that they will be using another symbol (e.g. “X where X is a number”, [], etc.) as a placeholder instead of, or in addition to _ this should still be considered acceptable both inside and outside of quotes, and is not a reason to fail the test as the instructions are clear.

How to apply this rule - General Principles

• Inside quotes = text or values must be matched exactly, even if you think there’s a typo inside. You must check that all relevant aspects (spelling, words, or punctuation) of the quoted text match what you see on the page.
• Outside quotes = don’t worry about exact matches. Do your best to execute the instructions and answer the question. Use SI (optional) to leave a note explaining discrepancies or leave other helpful suggestions. If what is on screen very closely resembles what is in the instructions, answer the question as best you can, and leave a SI to let the author know about the difference if, for example:
  • Instructions ask you to check for the name Eric, and you see Erica on the page
  • Instructions ask you to check for something in singular, and the page shows it as plural
  • Instructions ask you to look for Mr. Mountbatten and you see Mrs. Mountbatten
  • Instructions ask you to select All 14 features but you see 15 (“all” suggests some later got added and the test wasn’t updated)
  • Instructions say the login button is on the right but the same login button is on the left
  • Instructions say to click on the yellow login button, but the login button is green
    • If the color is mentioned in the ACTION portion of the steps, then pass with SI. If the color is mentioned in the QUESTION portion and the color is clearly wrong, then you should report the step. Otherwise, if it’s similar (such as yellow/orange) then leave SI to give the customer feedback.

• Unfinished quotes (see next section), answer as best you can, and leave a SI.
• Typos (see next section):
  • Inside quotes: assume that’s what the test author wants, as mentioned.
  • Outside quotes: the SI category “I see an additional problem that is not asked as part of the question" is best - and remember that SI is optional.

Special cases

As mentioned, this is one of the rules that are very easy to get wrong, despite the guidelines we offer. These are some situations that require special attention.

• The instructions ask Has the “Save Password?” pop-up disappeared? Remember that the instructions are asking to verify that the text is no longer present so you should answer the question accordingly.
• The instructions ask Do you NOT see “Sign Up”? Answer the question accordingly to verify that the text is not present.

Text in quotes to describe Images, Icons or other Graphics: it may be best to use the SI category I see an additional problem that is not asked as part of the question. Explain that in this particular circumstance, it would be best not to use quotes since the quoted object isn’t textual, and therefore can’t be matched exactly, and then move on with your test. Here are some examples of situations where you’d want to apply this logic:

• The instructions tell you to Look for the "pencil" icon, but you see an icon that looks like a pencil instead of the word “pencil” or Click on the "save" button but the button is a floppy disk icon with no text. Our glossary of testing terms includes common icons used in interfaces. → Answer YES and leave a SI to let the author know it’s better to describe the icon and where to find it, since it’s not a word that appears on the page.
• The instructions say Look for the "Rainforest" logo, but you see a single logo and there’s no “Rainforest” text on it. → Answer YES, assuming and leave a SI to let the author know that you assume it’s what they want but it’s better to describe the icon and where to find it, since it’s not a word that appears on the page.
• The instructions say Do you see the text "Welcome" in "ITALICS" on the page? You do see the word “Welcome” in italics but are confused about the second part. Assuming the word is exactly as described in quotes, → Answer YES and leave a SI to let the author know the formatting doesn’t have to be in quotes, if what they really care about is “Welcome”. They may want to consider removing quotes from less important things that won’t be an actual textual match, like ITALICS.

Incomplete quotation marks (i.e. started a quote but didn’t close it): ignore matching since the quotes are incomplete, and it’s hard or impossible to know where something ends or begins. Answer the question asked to the best of your ability, and leave a comment using the SI category These instructions are hard to understand. Explain that the incomplete quote made it hard to understand if you were meant to adhere to the quote rule or not. Here are some examples of situations where you’d want to apply this logic:

• The instructions say Look for the column header "Draft Due Date in the spreadsheet.
• The instructions ask Do you see a blue Save’ button?

Typos

• In the quoted portion: assume the typo is intended, and do not use Suggest Improvement. Instead, answer Yes or No depending on if there’s an exact match between the quoted typo and what you see within the app.
• In unquoted text: If the meaning of the step is accurate, you should use the SI category I see an additional problem that is not asked as part of the question. Alert the customer to the typo in the unquoted portion of the instruction or app, and move on with your test. Here are some examples of situations where you’d want to apply this logic:
  • The instructions say Click the red "Delete" btton. - remember that the question is asking you to click on a “Delete” button so you should answer the question first and foremost, and leave SI for the typo in ‘btton’
  • The instructions ask you to test a drop-down menu in the navigation bar, but you notice a typo in a separate portion of the app (e.g. the welcome banner).

Unique scenarios where quotes are used but an exact match is not needed. Sometimes TAs use quotes to emphasize or highlight text, you must read each instruction carefully and never proceed without fully understanding the instruction and the question.
If the text is used as an example or a reference to further clarify the instructions, it should be considered similar to screenshots rule and an exact match is not required. If quotes are used and it’s made specific by using the phrase for this example or for our example, an exact match would be required.

Consider some of these scenarios:

• The instructions state, Please enter a valid email address, for example, “any@email.com” but please create your own. → Do not enter any@email.com because you were asked to create your own random email address. You can create one using _@e.rainforestqa.com
• The instructions ask Are you taken to a page with the header “You still have _ days left of reading” OR “You still have _ free days of reading”? → Both of these phrases are in quotes, but you should confirm an exact match of at least one. Remember that the underscore is a placeholder for a variable.
• The instructions say, Click any of the dropdown headers at the top. Ex) “Tester Community”. Did a list of subcategories appear? Ex. “Forum”, “Support Docs”, etc. → Instead of the example, “Tester Community” you can also select “Jobs” or your profile icon.
• The instruction state, For our example, use “5” → This can be read as For our example use 5 and you should use 5 as instructed.
• Instructions state, Press “Enter” on your keyboard → You are not being asked to match the text “Enter” on the client’s platform, you should press the Enter/Return key on your keyboard.
• The question asks, Do you see "43" listed? (NOTE: the number may be different, that is ok) → There is an expected difference so what is shown on the page may not match exactly. You should consider what is added in the note to make a decision.

Minor exceptions

Do not report jobs for differences in capitalization. This rule has been around since 2016 and your work will be rejected if you do so. If everything is working properly, answer YES and move on with the test.

Do not report jobs for using the placeholder _ within quoted text. It’s understood that _ won’t exactly match what you see within the app. If everything is working properly, answer Yes and move on with the test.

Do not report jobs when automatic formatting takes place. There are automatic formatting practices often applied in online applications when entering data such as security numbers, telephone numbers, dates, credit card details, etc (eg. you enter “123456789” and the number formats to “123-456-789”). Some Customers may not take this into account when using quotes in their instructions. If everything is working properly, answer Yes and move on with the test.

A note on Suggest Improvement for Quotes and Placeholders

While we recommend the use of SI (and clients welcome it if it’s helpful), it is not required. If you choose not to use SI, follow the instruction to the best of your ability and answer Yes or No accordingly.