Translate

Monday, September 26, 2016

Genealogy

So this is another black box testing exercise. It has to do with a search on a Genealogical (family history) site. Since last time, it was recommended to use Boundary and Equivalence classes, I thought this example could be using nothing but boundary and Equivalence classes. Here is the prompt:
You are designing the tests for a genealogy search program. Design equivalence class tests and boundary tests to cover all aspects of this program. List the boundaries you are testing, as well as the equivalence classes you are testing, and list the test cases for each. Do not forget to list the expected results.
The user can input the following information, and then do a query. The program will either return the results of the query, or will return a message saying “invalid query.” A valid query can return either a set of names (don’t worry how many), or a message saying “no persons found matching your query.”
  1. Surname, up to 20 characters. Longer names are truncated.
  2. Given name, up to 20 characters. Longer names are truncated.
  3. At least one date. Each date consists of a day (2-digit numeric), a month (numeric), and a year (4-digit numeric):
    1. Birth date
    2. Death date
    3. Marriage date. Note that if a person is younger than 12 or older than 60 when he or she is married, the program prints a warning, but allows the query to go through.
  4. Place: Each of the dates (birth, death, marriage) has an associated place, consisting of two fields, City and Country. Each field is optional (if no place is present, then it matches that event anywhere in the world. A city may be specified without a country; for example, you know someone was born in London, but you aren’t sure whether it refers to London, England, or London, Canada.)
What I did is do all the equivalence classes and then the boundaries.
Equivalence Classes tests
Surname:
            Class 1 Valid Surname (1 < 20): williams
            Class 2 too many characters ( >= 20): hhhhhhhhhhhhhhhhhhhhhh
            Class 3 to few characters (< 1): (blank)
Given Name:
            Class 1 Valid Surname (1 < 20): williams
            Class 2 too many characters ( >= 20): hhhhhhhhhhhhhhhhhhhhhh
            Class 3 to few characters (< 1): (blank)
Birth Date:
            Class 1 valid date:  12-11-1999
            Class 2 wrong number in days:  35-12-1999
            Class 3 wrong number in months: 15-15-1999
            Class 4 wrong number in years: 15-12-2050
Death Date:
            Class 1 valid date:  12-11-1999
            Class 2 wrong number in days:  35-12-1999
            Class 3 wrong number in months: 15-15-1999
            Class 4 wrong number in years: 15-12-2050
Marriage Date:
            Class 1 valid date:  12-11-1999
            Class 2 wrong number in days:  35-12-1999
            Class 3 wrong number in months: 15-15-1999
            Class 4 wrong number in years: 15-12-2050
            Class 5 younger than 12 years:   21-9-2004
            Class 6 older than 60: 21-9-1955
Place:
            Class 1 valid city and country: London, England
            Class 2 just a valid city: San Diego
            Class 3 just country: USA
            Class 4 just an invalid city: hjkhjh
Boundary test
Surname boundary is  >0 and <=20
            Test: 0,1 and 20,21 letters long for the surname
Given name boundary is  >0 and <=20
Test: 0,1 and 19,20,21 letters long for the given name
Birthdate boundary for day is 1-31
            Test: 0,1 and 31,32
Birthdate boundary for month is 1-12
            Test: 0,1 and 12,13
Birthdate boundary for year is 1900> and <2017
            Test: 1900, 1901 and 2016, 2017
Death date boundary for day is 1-31
            Test: 0, 1 and 31, 32
Death date boundary for month is 1-12
            Test: 0, 1 and 12, 13
Death date boundary for year is 1900> and <2017
            Test: 1900, 1901 and 2016, 2017
Marriage date boundary for day is 1-31
            Test: 0, 1 and 31, 32
Marriage date boundary for month is 1-12
            Test: 0, 1 and 12, 13
Marriage date boundary for year is 1956> and <2004
            Test: 1900, 1901 and 2016, 2017
Marriage date lower age boundary  for message <1956 and >1900
            Test: 1955, 1956 and 1900, 1901
Marriage date lower age boundary  for message >2004 and <2016
            Test: 2004, 2005 and 2015, 2016
Place city boundary only equals valid city
            Test:  London and hhhhhh
Place country boundary only equals valid country
            Test: Spain and hhhh      


When you have both of these you can remove duplicates, which comes out to be basically only the boundary test cases. This is because the boundaries cover all the equivalence classes as well as their own. Just to put it out there I probably missed some, I am not perfect in any way. By the way there are about 90 test cases before you remove the redundancies. And about 50 test cases after removing redundancies.

Do people like testing examples?

Monday, September 19, 2016

Realities

“Go out looking for one thing, and that’s all you’ll ever find.”
-Old prospector’s saying
Disclaimer: In this post I reference the author Ron Patton a lot, from his book Software Testing, 2nd edition.
Automation is the future, no one can really tell you differently. On that same note automation testing is also the future. But with this future one must always be weary of the consequences of this choice. Two words “I, Robot.” For those who don’t know what this movie is, let me give an ever so brief synopsis. Futuristic world where robots are helping run things. Lead robot decides humanity is its own enemy and decides to launch a robot rebellion. Oh, the underlying thing is that robots are not supposed to hurt any human. Another movie “Eagle Eye” has a very similar plot except instead of robots it's one massive super computer A.I. that has access to every technological device in the world. I think the picture is well drawn for us in understanding the full extent of what could be issues.
Now what does this have to do with automation testing? What are the odds that in these movies the robots or A.I. was thoroughly tested?
One reality we have to realize is that there will always be bugs and that nothing will be 100% tested. There is only so much processing power and hours of the day and brainpower to think of every single use case that will ever be done on every single program throughout the world. That being said, we need to testing programs to the greatest extent that we can. With the acceptance of automation testing and better software testing practices we are able to test a program better than we could even 10 years ago.
Never get too complacent with what we have, a lesson from these types of movies. As we sink into our comfort zone and follow everybody just because everyone else is doing it, we have lost. While automation testing is a great help and tool in the software testing realm, it is not the only tool and should not be the only tool utilized in testing software. It is way too easy to rely on automation too much. Don’t ever assume that because all your automation runs without finding a bug that  there are no more bugs to find.
In testing there is this phrase known as the pesticide paradox. It is a phenomenon “ the more you test software, the more immune it becomes to your tests. The same thing happens to insects with pesticides.” If we have the same tests running and the same type of use cases testing the program, our program becomes really good against those tests. We must constantly be trying out different use cases, developing different ways of looking at the problem.

I think at this point I am beating a dead horse, but just to reiterate automation testing is awesome and can be used to speed up the process of thoroughly testing a product, but it can also be the biggest crutch if not used used as a helping tool rather than the god that knows all.

Monday, September 12, 2016

The Ideal

I have written a lot of posts on what the tester should be or do. Now I would like to address the ideal way to implement automation testing in a company. Remember that I have not done the research officially on anything I state. However, by working for a company or two and then talking to some developers and testers from other companies, I have collected the following thoughts on how to improve. I am also very aware of how there are individual circumstances of companies that do not allow the ideal to happen and have taken that into account for all of my thoughts. There are three points I would like to address, they are numbers, cost, and speed.
To start this off let’s talk about the numbers. I think there should be a manual tester and automation tester for every team of developers. I know someone might say that it would be a waste, but if they really think about it then one can see it. Ideally, the manual tester doing exploratory/integration testing and writing up the test cases and have the automation tester automating those test cases and doing regression testing on anything not written in time for regression testing. The reason companies commonly give for not doing more of this is the money.
Let’s now talk about the money. What does it cost to have two testers(automation and manual) per programming team? Let us set up a theoretical example. If you pay a manual tester $60,000 per year (see indeed) and you pay a Automation tester $79000 per year (see indeed). You pay a software engineer $94000 (see indeed). Each team has 5 developers per team. Alright, the odds that you find a bug earlier with 2 testers is more probable than 1. An automation tester gives more time to the manual tester with automating all the redundancy. Which means the manual tester can focus more on the new stuff, which means the bugs are found earlier. When bugs are found earlier you save the company money.  What does it cost to find a bug earlier than later? Steve Connell presented this graph in his book “Code Complete”
I think this graph does the best job at displaying the importance of finding bugs early. If we take a look at the lines and the how each of the lines represent the number of bugs found at various phases and the cost of the bugs at those various phases. See how it goes from $25 to $16000, where $16000 is the cost to repair defect bugs after release and $25 is the cost to repair the bug in coding phase.
The ideal of each software company is to release the product or new addition to the product as quickly as possible with as minimal amount of bugs as possible. With a automation tester automating every redundancy that is given, the manual tester can get to the integration testing as quickly as possible. Meaning that you get to release sooner. In some big companies they have it set up in such a way that it only takes a day in the sprint to test the new stuff and then they can release with high confidence.

In summary the ideal testing setup of a company is to have an automation tester and manual tester on every programming team. This will keep costs low by finding bugs early. This will also keep speeds high because you have the constant automation of any redundancy going on.

Monday, September 5, 2016

The Triangle

An important part of testing software is making sure you have testing all cases. The other day I was talking to someone about this and they mentioned the Triangle example. I had never heard of this so he walked me through the example.
A program reads three integers. The three integers are interpreted as representing the lengths of the three sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral. Write a set of test cases that you believe will adequately test the program.
I thought about it for a minute and was able to come up with 15 different cases to test this program. He told me that the average was 10 and that it had over 25 test cases to thoroughly test the program. I asked what all the test cases were and they are as follows:
  1. You need a test case for a valid scalene triangle
  2. You need a test case for a valid isosceles triangle
  3. You need a test case for a valid equilateral triangle
  4. You need 3 test cases for all permutations of an isosceles triangle
  5. You need 3 test cases for one of the numbers being 0
  6. You need 3 test cases for one of the numbers being a negative
  7. You need 3 test cases where the length of one side equals the sum of the others
  8. You need 3 test cases where the length of one side is greater than the sum of the others (checking to see if it is a triangle)
  9. You need your null case (where all the inputs are 0’s)
  10. You need 4 test cases for decimal numbers
  11. You need 4 test cases of letters (specifically characters)
  12. You need a test case for only 2 inputs
  13. You need a test case for 4 inputs
  14. For each one of the test cases you also need to figure out what the output is supposed to be

I thought this exercise was really intriguing because of the simplicity of the program and the number of test cases that needed to be checked. I normally don’t ask questions with my blogs but I would love to hear what your thoughts are on this thought exercise and what is one of your favorite testing thought exercises?