My Testing Methodology

Testing is always something I did naturally, not that I would have characterized it as such back then. Not in any sort of formalized approach but as a very curious child. My parents encouraged learning with the carrot and stick method and that did not bother me as I enjoy learning anyway. This led up to my birthday in 3rd grade.

Third Grade Birthday Present

As a third grader I asked for a $200 present for my birthday with another $150 in accessories, and most parents would generally say no to something like that and they should. My parents however bought it. To my surprise and enjoyment as that is what I asked for. If I were a cartoon that morning when I opened that present my eyes would have busted out of my head, my jaw falling through a few floors and drool that would have wiped out a few low-lying islands.

I’m sure you are all thinking of the game boy, super Nintendo or Sega but you will all be wrong, and honestly I could probably give anyone 2 hours to guess the right answer and nobody would still be unable to guess it.

Answer: a microscope.

Not a kiddy version made of plastic and weighs nothing. This was a microscope with 2 replaceable eye pieces and 3 built-in magnification lenses that weighed probably 7+ pounds. Complete with a box of empty slides, a small assortment of pipettes for fluids, slide covers to protect the sample and a set of samples that I can use to examine what I was looking at.

My thought process was something like:

How do you get to understanding of microbiology without a microscope?

OK so books would have been easier but an real microscope is so much more fun than a book especially for a third grader.

I wanted to see a few samples of bacterium and then try to find them in the local lake. I wanted to look at the germs that were on insects and lizards. I wanted to be able to look at my blood and to be able to see the virus that was kicking my butt when I became sick.

No self mutilation just to examine my scrapes before bandaging them up. I was a kid, enough said.

OK How Does this Apply to Testing?

Now most of you are now asking yourselves how does this apply to testing? I wanted to make my own generalizations of the different pathogens, viruses and bacteria and view living plant cells working then see if I can locate them in the real world. This act is the definition of testing. Granted, it would have been testing common knowledge to scientists around the world but it would have been new to me.

In retrospective, I think that present set the precedence for my lifelong learning obsession.

Present Day

Now how has that view and process changed since I have a few more years of experience and knowledge?

  1. Computers are much easier than biology.
  2. Much easier to start with the work other people have done first
  3. Should have asked for a game console
  4. Don’t worry I would not have gotten one

Besides the switch to computers I am still the same stubborn person who learns to much.

Hard Coded Tests? NO

Any tests that I write should have the least amount of hard coded test data in them as possible. This seems odd as typically there is a lot of static data that is used in the tests to validate the system. I don’t believe in any of it.

Well sort of. I prefer to write my tests as generic as possible and have as few parametrize inputs as possible.

How I achieve that goal is the fun part. I use randomized inputs as much as possible. These inputs are confined to what characters are allowed in the field. Failures are tolerated. If possible I like to use a random string generator that gets a different subset of allowed characters that should pass along with a few tests that should not pass and make sure they fail correctly.

Failing correctly is also a hard part to accurately account for. That is where you have to go out of your way and make the methods throw new exception types so that it is easier to debug later.

Element Not Found Exception does not tell you a lot.

How to Extend That Laziness?

At this point you are thinking that I have a lot of tests that I support and that it is hard to maintain. Well fret not! For the functional level tests that everyone has does the login process differ between the different types of devices that are out there. In reality, most likely no.

So I ask you: Are they separate tests for you?

Well why are they?

The functional test for logging into your application is always the same. There maybe a menu button that needs to be clicked, which is a bad user experience, but the test is still testing the same function and the test should be written identically. Now there will be more tests written for validating the navigational menu, but that is a separate test.

The current trend for accomplishing having the same user experience and interface is responsive design. There are a few ways that responsive design can be accomplished there is either mobile first design or there is graceful degradation. In my experience mobile first designed sites are easier to write tests first as navigation on the mobile devices end up being a primary thought and are developed better than graceful degradation.

So my methods that are used for testing also account for any responsive features that are incorporated into the application. This allows for less tests that are maintained as the actual steps are abstracted from the testing steps.

Abstract Thinking on Operating Systems and Browsers

In that same vein of thinking, why should I create specific tests per each browser and operating system combination that is possible?

Answer: It shouldn’t.

The functions should work regardless of the operating system or browser or even browser version. This is dependent on the requirements of the system under test, but if the combination is supported it should work.

Note all of those features do take sometime to build, but they all support my last piece of my testing methodology.

  1. I want to be able to easily create the test
  2. I want anyone to be able to quickly glance at the test and easily understand what the steps are and what the test is validating.
  3. I want everyone to be capable of creating tests quickly and easily

For experienced automation testers the test code is almost childish. That is how I want it. I do not want to have to sit around all day to figure out what does that one method do, I don’t want a new learning curve to add in a new test, I don’t want to be the only one capable to write the tests.

That last point is synonymous with the acceptance testing frameworks or business driven design testing.

TLDR: Make tests easy to write
Andrew Krug's Picture

About Andrew Krug

Automation consultant helping you deliver greatness effectively.

New York, USA