XP2016 – Does the Test Automation Pyramid matter and who really cares? – Panel Discussion

Standard

At XP2016 there was a panel discussion focused around the “Test Automation Pyramid”

Testpyramid

The discussion was facilitated using the “Fish Bowl” technique. The starting panel was:


Left to Right: Elisabeth Hendrickson, John Fergusson Smart, David Evans, Simon Brown, Steve Freeman

The discussion kicked off with the following are questions:

So what did we talk about?

The 90 minute discussion covered the following:

  • All models are wrong, some are useful
  • Does it matter? Maybe
  • Who cares? Not sure
  • It sucks because it ignores other types of automation that support testing. Automation can extend beyond Units, Integration and UI. I thought of Richard Bradshaws comments about “Automation in Testing”
  • The Exploratory Testing bubble that is sometimes placed on the top is stupid
  • What is the definition of a unit? No real clarification here
  • What is the the definition of an Integration Test? Tumble weeds……
  • People treat items higher up the pyramid as second class citizens. Tests at the top still add value although the feedback is traditionally slower
  • Pyramid can sometimes be used to assign responsibility. Testers focus on the top(E2E), developers do the bottom(Unit), someone somewhere does tests at the middle (Assuming they know what an Integration Test is)
  • The pyramid is based upon an old “3 tier architecture” model where the Front End was difficult to test and could only be done using “In Browser” tools such as WebDriver. In todays world, this is not the case. With front end frameworks such as AngularJS the front end can be broken down into Unit Tests.
  • What is Testing? Talked about the difference about Testing v Checking. Most of the discussion was about Checking but we should also think about Testing which is not currently part of the Test Automation Pyramid. The Pyramid is not about testing.

  • Uncle Bob Martin talked about how the Test Pyramid is commonly inverted, with most of the business rule tests implemented through the Front End.
  • There was talk about “Plumbing” and the Test Automation. I forget what we discussed exactly.
  • Did the Test Automation pyramid come from the “Food Pyramid“?

food_pyramid

  • The focus of the discussion towards the end of the panel discussion moved into documentation
  • Uncle Bob Martin compared a well written code based to an “Encyclopedia”; It contains a robust collection of state transitions but its hard to search.
  • The Test Automation Pyramid exists because we like to talk in Metaphors.
  • Final comment;
    • Horizontal axis = Amount of Tests
    • Vertical axis = Amount of Pain

Thats me ^

So….what do you think? Leave a comment below.

 

Advertisements

3 thoughts on “XP2016 – Does the Test Automation Pyramid matter and who really cares? – Panel Discussion

  1. bvccaneer

    Hi,
    I think #xp2016 guys were lucky to have this panel discussion 😉
    I would say “it depends” on the context … i like the questions what are definition of unit and integration and how exploratory ‘bubble’ sneaks into TEST AUTOMATION PYRAMID. btw. do you have answers from discussion?
    If it comes to 2 main questions:
    Does it matter?
    Who cares?
    ‘Maybe’ & ‘Not sure’ are correct answers. It all depends on company/employees coulture and awarness of this concept and their commitment to it.

    Like

  2. So, pretty much universal dislike for the pyramid there (and I’ve had some of these discussions myself with projects)… but, how can we better represent test automation in the modern age… and without giving the impression that everything can automated?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s