Specification by Example: a love story
~ Alister Scott
I attended a three day Advanced Agile Acceptance Testing/Specification by Example workshop in Melbourne last week run by Gojko Adzic, which I enjoyed immensely.
I took copious amounts of notes so I could regurgitate them here, but instead I decided to do something a bit different and write a story. It’s a story about how a team can apply what he taught us to improve their software development practices. It’s entirely fictitious (although I’d love to live in the Byron Bay hinterland!) and any resemblance to any real person or project is purely coincidental.
So here it is: Specification by Example: a Love Story (PDF). Enjoy.
25 thoughts on “Specification by Example: a love story”
Your story clearly shows the difference between basic automation scripts, Cucumber style specs and Specification by Example. I’ll be sharing this with my teams. Thanks for sharing with the community.
LikeLike
I like it :-)
LikeLike
Very Good Story (Y), the most interesting part is showing the evolution of from script to living documentation
LikeLike
Great illustration. But where was the love?! :)
LikeLike
Good point. I’ve added a final paragraph:
“The team had fallen in love in Specification by Example”
LikeLike
> I’ve added a final paragraph:
Yes you did. And it wraps up very nicely.
But shouldn’t it be:
The team had fallen in love *with* Specification by Example.
Or is this a US/Aus difference?
Thanks,
Brad.
LikeLike
Nice story, I like the way you develop the specifications, but I think you can go further. Your final solution requires you to read comments and then also translate them into a table. I think the following is much simpler:
Feature: Beautiful Tea Shipping Costs
Scenario: Australian customers pay GST
Given the customer is from Australia
When the customer buys something
Then the customer pays GST
Scenario: Overseas customers don’t GST
Given the customer is from overseas
When the customer buys something
Then the customer does not pay GST
Scenario: Australian customers can get free shipping
Given the customer is from Australia
When the customer buys lots
Then the customer gets free shipping
Scenario: Australian customers can get free shipping
Given the customer is from Australia
When the customer buys bugger all
Then the customer pays for shipping
Scenario: Overseas customers can’t get free shipping
Given the customer is from overseas
When the customer buys bugger all
Then the customer pays for shipping
Scenario: Overseas customers can’t get free shipping, even if they buy lots
Given the customer is from overseas
When the customer buys lots
Then the customer pays for shipping
all best
Andrew
LikeLike
Thanks for your comment Andrew.
I guess we’ll have to disagree. I dislike having multiple scenarios as you have to read through each one. I like the table as it shows you 5-6 key examples all together which you can easily compare and contrast.
The comments aren’t exactly required, and aren’t executed, but they clearly explain the examples in the table below.
I also noticed your scenarios don’t include the $100 value. I think that’s key, because whilst it is specific, it is the rule for free shipping. Without including this in the specification, you have to dig into the system code to work out this value, which is the thing I am trying to avoid.
Sure, if this $100 is updated you need to update your specs, but this have the opposite effect, if the $10o was inadvertently changed, then these specs would fail immediately, and you could easily see why.
LikeLike
Alister
Of course there is more than one way to skin a cat, and the table example does at first seem concise. However in general I think tables are error prone and harder to understand and generally overused.
I have to disagree with you about the comments above the table. They are essential without them the feature makes no sense.
Whether the $100 value should be in the spec is very context dependent. I left it out to highlight a different way of doing things. The context is very much about whether the business really wants/needs the exact value in a feature. If you do put the value in a feature, you still have to specify the value in the code somewhere so now you have two places where the value is defined (not very DRY). In addition when the the real value is changed deliberately and the application still works, the feature failing will be a false result. You have to balance the likelihood and consequences of each type of change.
Reading through the many smaller scenarios highlights, that there are things missing from the specification (for example exploiting the potential of free postage to encourage further purchases for an order). Tables make this harder to see. The smaller scenarios show that you actually have two or three shipping features here (overseas shipping, domestic shipping …). So you could consider a refactor to separate these different areas; having small simple scenarios makes refactoring easier. There are strong parallels here with the composed method pattern in coding. Additionally step definitions are far easier to write and re-use in these simple scenarios.
LikeLike
It seems ironic that you’re talking about DRY principles when your version of the examples continually repeat the words “Given the customer, When the customer, Then the customer”.
Reading repetitive scenarios like this is extremely tedious, which means people don’t read them properly, which means bugs slip in. That’s my experience anyway.
LikeLike
Having the value in there is useful for a number of reasons. It’s ‘food’ for the dev as they get to the writing of unit tests, it gives them an exact value, instead of having to hunt down a PO and say ‘wth is the dividing line between bugger-all and lots?’ It It allows the tester reviewing the story to add a few more rows and make sure the boundaries are covered, and it expresses the business rule of what exactly defines when someone gets treatment x or y.
It also allows for some discussion when the scenario is initially presented, such as “so, if someone has $99.99 then they don’t get free shipping? or are we going to round up anything above $99.00? Is the Tax included or is that before tax?
In short, at least as far as I’m concerned, when it comes to things like this, specific examples good, vague examples bad.
LikeLike
David,
Thankyou for reading my scenario’s. The repetition in my version is tedious, but that just indicates there is more work to be done.
As I pointed out there are actually several different scenarios being covered by the table example. When you expand that into the format I’ve used this becomes more obvious. Now we can improve readability further by dividing things up and refactoring. The following divides on shipping destination as an example but you could divide on GST/nonGST or buying lots/little
Feature: Overseas Shipping
Background:
Given the customer is from overseas
Scenario: Buying lots
When the customer buys lots
Then the customer pays for shipping
Scenario: Buying bugger all
When the customer buys bugger all
Then the customer pays for shipping
And now you have easy to read features and a place to further explore overseas shipping.
LikeLike
Alister, forgive me if my question seems obvious (I’ve never used cucumber), but how are things actually tested in the last phase? More specifically, is it:
a) using a black box, ie integration test on a test website deployed somewhere?
b) testing the internals (in this example, it would mean testing, for instance, the service that calculates shipping rate).
The question arises because while b) seems logical, we’re actually losing what was being tested in the selenium script. For instance, if you forget to put the “order” button, the service might be perfect but the system won’t work. On the other hand, if you’re in a) and still testing the system as a whole, then how do you refrain from having to do the brittle stuff like having to use xpath or css to select buttons or inputs?
LikeLike
I have enjoyed this love story and I have shared it with my team at work. In your conclusion you use the acronym “BA/SME.” I am guessing that SME is subject matter expert, can you please confirm this and give me a definition of BA?
LikeLike
Yes, SME = Subject Matter Expert and BA = Business Analyst. Thanks
LikeLike
Good stuff mate. Explained very well :)
LikeLike