Software Testing

TestStorming - A Collaborative Approach for Rapid Test Design

Randy Rice's Software Testing & Quality - Fri, 10/02/2015 - 21:56
Thanks to everyone who attended my webinar today on TestStorming(TM)!

Here is the recorded presentation:
Here are the slides:

Here is the article on my website:

And, finally, here is the link to the METS spreadsheets I mentioned today:

Have a great weekend!

Categories: Software Testing

QA Music: I Am The Fire

QA Hates You - Mon, 09/28/2015 - 04:17

Halestorm for a Mondy morning.

Categories: Software Testing

Test Environment Responsibilities

Eric Jacobson's Software Testing Blog - Wed, 09/23/2015 - 09:51

We never have enough of them.  They never mirror production.  They never work.

My opinions at my current company:

  1. Who should own test environments?  Testers.
  2. Who should build test environments?  NOT testers.  DevOps.
  3. Who should request test environments?  Testers.
  4. Who should populate, backup, or restore the test data in test environments?  Testers.
  5. Who should configure test environments to integrate with other applications in the system?  NOT testers.  DevOps.
  6. Who should deploy code to test environments?  NOT testers. Whoever (or whatever) deploys code to production.
  7. Who should control (e.g., request) code changes to test environments?  Testers.
  8. Who should create and maintain build/deploy automation?  NOT testers.  DevOps.
  9. Who should push the “Go” button to programmatically spin up temporary test environments?  Testers or test automation.

Fiddling with test environments is not testing work, IMO.  It only subtracts from test coverage.

Categories: Software Testing

QA Music – I’ve Had This Dream Before

QA Hates You - Mon, 09/21/2015 - 04:38

Pop Evil, “Footsteps”:

Categories: Software Testing

QA Music – I’m A Savage/It’s Automatic

QA Hates You - Mon, 09/14/2015 - 04:41

Shinedown, “Cut the Cord”:

Categories: Software Testing

Is your team getting the most out of user stories?

The Quest for Software++ - Wed, 09/09/2015 - 07:41

Have you struggled to split user stories into small but valuable chunks? Do you have problems prioritising stories or getting commitment from business stakeholders on what they want to achieve? Do you have issues deciding when a story is done or how many other stories you really need to achieve a business objective? Are you managing large amounts of stories that are problematic to estimate, prioritise or plan for?

If any of these problems affect your team, join me for a two day hands-on workshop in London, bring your product owners and business sponsors to learn how to get the most out of user stories. The participants will learn how to ensure that things coming into their work stream are defined well, split to be small enough but valuable, and achieve the big benefits of adaptive planning and that you can expect from great user stories.

The workshop happening on October 21-22 in central London. If you book before September 20th, use the promo code earlybird to save 200 GBP. Click here more info and to register.

QA Music: The Choice Is Ours To Make

QA Hates You - Tue, 09/08/2015 - 04:54

In the 21st century:

I've thoroughly enjoyed Over the Top, a collection of alternate histories of World War I.

— John Farrier (@JohnCFarrier) September 7, 2015

@QAHatesYou I had to look that one up. I prefer Stallone's squid arm attack.

— John Farrier (@JohnCFarrier) September 8, 2015


Adding one and two star movies to Netflix watch list. Thankfully still haven't sunk to Van Damme or Segal movies.

— Jim Holmes (@aJimHolmes) September 8, 2015

Jeez, I need a jump front kick with guitar and some direct-to-video 80s action films, stat.

“Winner Takes It All” by Sammy Hagar. From the Stallone film Over the Top. Which was not, in fact, direct to video; it was a major theatrical release.

Categories: Software Testing

GTAC 2015 Coming to Cambridge (Greater Boston) in November

Google Testing Blog - Sun, 09/06/2015 - 09:27
Posted by Anthony Vallone on behalf of the GTAC Committee

We are pleased to announce that the ninth GTAC (Google Test Automation Conference) will be held in Cambridge (Greatah Boston, USA) on November 10th and 11th (Toozdee and Wenzdee), 2015. So, tell everyone to save the date for this wicked good event.

GTAC is an annual conference hosted by Google, bringing together engineers from industry and academia to discuss advances in test automation and the test engineering computer science field. It’s a great opportunity to present, learn, and challenge modern testing technologies and strategies.

You can browse presentation abstracts, slides, and videos from previous years on the GTAC site.

Stay tuned to this blog and the GTAC website for application information and opportunities to present at GTAC. Subscribing to this blog is the best way to get notified. We're looking forward to seeing you there!

Categories: Software Testing

GTAC 2015: Call for Proposals & Attendance

Google Testing Blog - Sun, 09/06/2015 - 09:27
Posted by Anthony Vallone on behalf of the GTAC Committee

The GTAC (Google Test Automation Conference) 2015 application process is now open for presentation proposals and attendance. GTAC will be held at the Google Cambridge office (near Boston, Massachusetts, USA) on November 10th - 11th, 2015.

GTAC will be streamed live on YouTube again this year, so even if you can’t attend in person, you’ll be able to watch the conference remotely. We will post the live stream information as we get closer to the event, and recordings will be posted afterward.

Presentations are targeted at student, academic, and experienced engineers working on test automation. Full presentations are 30 minutes and lightning talks are 10 minutes. Speakers should be prepared for a question and answer session following their presentation.

For presentation proposals and/or attendance, complete this form. We will be selecting about 25 talks and 200 attendees for the event. The selection process is not first come first serve (no need to rush your application), and we select a diverse group of engineers from various locations, company sizes, and technical backgrounds (academic, industry expert, junior engineer, etc).

The due date for both presentation and attendance applications is August 10th, 2015.

There are no registration fees, but speakers and attendees must arrange and pay for their own travel and accommodations.

More information
You can find more details at

Categories: Software Testing

If I Followed Directions, I’d Still Be Looking

QA Hates You - Tue, 09/01/2015 - 13:00

The instruction in the installation wizard is Press Execute.

There is, of course, no Execute button in the wizard. Nor, for the last four or five decades, is there one on the keyboard.

Lessons, of course, include:

  • Check to make sure your application’s text matches the interface, including instructions and error messages.
  • Test the installer.

QA Music: QA Tells You The Odds

QA Hates You - Mon, 08/31/2015 - 04:28

According to Michael Jackson, they’re not good.

That’s “You Can’t Win” also known as the Crows’ Anthem from The Wiz.

Categories: Software Testing

A Ping Pong Defect

QA Hates You - Fri, 08/28/2015 - 03:13

While sitting in a restaurant, I saw that the closed captioning on the sports program was frequently emitting a string of random characters in the speech:

Forensically speaking, we could assume that this bug occurs in one of the following places:

  • The software transliterating the text to speech. That is, when the software encounters a certain condition, it puts a cartoon curse word into the data.
  • The network transmitting the information. That is, the transmission of the data introduces garbage.
  • The device displaying the transmitted information. That is, the television or satellite box that introduces the captions into the picture inserts the junk every two lines or so.

Okay, I’ll grant you the fourth option: That the broadcasters were actually cursing that much. However, given that the FCC has not announced fines daily, I’m willing to say that it’s nonzero, but unlikely.

The beauty of a defect that could occur almost anywhere, between disparate parts of the product and across different teams and technologies, means that it could ultimately be nobody’s fault. Well, if you ask one of the teams, it’s one of the other team’s fault.

You know, a little something squirrelly happens, you log a defect, and the server, interface, and design teams spend megabytes reassigning the defect to each other and disclaiming responsibility. It drives me nuts.

So what do you do? You find a product owner or someone who’ll take charge of it and pursue it across fiefdoms or who’ll put the screws to the varied factions until it gets fixed.

Because everybody’s got something they’d rather be working on than somebody else’s problem. Even if it’s everybody’s problem.

Categories: Software Testing

Apparently, The Screen Size In Production Is Different

QA Hates You - Tue, 08/25/2015 - 07:28

At least, I hope this is the result of the screen size being different in production than it was in the spec.

Otherwise, the implication would be that the interface was not tested.

Remember when you’re testing that the spec or requirements are merely suggestions, and you should go afield of your testing matrix as often as you can.

Categories: Software Testing

Windows 95 Nostalgia

Alan Page - Mon, 08/24/2015 - 12:48

I feel like today’s a good day to share a few stories about my first few months at Microsoft, and the (very) small part I played in shipping Windows 95.

My start at Microsoft is a story on its own, and probably worth recapping here in an abbreviated form. I started at Microsoft in January 1995 as a contractor testing networking components  for the Japanese , Chinese, and Korean versions of Windows 95. I knew some programming and even a bit of Japanese (I later became almost proficient, but have forgotten a lot of it now). I also knew, for better or for worse, a lot about Netware and about hardware troubleshooting, and that got me in the door (and got me hired full time 5 months later).

Other than confirming that mainline functionality (including upgrade paths) were correct, there were two big parts of my job that were unique to testing CKJ (Chinese, Korean, Japanese) versions of Windows. The first was that at the time, there were a dozen or so LAN cards (this was long before networking was integrated onto a motherboard) that were unique to Japan, and I was (solely) responsible for ensuring these cards worked across a variety of scenarios (upgrades from Windows, upgrades from LanMan, clean installs, NetWare support, protocol support, etc.). One interesting anecdote from this work was that I found that one of the cards had a bug in its configuration file causing it to not work in one of the upgrade scenarios. Given the time it typically took to go to the manufacturer to make a fix and get it back we decided to make the fix on our end. Because I knew the fix (a one liner), I made the change, checked it in, and that one liner became the first line of “code” I wrote for a shipping product at Microsoft.

The other interesting part of testing CKJ Windows was that Windows 95 was not Unicode; it was a mixed byte system where some (most) characters were made up of two bytes. Each language had a reserved set of bytes specified as Lead Bytes, that indicated that that byte, along with the subsequent byte were part of a single double-byte character. Programs that parsed strings had to parse the string using functions aware of this mechanism, or they would fail. Often, we found UI where we could put the cursor in the middle of a character. The interesting twist for networking was that the second byte could be 0x7c (‘|’), or 0x5c (‘\’). As you can imagine, these characters caused a lot of havoc when used in computer names, network shares, paths, and files, and I found many bugs testing with these characters (more explanation on double-byte characters, along with one of my favorite related bugs is described here).

While I didn’t do nearly as much for the product as many people on the team who had worked on the product for years, I think I made an impact, and I learned so many things and learned from so many different people.

(potentially) related posts:
  1. One of my Favorite Bugs
  2. My last fifteen years (part 2)
  3. Twenty Years…and Change
Categories: Software Testing

Suggestions For Your QA Mission Statement

QA Hates You - Mon, 08/24/2015 - 05:27

Victor Frankenstein’s creation speaking in Mary Shelley’s Frankenstein pretty much sums up my testing approach:

I will revenge my injuries: if I cannot inspire love, I will cause fear; and chiefly towards you, my arch-enemy, because my creator, do I swear inextinguishable hatred. Have a care: I will work at your destruction, nor finish until I desolate your heard so that you shall curse the hour of your birth.

Nineteenth century curses are the best.

Here’s a statement of work from Frankenstein himself later in the book:

My present situation was one in which all voluntary thought was swallowed up and lost. I was hurried away by fury; revenge alone endowed me with strength and composure; it moulded my feelings and allowed me to be calculating and calm, at periods when otherwise delirium or death would have been my portion.

Categories: Software Testing

Announcing DaSpec — awesome executable specifications in Markdown

The Quest for Software++ - Sat, 08/22/2015 - 19:47

It’s my great pleasure to announce the immediate availability of DaSpec v1.0, the first stable version ready for production use. DaSpec is an automation framework for Executable Specifications in Markdown. It can help you:

  • Share information about planned features with non-technical stakeholders easily, and get actionable unambiguous feedback from them
  • Ensure and document shared understanding of the planned software, making the definition of done stronger and more objective
  • Document software features and APIs in a way that is easy to understand and maintain, so you can reduce the bus factor of your team and onboard new team members easily
  • Make any kind of automated tests readable to non-technical team members and stakeholders

DaSpec helps teams achieve those benefits by validating human-readable documents against a piece of software, similar to tools such as FitNesse, Cucumber or Concordion. The major difference is that DaSpec works with Markdown, a great, intuitive format that is well supported by a large ecosystem of conversion, editing and processing tools. Run and play with the key examples in your browser now, without installing any software, to see what DaSpec could do for you.

DaSpec’s primary target are teams practising Behaviour Driven Development, Specification by Example, ATDD and generally running short, frequent delivery cycles with a heavy dependency on test automation. It can, however, be useful to anyone looking to reduce the cost of discovering outdated information in documentation and tests.

For more information on what’s new in version 1.0, check out the release notes.

The first version of DaSpec supports automation using JavaScript only. We plan to port it to other platforms depending on the community feedback. Get in touch and let us know what you’d like to see next.

CI Test Automation Part 1 – Feature Branches

Eric Jacobson's Software Testing Blog - Fri, 08/21/2015 - 15:16

My dev teams are using Git Flow.  A suite of check-in tests run in our Continuous Integration upon merge-to-dev.  The following is the model I think we should be using (and in some cases we are):

  1. A product programmer creates a feature branch and starts writing FeatureA.
  2. A test programmer creates a feature branch and starts writing the automated tests for FeatureA.
  3. The product programmer merges their feature branch to the dev branch.  The CI tests execute but there are no FeatureA tests yet.
  4. The test programmer gets the latest code from the dev branch and completes the test code for FeatureA.
  5. The test programmer merges their test code to the dev branch, which kicks off the tests that were just checked in.  They should pass because the test programmer ran them locally prior to merge to dev.
Categories: Software Testing

Improving ViewState Tokens

LoadStorm - Fri, 08/21/2015 - 10:13

ViewState tokens are a common performance issue I come across while helping customers. I’ve found some information on how to get performance improvements by modifying ViewState tokens.

What is a ViewState token? It is ASP.NET’s way to ensure the state of the page elements sent and received match. It is an encoded string sent out to the user which gets sent back to your server in a POST request. The main problem with this kind of token is that it needs to represent every form element on the page. This gets large when complex elements such as a drop-down menu is in the form. Countries, companies, customers, or other database tables with many entries are good drop-down examples. In some cases it can double the size of your page. The largest ViewState I’ve seen was 300,000 characters long. The generated HTML for that page was at least 600,000 characters. That’s a lot of response text.

Performance Impacts

As the ViewState grows larger. It affects performance in the following ways:

  • Increased CPU cycles to serialize and to deserialize the ViewState.
  • Pages take longer to download because they are larger.
  • Large ViewStates can impact the efficiency of garbage collection.

According to one source I found:

ViewStates work best when serializing basic types such as strings, integers, and Booleans. Objects such as arrays, ArrayLists, and Hashtables are also good with those basic types. When you want to store a non-basic type, ASP.NET tries to use the associated type converter. If it cannot find one, it uses the expensive binary serializer. The size of the object is proportional to the size of the ViewState. Avoid storing large objects.

Improvement Options

From much of what I read there are only three options:

  1. Remove ViewStates when they’re not needed.
  2. Remove any unnecessary elements in the form.
  3. Compress the ViewState to reduce the data transferred.

I rarely see ViewStates used that are not needed, and in some cases a few elements can be removed from the form. Compression is often the best improvement for the end-user experience. Keep in mind that compressing and decompressing the ViewState causes more CPU work. This could have a poor impact on performance at scale.

Compress a ViewState whenever it is above a particular size:

More suggestions on compressing the ViewState:


The post Improving ViewState Tokens appeared first on LoadStorm.

Extra Naughty Strings

QA Hates You - Thu, 08/13/2015 - 07:40

As the child of two former United States Marines, I already know plenty of naughty strings; however, a client pointed me to this resource on GitHub: Big List of Naughty Strings.

It is a pretty comprehensive list that includes a couple things not in my standard bag of tricks.

Until now.

Categories: Software Testing

Word for the Day: Recrudescence

QA Hates You - Wed, 08/12/2015 - 05:31


Recrudescence is the revival of material or behavior that had previously been stabilized, settled, or diminished. In medicine, it is the recurrence of symptoms in a patient whose blood stream infection has previously been at such a low level as not to be clinically demonstrable or cause symptoms, or the reappearance of a disease after it has been quiescent.

I don’t normally mention the mouthfeel of words, but this one has it.

I’m looking forward to using this when reopening bugs whose behavior recurs.

Also note I plan to mispronounce it as re-CRUD-escence.

Categories: Software Testing