QA Hates You

How can you tell an experienced automated tester from either an automated testing software solution vendor or someone who’s heard a buzzword dog whistle and is salivating on cue at the chance to work somewhere that’s included Automated Testing on a job posting?

An experienced automated tester is going to spend more time trying to tell you what automated testing isn’t instead of what it is. Because you’ve probably got a pie-in-the-sky you’re slicing to share for dessert.

Cue Trisherino:

I think people have a tendency to greatly underestimate the difficulty of writing a good GUI-automation suite. It’s not as simple as record and playback, and it’s not like building regular software.

I’ve seen experienced developers underestimate it many times. Inevitably, they end up getting very frustrated and complaining about how rubbish the automation tools are. And yes, the poor quality of automation tools is definitely part of the problem. Developers are used to using very polished tools, lovingly crafted by developers, for developers. Testers are used to using either bloated overpriced commercial tools, condescendingly oversimplified so that “anyone” can use them, or well-meaning but under-maintained free tools, struggling to keep up with the latest technologies.

What she said.

You know why developers are so hot for automated testing? They’re used to thinking they can push software around. Unlike some QA people who refuse to be intimidated by their obvious GENIUSSSS!

Frankly, I cannot listen to this song without thinking of a street gang. Did I say a street gang? I meant my QA team.

” target=”_blank”>

Love and darkness and my sidearm. Hey, élan, indeed.

And our southside? Conference Room 2, baby. Where the status meeting happens.

The Five Key Myths About HTML5 contains a lot of insight into market penetration and tipping points regarding new technologies, including browser market share and Flash version penetration.

There’s a tipping point where a technology reaches enough users to be worthwhile for designers and developers to use; however, the thesis is that HTML5 isn’t there yet.

But you can apply the lessons from the article to other technologies your organization might want to use.

You know what you need to add to your standard testing repertoire?

On a recent Monday morning, inside an unremarkable, low-slung building in Yonkers, N.Y., blue-coated technicians conducted lab work. Among their calibrated tools: Cheez Whiz, pig’s blood and Maine coon cat hair.

They’re testers for Consumers Union, the nonprofit publisher of Consumer Reports magazine and website whose ratings drive spending decisions on some 3,000 product models annually, from vacuums and lawn mowers to strollers, shower heads and smoothies.

That’s some boundary analysis I wish I could try.

In the lab, technicians are always developing ways to accelerate the wear and tear products normally experience over the course of years. Treadmills, for instance, get pounded repeatedly with a spinning metal drum imbedded with rubber green balls that mimic running feet. (The device’s nickname: Johnnie Walker.) Stainless-steel grills are stationed in a conditioning chamber and pelted with a salty spray to test for rust potential. Sometimes the goal is to mimic extreme scenarios, like an accident, in which bicycle helmets get strapped on a metal head form and dropped from various heights onto sharp and rounded objects.

I try to apply that same sort of creativity when I face an application.

Meanwhile, do you suppose mechanical engineers around the world post comments on the Consumer Reports Web site about how no one would do that? Probably not, because unlike software “engineers,” mechanical engineers have a sense of shame.

Don’t be held in contempt for Selenium discharges:

Patriot Coal Corp. was found in contempt of court by a federal judge on Wednesday and ordered to clean up selenium pollution at two mines in West Virginia.

I guess that’s not something a tester is going to do necessarily, but I don’t think it’s effective branding for the automated testing product.

I was rolling through some programs (that is, contest Web sites) the other day, and I hit upon a motherlode of misspelled error messages:

Click for full size

Click for full size

Click for full size

Some developers will make the case that the last one refers to the Your Birthdate field, but it’s still weird to read.

I’d like to point out that I did not have to troll a bunch of contests to find these three examples. This is three out of five I visited. In 60% of the Web sites I visited, the error messages contained misspellings and other problems (such as poor placement in the last case).

A user who encounters an error and then encounters your error message, if your error message is rife with misspellings or other problems, the user might think that the error was the fault of your crappy application and not his or her problem.

So you need to pay attention to your error messaging and display as much as any other feature of your application. Because if your error message isn’t part of the solution (that is, helping the user understand what’s wrong and how to fix it), it’s part of the problem.

The instructions for this database of teacher salaries in Missouri has an interesting example:

click for full size

The instructions? – To see results for people with the last name of “Null” type in Null*.

Hey, that’s always a good idea. What happens if I search for/enter null?

In this case, we discover that there are a lot of Nulls making good money teaching in the state of Missouri.

But I can tell a lot of thought went into writing that sample search term. Thought that could better have been spent on the title tag for the page.

Who hasn’t felt like this?

The key is to take it out on the software, brothers and sisters. It’s not just a vocation, it’s therapy.

Found on the Twitterverse: 10 things non-technical users don’t understand about your software:

If you are writing consumer software you have to understand that you and your average user have a very different level of understanding of computers. When you first start doing support it can be a shock to realize just how vast this gulf is. It doesn’t mean that your users are stupid, just that they haven’t spent the thousands of hours in front of a computer that you have.

This echoes the Roberta scenario that I outlined earlier. When your development staff and design staff get together to design something, they need to remember that not everycat is as hep as they are, dig? Complication, and deviation from other interface standards, will induce user error, and the worst bugs of all are the bugged users.

So your designers have constrained the input length on your application so you cannot enter more characters than the database can handle. If the developers force the string into all caps, have I got a nasty little trick for you. Ladies and gentlemen, the German eszett:

Also, the eszett or scharfes S (ß) is used. It exists only in a lowercase version since it can never occur at the beginning of a word (there are a few loan words starting with an s followed by a z (e.g. Szegediner Krautfleisch but that is not the same as the eszett which counts as one letter).

In all caps it is converted to SS….

There’s a new unicode symbol for the capital version, but a lot of old applications will still force that into an SS. So a word like confuße might get uppercased to CONFUSSE, and if you set the string to the maxlength, uppercasing it will blow that up.

To be honest, I did discover this when I was working on an application for a German customer and I (and only I of a team of far more seasoned QA people than I at the time) sought out the German alphabet to learn its vagaries.

I just ruined a little of my mystique, didn’t I?

However, if your application might possibly be localized to German, you have my permission to use this. Use this new power only for good. Strangely, though, QA good means evil to everyone else, but that’s not our fault.

It’s not as legitimate as the Indians making up a new symbol for the rupee, but a company in Michigan has made up its own punctuation and is happy to sell the character to you for $1.99:

A Michigan company announced the release of software Tuesday that introduces new punctuation to the typed word: The sarcasm mark.

Sarcasm Inc. of Washington Township said the SarcMark, which resembles an open circle with a dot in the center, can be installed on computers via a program that can be downloaded from sarcmark.com for $1.99.

It’s almost worth $1.99 just for how much it will annoy your developers.

(Link seen here.)

You’ve heard of the Twitter Fail Whale? Here’s the BBC Fail Clown:

Click for full size

Try harder to not fail again, or again you must face the creepy dead clown.

Remember when Google held a mock funeral for IE6, and the designers had a party?

Do you think they’ll do the same if Flash goes the way of the dodo?

I’m the last person on earth who wanted to believe Steve Jobs when he told Walt Mossberg at D8 that “Flash has had its day.” I took it as nothing more than showmanship when Jobs shared his thoughts on Flash and wrote that “Flash is closed and proprietary, has major technical drawbacks, and doesn’t support touch based devices.” After spending time playing with Flash Player 10.1 on the new Droid 2, the first Android 2.2 phone to come with the player pre-installed, I’m sad to admit that Steve Jobs was right. Adobe’s offering seems like it’s too little, too late.

So, who’s up for some Silverlight?

(Link seen here.)

More Motley Crue for you:

If you can shout at the devil, you can certainly convince that developer that a blue screen probably is a critical defect and not a medium, fix-if-we-have-time priority.

I’ve recently engaged to do some load testing oversight, and as such, this required some different asterisks for my estimate than my regular functional testing estimates. Some are applicable to load testing in general, but some are specific to the role I’ve taken as sort of a project manager wielding a prod over a third company that will provide the actual scripting and running of the tests.

Here are the asterisks I came up with, the torpedoes that could sink the project or, at the very least, the estimate:

• Problems with the environment or application require additional work. If the testing environment is not configured correctly or does not mirror the anticipated production environment, the load tests might require additional starting/stopping and run times.
• Functional defects impede test scripts. If the test scripts encounter functional defects, the load tests might require additional iterations after defects have been corrected.
• Functional workflow remains the same between load tests. This estimate anticipates that corrections to any defects/load issues found will not require refactoring or rewriting the test scripts. If the application changes in such a fashion, the test team will need additional time to adjust and retest the scripts.
• Communication between test liaison and test vendor runs smoothly.

What other factors would you add to this list? Why do I bother with these open-ended questions? Because I just like the sound of my own keyboard clicking (yeah, it’s an old school keyboard).

For a more positive spin on load testing efforts, see 7 Steps to Load Testing Bliss.

A quote in the back of a Baseline magazine got me to thinking.

When asked to characterize their ability to thwart internal breaches, only 34% of respondents are ‘very confident,’ but that response rises to 56 percent [sic] when respondents are asked about their ability to thwart external breaches. – Deloitte’s 2010 Financial Services Security Survey

Frankly, in the QAHY school of QA philosophy, any confidence is overconfidence. If you think you’re catching the bugs and are ready to open the bar, you’re probably missing something, and it’s not going to miss you.

MelBugai asked:

Anyone seen this software testing company before? I’m so very skeptical, http://www.testing4success.com/

A testing company that uses a number instead of a word in its name? Dubious. One looking for resources, and by “resources,” I mean “Volunteers”:

Click for full size

I could do actual work for you, and I get to include that on my resume? Let me think about that. Sounds about as much fun as testing an open source project, except with the knowledge that you’re making money from my servitude.

I just have to ask you one question: if you’re hiring experienced tester who are working for free even though you’re paying for it, what sort of service do you think you’re getting?

Here’s a SQL Injection Cheat Sheet for you.

Remember to check your form fields for these bad dogs when you can.

(Link courtesy the Twitterverse.)

I’m really starting to fall out of love with Firefox. It’s becoming a resources hog, makes coming out of hibernation take a long time, and constantly doesn’t play nice with Flash these days. Additionally, I get this particular condition frequently:

Click for full size

This occurs if you have a large number of tabs open, taxing the browser, and then you use the keyboard to open the Bookmark menu (ALT, then B, then down arrow to start moving down it). Sometimes, it throws open the file menu and then leaves it open when you expose the Bookmarks menu.

On a severity scale, this is a low/cosmetic defect, but it just adds to the sense that the application is turning to crap.

How many cosmetic bugs does your organization think it can leave in a released application before the user thinks it’s crap? I bet the real number is far lower than your dev team thinks it is.

I found this hanging out of the receipt printer at a gas pump:

You know, if you’re copyrighting your single line test messages, maybe you’re overdoing it a little.

Does anyone want to guess whether this printer was untested or whether concerns about the inappropriate nature of this copyright were undiscovered or ignored?

I think I’ll go with tested, but undiscovered.