ABAKAS - Catherine Powell

We got an error message in our logs the other day that said:divbr //divdivOperation '' not permitted/divdivbr //divdivAfter a bit of digging, we discovered that we're logging in ASCII, and for some reason this was attempting to do something that could not be represented in ASCII. When we finally got debugging to show us something, it was just junk that happened to render as a weird symbol./divdivbr //divdivThis defect is now known internally as "the operation a href="http://en.wikipedia.org/wiki/Love_Symbol"formerly known as/a a href="http://en.wikipedia.org/wiki/Prince_(musician)"Prince/a."/divdivbr //divdivbr //divdiv(For those of us playing along at home, the underlying issue was attempting to use the wrong portion of memory at the wrong time, with an odd result.)/divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-1815376546173632706?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/T3lZIOc3DXs" height="1" width="1"/

I recently started a new project. It's greenfield, very exciting, and best of all, it's my idea. Unfortunately, this also means that whatever specs need to exist, well, I'd better get to work on creating them!divbr //divdivFor the basic idea, we've set up a a href="http://basecamphq.com/"Basecamp/a project, and I'm putting stories in there. The stories are high level, because we've all worked together for long enough we can communicate in shorthand. We can say, "let's build user login based on email and password" and understand that implies session timeout, hiding everything behind the login unless we specifically mention otherwise, etc. And then there are the nitpicky little details. What are the requirements for password strength? What are the error messages the user should see when they fail to login?/divdivbr //divdivFor those, we're communicating through code. Yes, code can be communication, too./divdivbr //divdivI'm a tester, so we're communicating through tests for these types of little details. After all, I'd have to write these tests anyway. So I just write them a little earlier, check them in (failing), and the person writing login simply makes them pass. For the record, I didn't come up with this idea. It's been around for years calleda href="http://en.wikipedia.org/wiki/Test_Driven_Development" test driven development/a and a href="http://en.wikipedia.org/wiki/Behavior_Driven_Development"behavior driven developmen/at (and probably several other things)./divdivbr //divdivI've learned a few things over the years that are helping to make this a success, though. It's far from guaranteed./divdivbr //divdivCommunicate through code when:/divdivullibThe audience is engineers./b No code is really English* so use code only when communicating with others who are comfortable in code - developers, testers, some analysts./lilibThe requirements are small and concrete./b If there is a lot of interaction or a bigger picture, then express it in some way that describes the bigger picture. This is almost certainly not a single test or a series of tests. Use tests to communicate password strength requirements; don't use them to define an interface between two systems (that one probably wants a data flow diagram to make it intelligible to others)./lilibYou're not co-located. /bWhen you're in Boston and the other person on the project is in San Francisco, you can't talk, and writing things down in code is easier than writing code and also something else to communicate all the details./lilibYou want one location for information./b You will eventually have the information in code anyway, in the form of implementation and tests. If you only put it in code then you'll save some time trying to maintain and reconcile multiple copies of the information./li/uldivbr //div/divdivDon't communicate through code when:/divdivullibYou're talking big picture. /bVision is a large, shimmering, wonderful thing. Express it in words, drawings, videos, and other things that can show your excitement. Code is many things, but it doesn't convey excitement./lilibYou need to communicate with outsiders./b If they can't see your code, then they can't get information from it./lilibYou need to communicate with non-coders./b I don't speak German to people who don't speak German; they're simply not going to understand. The same thing applies to speaking code to people who can't read code./li/ul/divdivbr //divdivCommunicating through code is one of those things that sounds like a great idea. Often, it is a great idea. Sometimes, it's a pretty terrible idea. As with any communication, consider your audience, think about why you're communicating, and then decide if code is an appropriate communication mechanism./divdivbr //divdivbr //divdiv* I had a conversation recently with James Bach and a few others about this specific topic. Code isn't English, no matter how many English words is uses and how much it really does kind of look like English. There are far more rules and restrictions in code than in English (or any other human language). Credit goes to James for helping me think about this one a little more./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-6612986219831753525?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/T7mLxt9X_28" height="1" width="1"/

I have a client who is starting to look at the performance of their product. They've spent a good amount of time building the product, adding features, creating documentation, etc., but they haven't done anything explicitly to make it faster. And now it's time to make it faster.divbr //divdivA group of us working on the "performance improvement" project got together to talk about how we'd start going this, and the first question that came up was this:/divdiv"Well, which configuration are we going to do?"/divdivbr //divdivThis is a legitimate question. This software can be deployed in two different modes and several different sizes. Which one should we try to make faster first?/divdivbr //divdivHere's a little secret from every single performance optimization effort I've worked on: It doesn't matter. If you haven't done any performance optimization at all, your first attempts will almost certainly help everything. It's like a global setting "turbo = true" in your code, that first round of performance enhancements./divdivbr //divdivOf course no one intentionally writes slow code, and we don't intentionally design it to be slow. But inefficiencies creep in. Maybe they're the fast way to implement something, maybe they're the easy way to implement it, Maybe it's a more junior developer who doesn't yet know that you don't do a linear search here because this data structure can get large. Whatever the cause, there's probably a bit of sloppiness, and fixing that will help performance across the board./divdivbr //divdivPick a configuration to work on, absolutely. That will help the performance team produce comparable results and provide some focus. But don't stress too much over the choice. At least at the beginning, you're likely to improve more than the thing you're focusing on. Later on this will change, but for as long as it lasts, accept the turbo button you're enabling in your product./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-3196759958580662567?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/F3-BiUngtOo" height="1" width="1"/

div style="text-align: left;"If I had to write a script for testing an elevator, when I got to the section where I had to push buttons, I would write something like this:/divdivbr //divdivspan class="Apple-style-span" style="font-family:'courier new';"def pushFloorButtons(floor)/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" pushButton(floor)/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" confirmDoorsOpenOn(floor)/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"end/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"br //span/divdivspan class="Apple-style-span" style="font-family:'courier new';"for 1..10 do |floor|/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" pushFloorButtons(floor)/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"end/span/divdivbr //divdivbr /divdivMy script would never have anticipated the elevator I got on today:/div/div/divdiva href="http://4.bp.blogspot.com/_hrCjqEFBjyE/THhGbpU_yYI/AAAAAAAAAOg/TEnEEzKd32A/s1600/08272010099.JPG"img src="http://4.bp.blogspot.com/_hrCjqEFBjyE/THhGbpU_yYI/AAAAAAAAAOg/TEnEEzKd32A/s400/08272010099.JPG" border="0" alt="" id="BLOGGER_PHOTO_ID_5510231584852134274" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 300px; " //a/divdivbr //divdivAnd yes, I did get off at 2.5. It looked a lot like 2. Just another parking garage./divdivbr //divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-363373194179299056?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/Ps2sT70lOwE" height="1" width="1"/

divLet's say we have a new engineering leader - a QA lead, a support manager, or a development manager. One of the things that usually happens when someone is promoted to lead is that there is a lot more available information. There are more meetings that involve other groups. There are email threads that expose information previously unseen./divdivbr //divdivA few examples:/divdivullithe new QA Manager now sees all support tickets automatically/lilia new tech lead for a product now gets an email describing marketing plans for that product/lilithe new development manager goes to a cross team weekly status meeting involving product management and several other development teams/li/uldivNow we have someone with a lot of new information and a feeling that they need to "step up" and "show they can do this job". This combination can lead to a dangerous new attribute:divbr //divdivbThey always have an opinion./b/div/div/divdivbbr //b/divdivYou an tell this is happening because all of a sudden your newly minted manager is replying to every email thread and speaking up in every section of every meeting, even when those things are at best tangentially related to his area of expertise./divdivbr //divdivFor example, the new tech lead gets cc'd on a communication indicating this release is going to be formally launched at a major industry conference. He replies, "What percent of our intended users are going to be at that conference? Do we know how we're going to measure if this is the right conference so that we can connect to next year's conference?" The question is valid, and almost certainly well-intentioned, but the tech lead is way out of his knowledge area at this point. In addition, the tone of the response is probing and almost aggressive; sure, it's "just a question", but it's a question that indicates doubt, with connotations of "I doubt your decision". If it happens once or twice, then it's no big deal - just someone learning about the new things to which he has been exposed. If it happens frequently, then there's a problem./divdivbr //divdivThis kind of probing questioning outside a person's core area is a sign of a lack of trust./divdivbr //divdivWhether or not our new tech lead actually doesn't trust marketing to do its job (or sales, or support, or other dev teams), frequent responses and offers of opinions and doubts on topics outside his core area certainly makes it look like he doesn't trust his new peers./divdivbr //divdivMy advice to the new manager is this:/divdivbr //divdivSit back and watch for a while. Keep in mind where you came from; that's your core area about which you will almost always have opinions, and that's probably why you're here. But in things that are new to you and that are outside your core area, just watch and listen. You don't have to have an opinion yet - you just have to learn./divdivbr //divdivOver time you'll figure out how to tell when a decision is good or not in these new areas. You'll also discover how to tell when the outcome of a decision matters and when it's not important enough to raise a fuss over. Then - and only then - it's time to start speaking up./divdivbr //divdivShow trust. It's not necessary for you to weigh in on everything. Let everyone else do their job, too, without your interference./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-5815860414918966361?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/2qk8uCnzC5c" height="1" width="1"/

I fell into a discussion on the software-testing mailing list last week and over the weekend, about software metrics. One assertion that was repeated a few times was the idea that "we need metrics because management wants them".divbr //divdivLet's examine that. Why does "management" want a metric? (By the way, anyone with Lead, Manager, or Director in their title: you're officially management. Welcome to the party.)/divdivbr //divdivHere's the first thing I learned about metrics when I joined the ranks of management: the number isn't what matters. What matters is the information within and around that number that lets me make the decision I need to make. It's a sound bite and when done well it conveys an entire tone and history that I can easily consume./divdivbr //divdivbA metric is simply information distilled into a manageable chunk./b/divdivbr //divdivSo if you don't like providing metrics like "number of showstoppers found per release" or "defects per thousand lines of code", that's fine. Find another way to provide "management" with what they need./divdivbr //divdivIt's rather like the "quants" who use detailed statistical analysis of past market information in an attempt to predict future market movements. They're really just creating metrics with a whole lot of math behind them. And those metrics are for one thing only: to predict the likelihood of future events. In the "quant" case, they want to predict whether stock X will increase in price. In our case, we want to predict whether product Y will ship on the currently-targeted date with Z level of acceptable post-release work for handling any problems./divdivbr //divdivWithout using metrics, then, how do we tell management what they need to know?/divdivbr //divdivLet's take as a given the following:/divdivulliany number the team can measure and is tied to their "worth" (bonus, likelihood of remaining employed, continued donut Fridays) is a number the team will eventually figure out how to game/lili"management" wants a measurement that isn't reliant on a single person's "gut" or ability to divine status and release readiness. I don't want my "magic release predictor" to leave the job because then I'm really out of luck./lilimeasurements are proactive, taken and identified prior to a release decision/li/ul/divdivbr //divdivNotice that there are certain things we have specifically not excluded: /divdivulliqualitative (i.e., non-numeric) metrics/lilithe reliance on our skills and expertise. /li/ulMetrics can have room for humans; it's not all numbers and systems (or it doesn't have to be)./divdivbr //divdivHere's a metric:/divdivEvery week the engineering (dev, test, prod mgmt, etc) team takes an anonymous vote: "would you release the software today?". When that number passes some threshold (say, 85% yes), then the engineering team feels pretty good about the product, and that metric turns to go. (You could even baseline this by asking that same team, "Knowing what you know now, would you have released the last version? the version before?". After all, some people are so risk averse, they'll probably never say yes. Others are so risk accepting, they'll probably say yes immediately.)/divdivbr //divdivHere's another metric:/divdivWhat was the total cost of defects found in the field for the past few releases (say, number of defects, plus cost to handle, plus % of customers affected)? Is that number going up or down? If it's going up, then we've got a problem. If it's going down, then this team is doing better and better. Let's keep doing what we're doing./divdivbr //divdivAre these metrics that can be gamed? Sure - see the above assumption that any metric can be gamed. Do they have risks? Sure - and so does everything we do. If we want to stop hiding behind numbers, then let's do it. But let's recognize what management needs - information so that they can make the best decision they know how to - and let's figure out how to distill our knowledge, thoughts, and opinions down for them. /divdivbr //divdivbWe're communicators, and a metric is an opportunity for communication. /b/divdivbr //divdivUse your metrics as a distillation technique; that's all they are - a sound bite. It's lossy, but it's still communication. Embrace the chance to provide information./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-3211304236930991187?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/qhl8z-Wjv-E" height="1" width="1"/

J Michael Hammond happened to mention on a mailing list that he had created a heat map and in doing so had semi-intentionally popularized the phrase "pukey green".divbr //divdiv"Pukey green" is a igreat/i descriptor./divdivbr //divdivWe've been around long enough to know that "red" is bad or failing. "Green" is good, or passing. That permeates our builds, our tests, our status reports - it's everywhere./divdivbr //divdiv"Pukey green" is great for that precise reason: it's green, so it must be good. But it's pukey, which sounds bad. This is the verbal equivalent of a shrug./divdivbr //divdivI'd use "pukey green" to describe something that just doesn't feel right, even though so far it looks okay. On the outside, it's working fine, but there's something I can't yet put my finger on that makes me not trust it./divdivbr //divdiv"Pukey green" items are the ones that you can't say why you shouldn't ship them, but you just know you're going to get a lot of customers complaining about it later./divdivbr //divdivThanks, good Mr. Hammond, for making my day, and giving me a really vivid way to describe that "yes, but..." feeling./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-766453723399688979?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/frQrkLMtHv4" height="1" width="1"/

divWe're working on a rather large project at the moment. It's fun, it's from scratch, and it's basically engineering driven, since our internal engineering team will be using it. We have enough ideas for it that we could build for at least six to nine months./divdivbr //divdivThere's no way we can wait nine months before we ship it. We need to be using this thing within about two months. Earlier would be better./divdivbr //divdivSo as with any product, we have to trim down our feature set. We'll have to ship something first, and then add to it. This is causing a lot of angst because we know that our consumers will spend weeks building out file formats and changing it later is going to be hard. We don't want to have to ask our customers to scrap the work that they've done, but we can't make them wait until it's perfect./divdivbr //divdivWe use the Don't Preclude Principle:/divdivbYou don't have to build it yet; you just have to leave yourself room to build it later./b/divdivbr //divdivWe don't actually have to build everything we're going to wind up needing. We simply have to decide what the format will look like when we're done. Then we build pieces of it, and make sure we leave holes for the pieces we aren't doing yet. This gives us a much cleaner upgrade path. In the case of this file format, it means that right now we have something that simply ignores the values in some places. Later, when we build those features, we'll figure out what to do with the values. It's a simple thing, and only a tiny amount of work now to ignore something, but it'll prevent us from having to migrate formats later./divdivbr //divdivThink first, trim second, build third. Don't preclude your future ideas, but don't worry about building it all now. Just think far enough ahead that you know what you're not doing as well as what you're doing./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-3444686907750945616?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/Iok2vJwC7gE" height="1" width="1"/

I've written before about a href="http://blog.abakas.com/2009/03/red-flag-words.html"words that trigger/a a href="http://blog.abakas.com/2009/10/magic-words.html"certain reactions/a. There is an additional class of words and phrases that we need to be very careful about: generalizations.divbr //divdiviWhat We Say:/i "It fails randomly"/divdiviWhat We Mean:/i "I ran this test three times. It failed twice and passed once, and I really don't understand why."/divdiviWhat's the Problem:/i Random means without known pattern, really. Using the word "random" here is an overgeneralization; you need more attempts to see if there's a pattern or not./divdivbr //divdivdiviWhat We Say:/i "It's slow"/divdiviWhat We Mean:/i "It ran at 50 MB/s. I was hoping for 100 MB/s."/divdiviWhat's the Problem:/i Slow means something different to everyone. Sometimes a little slow is okay, but very slow is not. Sometimes slow is what you think, but your expectations are what is in error. Here it pays to be more precise: "about half the speed I usually see in X other test"./div/divdivbr //divdivBe careful of generalizing too soon. Many of us engineers - developers, QA, support, etc. - thrive on detail. Skipping that detail and going to generalization simply makes you sound either lazy or panicky. So when you're using words that are imprecise:/divdivullirandom, /lilislow, /lilisometimes,/liliweird,/li/ul/divdivask yourself if you can characterize it more specifically. You'll get a lot more help from the people you're working with that way./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-4923288666310611529?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/9aXyjSCIjtc" height="1" width="1"/

div style="text-align: left;"I've been in a situation at work recently in which we've had to provide a test plan to our customer. This isn't a big deal; we're happy to provide our test plan. Interestingly, we sent this to project management for review before it went out, and got a bunch of feedback that wording should be changed in various areas to make the test plan less scary to customers. Here's the thing:/divdivbr //divdivThe test plan is isupposed/i to be scary./divdivbr //divdivYour test plan describes all the nefarious things that you will do to the software before those nefarious things can happen in the field. You're going to put it under heavy load to see where it cracks. You're going to subject it to repeated hardware failures. You're going to feed it bad parameters and missing parameters. Sure, you'll test the happy path, too./divdivbr //divdivYour test plan teaches your software and your developers just like training teaches a soldier. You certainly hope these things don't happen in the field, but you train for them just in case./divdivbr //divdivMake sure your test plan looks like this:/divdiva href="http://4.bp.blogspot.com/_hrCjqEFBjyE/TGV6FjemMHI/AAAAAAAAAOY/P3Ec7crdaJU/s1600/nicholson.jpg"img src="http://4.bp.blogspot.com/_hrCjqEFBjyE/TGV6FjemMHI/AAAAAAAAAOY/P3Ec7crdaJU/s400/nicholson.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5504940355371806834" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 260px; height: 267px; " //a/divdivbr //divdivbr //divdivbr //divdivDon't let it look like this:/divdiva href="http://4.bp.blogspot.com/_hrCjqEFBjyE/TGV6Fvn-n8I/AAAAAAAAAOQ/RvBNl9UAgtA/s1600/rw-soldier1-nobelt.JPG"img src="http://4.bp.blogspot.com/_hrCjqEFBjyE/TGV6Fvn-n8I/AAAAAAAAAOQ/RvBNl9UAgtA/s400/rw-soldier1-nobelt.JPG" border="0" alt="" id="BLOGGER_PHOTO_ID_5504940358632382402" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 243px; height: 400px; " //a/divdivbr //divdivbr //divdivFor better or for worse, a scary test plan is comforting to your customers. It tells them that you're prepared for and thinking about when things are going as expected, but also that you're prepared for when things don't go as expected. Life happens and sometimes it's scary; let your test plan prepare you for it./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-4246763640797419052?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/Eul5UO2Cses" height="1" width="1"/

We have a problem this morning at the office in the lab. We have DNS entries that are basically laid out like this:divhardware-type-1: 10.0.21.1XX/divdivhardware-type-2: 10.0.21.2[01-75]/divdivhardware-type-3: 10.0.21.2[76-254]/divdivbr //divdivIn general this works rather well; it sure makes it easier to figure out what type of machine you have, where it is in the lab, etc./divdivbr //divdivUnfortunately, we just installed our 76th machine of hardware-type-2. And we're out of IPs in our DNS scheme. Oops. Time to change our DNS scheme. It's not really a big deal, and it's not really an error. After all, when we set up this scheme, we had a total of 5 machines of hardware-type-2 and didn't think we'd ever get more than about 50./divdivbr //divdivAll we have here, as my colleague John calls it, is a case where "good planning meets contact with the enemy"./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-5482610321682177284?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/uGgB9YG_8ac" height="1" width="1"/

divSometimes when you're working with someone, you see a project start to go a bit wrong. Maybe it's not proceeding as quickly as it should, or maybe it's someone trying something new. As a manager, how do you know when to step in?/divdivbr //divdivThis is the heuristic I use:/divdivbr //diva onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_hrCjqEFBjyE/TGBgPAf7gRI/AAAAAAAAAOI/t4l8KZX9zgo/s1600/SpeakUpDecisionTree.png"img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 285px; height: 400px;" src="http://2.bp.blogspot.com/_hrCjqEFBjyE/TGBgPAf7gRI/AAAAAAAAAOI/t4l8KZX9zgo/s400/SpeakUpDecisionTree.png" border="0" alt="" id="BLOGGER_PHOTO_ID_5503504555594973458" //adivdiv style="text-align: left;"span class="Apple-style-span" style="color:#0000EE;"ubr //u/span/divIt's pretty simple: if they're going to be publicly embarrassed, if they're never going to finish, or if it can't be fixed, speak up. Otherwise, keep your mouth shut and let the person doing the project learn./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-3566821100679713817?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/puDbNlMO2-E" height="1" width="1"/

divSometimes good things happen, and how we deal with the good changes is just as important as how we deal with the negative changes./divdivbr //divdivCase in point: I just promoted someone on my team to QA Lead. (Congratulations, John!) He's been around for a few years as a QA Engineer, and when the slot opened up, he stepped up and asked for it. He's going to do really well at it: all in all, this is a positive change./divdivbr //divdivIt's still a change. And we still have to be a bit careful about it./divdivbr //divdivThere are a few things we need to look out for here:/divdivulliMaking the transition (aka getting through the change itself)/liliBroadcasting the change/liliProviding autonomy/liliProviding support/li/ul/divdivSo what did I do? In this case, I left. I went away for three days on vacation and didn't answer any emails or phone calls that the new QA Lead should be able to handle. Instead, I just forwarded everything his way. We made the transition by refusing to allow the old ways to work; he did everything the QA Lead should be doing, and I did none of it. Because he was publicly responding to requests, we broadcasted that this was his area now. And by simply forwarding things without telling him what to do, he got autonomy in deciding how to respond. Privately, any questions he sent me, I answered, thus giving him a safety net in deciding how to respond./divdivbr //divdivIn a nutshell, we need to make the change actually happen. In order for the new way to take effect, the old way needs to be put behind us. So we took away the old way (me) and left only the new way (the new QA Lead)... publicly. Privately, we established an open line between the old way and the new way, so that the new way had all the information he needed./divdivbr //divdivWhen we make a change, even a positive one, we need to address the change directly to make sure it goes smoothly. Let's keep the positive, well, positive./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-5636907774477533552?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/CdDmzidUmRA" height="1" width="1"/

divWe use oracles when we test - things to which we can compare the behavior of our system. Usually an oracle is something we consider correct (just like the one at Delphi). Sometimes, the challenge of a test is identifying an oracle./divdivbr //divdivAnd sometimes there is no oracle that is right all the time. Sometimes the best oracle we have can only tell us if something is probabilistically correct (i.e., whether it's "plausible")./divdivbr //divdivFor example:/divdivLet's say we're testing a system and that system distributes data among machines according to the first letter in that data. Let's further say that the data is all letters and there's a lot of it - billions of entries./divdivbr //divdiv-------------------------------------------------/divdiv| 1: A-F | 2: G-N | 3: O-S | 4: T-Z |/divdiv-------------------------------------------------/divdivbr //divdivWe ran the data through and want to know if it got distributed correctly./divdivbr //divdivHow do we do it?/divdivbr //divdivThere are a lot of ideas we can try:/divdivullibgrep in each of the buckets!/b Here's a hint: at any volume, you're just going to hang up your computer - this one doesn't really work./lilibrun a sample data set through./b This one works for small scale tests but won't tell you if you've got a problem in the field or a subtle bug at volume. This is a great first test, but not an only test./lilibstatistics./b This one is powerful, although not deterministic./li/uldivLet's talk about creating an oracle from statistics. If, for example, each of our entries is an English word. We can look up the a href="http://en.wikipedia.org/wiki/Letter_frequency#Relative_frequencies_of_the_first_letters_of_a_word_in_the_English_language"frequency of the first letters of English words/a. That can be an oracle. Your letter counts should be about equal to the overall frequency./div/divdivbr //divdivBecause this is a statistics-based, or probabilistic, oracle, it won't be perfectly correct. You may be off by a half percent or a percent. The point is that over time and over enough entries, you should expect your statistics to about match your oracle. And thus you have a probabilistic oracle./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-4348655028409364158?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/i4QpzJNAlLE" height="1" width="1"/

div style="text-align: left;"I was flying home from CAST 2010 yesterday, and I found a bug. (Guess those testing conferences wear off on you!) I was on a two-part flight (Grand Rapids - Baltimore - Boston), and had upgraded to an exit row seat on one of them, at a cost of $20. The second flight was a standard seat./divdivbr //divdivThis is what the terminal showed me when it came time to pay for the upgrade:/divdivbr //divdiva href="http://1.bp.blogspot.com/_hrCjqEFBjyE/TFrXmoouCVI/AAAAAAAAAOA/koVhy4EExNo/s1600/08042010085.JPG"img src="http://1.bp.blogspot.com/_hrCjqEFBjyE/TFrXmoouCVI/AAAAAAAAAOA/koVhy4EExNo/s400/08042010085.JPG" border="0" alt="" id="BLOGGER_PHOTO_ID_5501946953529690450" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 300px; " //a/divdivbr //divdivbr //divdivbr //divdivFor the record, I was only charged $20. Go Airtran kiosk!/divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-8815750043330397782?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/q3mMl88HtVI" height="1" width="1"/

These days, many of us work in Agile teams or in integrated SCRUM teams. We're no longer the development team and the test team and the design team. Instead, we're one team working together to ship product. At least, that's what we tell people.divbr //divdivIf this is you, though, are you ireally/i a team? Is this really all one group going toward a single goal?/divdivbr //divdivThree simple questions will help figure this out:/divdivulliDo you assign tasks to individuals?/liliDo developers do test tasks when the team is crunched? Do testers help with system configurations when the team is trying to get set up for new development?/liliWhen a bug is found in the field, who gets yelled at and tries to figure out how to prevent that kind of problem from making it out to customers again? Your testers? Or your team?/li/uldivIf you're still playing "developer" and "tester" roles, then it doesn't matter whether you call yourself one team or two, you're still restricting yourself. You're still saying that you do your bit, the other guy will do his bit, and if we're all good and lucky, then good things will happen. That's not a team. That's a group of people./div/divdivbr //divdivA team is a group of people working toward the same goal. Some have more experience than others in different things, but if you're really a team, then each member will happily do whatever is standing between the team and release./divdivbr //divdivThe difference between a group and a team is very simple:/divdivbr //divdivbTeams succeed together and teams fail together. /b/divdivbGroups pass or fail, each individual alone./b/divdivbbr //b/divdivIf you want to be a team, be a team, not a group./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-3558910408512807739?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/HaDpVQVvqNw" height="1" width="1"/

I've been doing a lot of phone screening lately, and there have been some real doozies. I've heard all sorts of things:divbr //divdivi"I left my last job because everyone was incompetent but me. It went through two whole teams like this." /i/divdivWow - ego coming through! There are definitely incompetent people out there but to run into such a high proportion says that you either have truly terrible luck .... or that there's something wrong with your idea of competence./divdivbr //divdivi"Oh, my weakness is I don't do automation."/i/divdivThis is only strange once you know that the position is for a QA Automation Engineer./divdivbr //divdivi"I'm just looking for a job before my skills get too rusty."/i/divdivOn the one hand, this one is kind of sad. On the other hand, he's not doing anything to keep up or expand his skills, apparently, and that's a problem. I don't want to have to push for every new learning experience./divdivbr //divdivThere are a lot of decent resumes out there. Phone screens really weed a whole lot out, though. Be careful what you say; the hiring manager is probably listening./divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-1136651773641216801?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/iGw-D4Q4JMg" height="1" width="1"/

The saga of using Perl code to communicate via Jira's SOAP API. See a href="http://blog.abakas.com/2008/12/perl-soap-interface-to-jira.html"part I/a and a href="http://blog.abakas.com/2010/07/perl-soap-interface-to-jira-part-ii.html"part II/a here. In part I, we logged and updated issues. In part II, we were able to get status information about an issue. In our final part, we'll actually reopen the issue:divbr //divdivdivspan class="Apple-style-span" style="font-family:'courier new';"#####################################################################/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"# Reopen a Jira issue/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"#/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"# @param params{issue} The issue key (e.g., PERMA-12345)/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"# @param params{assignee} The intended assignee/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"#/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"# @return The jira issue number/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"##/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"sub reopenIssue {/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" my ($self, %params) = assertMinArgs(1, @_);/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" $params{issue } ||= '';/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" my %issue;/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" $issue{issueKey} = SOAP::Data-type(string = $params{issue});/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" $issue{assignee} = $params{assignee};/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" $issue{summary} = $self-_getIssueSummary(issue = $params{issue});/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" $issue{newStatus} = SOAP::Data-type(string = "3"); #reopened/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" my $result = $self-_request('progressWorkflowAction', 1, \%issue);/span/divdivspan class="Apple-style-span" style="font-family:'courier new';" return $result;/span/divdivspan class="Apple-style-span" style="font-family:'courier new';"}/span/div/divdivbr //divdivIt's pretty simple, in the end. We're using the span class="Apple-style-span" style="font-family:'courier new';"progressWorkflowAction/span call, which changes an issue from one status to another. We simply create our issue object with all the required attributes (in our case, summary, assignee, the new status, and the issue key) and then call progressWorkflowAction. This call will fail if the user doesn't have permission to make that transition, or if the transition doesn't apply (e.g., attempting to close a closed issue)./divdivbr //divdivAnd that ends the Jira and SOAP tutorial, at least for now. Good luck!/divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-6047400685340482494?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/3zh3Oyj4D_Q" height="1" width="1"/

divWhen I'm interviewing candidates, I like to ask a lot of questions and present a lot of different types of problems to see how the candidate thinks. There are some things that I look for in more junior candidates specifically. For junior candidates, I'm interviewing for potential more than for what they've done, and that shades what I ask./divdivbr //divdivb"You've been dropped in the middle of the Linux kernel, never having worked on kernels before, and asked to fix a thread deadlock. How do you go about it?"/b/divdivWith this question, I'm looking to see how the candidate learns. Our system is not particularly small, so I want to see how they approach it. Answers that imply they will try to learn the entire system before doing anything are generally not good; this isn't something you can hold entirely in your head without living in it for a while, and I want someone who's going to jump in. Incidentally, we use Linux, so this question isn't totally off base./divdivbr //divdivb"When you're working on a project in a team, how often do you check in? Any tricks or preferences there?"/b/divdivWith this question, I want to see how the candidate works in a team, and how the candidate approaches source control. Some candidates will go away for weeks or months and build a huge, beautiful, complete thing, and then have a heck of a time checking in because all the other code has changed in the meantime (this is a red flag). Other candidates check in near constantly, which means lots of reviews (also a red flag). Every team falls somewhere along this spectrum; I want to see if the candidate matches our rhythm, or is even aware there is a rhythm to match./divdivbr //divdivb"Your team of five has been handed a feature to build: you're going to build the crash recovery mechanism. How do you split up the work?"/b/divdivJunior candidates won't actually be dividing work, but they'll be on a team, and I want to know how they're going to work with that team. One danger of really junior candidates is that they mostly have experience working on school projects or isolated code areas, and often alone. Our company doesn't work that way, and we need to know that a candidate is going to be comfortable building part of a feature when the other part is being built by a teammate at the same time./divdivbr //divdivMost of these questions revolve around seeing how a junior candidate works in a team environment, since they likely don't have this much experience. We can teach coding skills, we can teach learning techniques, and we can teach development practices, but only to someone who is aware there is something to learn./divdivbr //divdivbr //divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-5871152413954264530?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/XYLPk2dvWEY" height="1" width="1"/

In testing, an oracle is something we look to that can define the "should" of an application. The oracle (or oracles) tell us what an application should do, how the UI ought to look, how the application ought to perform, etc.divbr //divdivOracles can take many forms:/divdivullirequirements specifications/liliscreen mockups/liliother (competing or complimentary) products/lilistyle guides (yours or someone else's)/lilisimilar or related functions in the same application/lilia previous or alternate version of the application being tested (think porting projects)/li/uldivWhen you start to test your application, one of your first jobs is to find the oracle(s). This will help guide your future tests, bugs identified, and behavioral expectations./div/divdivbr //divdivSometimes finding the oracle is easy; for example, if you're handed a requirements specification, then you've got at least one of your oracles. Other times, you'll have to get creative./divdivbr //divdivFor example, a friend recently approached me with a problem:/divdivbr //divdiv"We're processing files, and we discovered that on this file server with 573million files, we detected 19 million ZIP files. Is that reasonable?"/divdivbr //divdivThis is a problem in search of an oracle. It's infeasible at that scale to hand-check all of the files. We're not yet sure we can trust our detection software, though; after all, that's what we're testing. So what's our oracle?/divdivbr //divdivSeems hopeless. "Hopeless" is just another word for "needs creativity". We can do this./divdivbr //divdivOur problem revolves around the percentage of files on a file server that are of a certain type. There are two ways I think to go about this:/divdiv1. run one or more other file type detectors and see how much they agree with my results/divdiv2. find other file servers (or reports about other file servers) with a typical file type breakdown. Academic papers, storage vendors, and OS vendors are great sources for this kind of information./divdivbr //divdivWhat would you use for an oracle in this situation?/divdiv class="blogger-post-footer"img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/8341729382206275662-7575812683433857991?l=blog.abakas.com' alt='' //divimg src="http://feeds.feedburner.com/~r/Abakas/~4/VC6AFh03kWk" height="1" width="1"/