Everybody Tests

Success is a journey, not a destination. So stop running. As a tester, one of my most repeated routines includes the following: downloading a new build from our CI server uninstalling the build from my test devices (note the plural) launching the app for a specific set of tests or investigations.  This happens many times a day, each day, all year. This routine also has many variations based on whether I just want to clear app data and relaunch it, whether I have a different method to launch the app in mind (different starting activity, different intent extras, etc). On top of all of this, there are details about the build which are relevant to any resultant actions I take in our task/bug tracking suite such as build number, device information, etc. This kind of jumble is cumbersome and time-consuming to manage manually. The whole reason to manage all of that process and information is to maintain a consistent practice of reliable reproducible tests and rich environment an...

"I told you. I wake up every day, right here, right in Punxsutawney, and it's always February 2nd, and there's nothing I can do about it." ~ Phil Connors, Groundhog Day The life of a tester involves a lot of repetition. In even ideal environments where you can focus mainly on exploration (rather than rote, unthinking execution of endless, soul-crushing test case minutia) you will still find the need to repeat some of the same tasks many times. Some of those tasks are mechanical and serve an operational purpose (e.g. deploying a fresh environment in which to play). These you can script away to great effect (and I'll detail some of how I do that on mobile devices in a different post). Other tasks are more for communication such as generating detailed, informative bug reports. This is typically viewed as the bread and butter of a tester's life. Whether you see it that way or not, there is value in recognizing the structure of those reports and automating a...

I love monkeys; the way they look like us, the way they act like naughty children, even the way they're misused as analogs for random-event-generators in statistical metaphors. When it comes to the Android platform, we have a very naughty monkey of our own: the UI Exerciser Monkey . On the one hand it is extremely useful to have an off-the-shelf option for random UI stress testing. On the other hand the monkey is kind of limited in some of its capabilities (e.g. it gets stuck if it hits a login/logout page and the app doesn't use the " isUserAMonkey() method wisely on click listeners) and scary in others (e.g. potentially dialing phone numbers, sending emails, cranking on music, taking unnecessary screenshots, etc). What the monkey DOES provide is a dumb event generator that flails with swipes, touches, and keypresses. What the monkey DOES NOT do is spider your app, carefully looking for dead-ends, traps, and unhandled touch states. In my typical dealings with the UI ...

Would W. Edwards Deming have a QA team if he ran a software company? For those of you who don't yet know where all this Agile stuff REALLY got started, here's some homework. reference:  http://en.wikipedia.org/wiki/W._Edwards_Deming .  The entire nation of Japan might be justified in thinking he's a pretty important guy. When I came across that article and it immediately forced me to question how any company that chooses to claim Agile development practices can have a QA team. My world was rocked just a little. Fascinating bit of history there, isn't it? If you're cheating and haven't yet read that article while simultaneously not yet knowing the man, his philosophies on Quality, and are simply reading my blog post further, you're cheating. Seriously go read it. I'll wait. For a quick summary, here are some important points to Deming's methodology: Quality is everyone's responsibility Eliminate quality control Encourage quality over ...

"You're some kind of big, fat, smart-bug aren't you?" ~Johnny Rico, Starship Troopers (1997) One of the most important things about writing bugs is making them descriptive but concise. Screenshots, video, debug logging, and hardware snapshots are all fairly standard or available to Android testers these days and can save you enormously on text when communicating what constitutes the bug. Sometimes though, the app gets into a weird state due to some transient data issue where you may not own the API or the complexity with forcing the app data into a certain state is prohibitively high. In those cases it is very handy to directly inspect the data the app has in its own directories. Getting at this data is trivial on emulators and rooted devices but due to file system permissions, this data is otherwise completely private to the app itself. If you're like me, you tend to test using devices rather than emulators and you probably prefer not to root your devices sin...

Wow! It has been a long time since I've updated this blog; so long, in fact, that I a whole new version of Android has been announced (okay maybe 2 or 3). Like many of you, I was paying very close attention to the live streamed sessions at this year's Google I/O. At first I thought L contained nothing new for my QA interests but upon further digging I found a few tiny gems: "You can execute shell commands from your instrumentation test with the new android.app.UiAutomation.executeShellCommand(). The command execution is similar to running adb shell from a host connected to the device. This allows you to use shell based tools such as dumpsys, am, content, and pm." From the API Overview document for Android "L" Developer Preview Oh that's nice.  I guess that means I don't need these anymore: -     public static void runShellCommandStringArray(String[] command) throws IOException {        try {.. -   ...