Everybody Tests

When you label a bug P1 and are asked to justify that by explaining the impact on the user: When they force you to run the sprint demo and a new bug shows up in the middle of it: When a developer says "show me" and the bug miraculously stops happening: When your bug report is rejected on the grounds of "design by defect" not on the merits of the issue itself: When you spend all day searching the spec documentation for a subtle requirement justifying the bug you're reporting only to refresh the page and see that requirement was just deprecated: When you find a bug so pernicious, it takes the senior developer a week and a half of steady work to fix it, then you immediately find another bug in the fix: When you see the build in production and immediately find a spleling error: When you're asked to estimate the amount of effort it will take to test a specific user story during sprint planning and someone on the team...

In 2013 at a Google-hosted event in New York called the " Google Test Automation Conference ", or GTAC for short, a team of Google engineers tried to sell the attendees the idea that a device lab consisting of real world devices is not as maintainable or optimal of a test bed as a vast server farm running emulators. That's right. Google said that in order to avoid the tyranny of dealing with the real world everyone should have a Google-scale server farm for testing their apps ( video , slides ). And then 2 months later they bought Appurify . Those are pretty epic mixed signals. Example of less paradoxical signaling. Now don't get me wrong, I would LOVE to have Google's server resources for my automated testing. But like many companies who do not make mobile apps or even software in general as their primary business, my current employer seems to prefer to invest more shall we say "modestly" in their app test automation environment resources. ...

Android Fragmentation, UI Testing and You Last year, Eddie Vassallo over at Gigaom.com joined the rest of the well-informed tech bloggers in agreeing that at least as far as developers are concerned, " [it's] 2014, and Android fragmentation is no longer a problem. " While the developers have tools and patterns that will allow them to successfully build appealing, rich experiences for any screen size and support devices many years past their prime, just the term "Android fragmentation" can intimidate many people trying to decide what phone or tablet to buy. The developer has the ability to access powerful adaptive interface layout APIs and rely on Google Play Services to update crucial dependencies such as Maps and Push Notifications. That behind-the-scenes flexibility is meaningless to an end user staring at a dizzying array of devices from OEMs hellbent on stamping their unique mark on the user experience. Looking at phones such as the Samsung Galaxy Note...

So I'm definitely not an iOS guy. I barely have the bandwidth to keep tabs on the latest and greatest features of the consumer-facing stuff. Keeping in touch with the developer/tester tools is hard too. So it shouldn't be a surprise that I'm just now finding out about xcrun simctl If you're an Android guy and you're familiar with ADB, this is a LOT like that. In fact, after looking through the documentation on xcrun simctl and reading up on it on StackOverflow , it occurred to me that it is finally possible to do what I do on Android with my favorite script ever but on iOS devices/simulators instead. For reference, here's my blog post about that script. The bottom line is that as a tester who regularly uses multiple devices concurrently, the shell script in conjunction with TextExpander is a massive time and mental-space saver for me, making me a whole lot faster and more effective at my job. If you're an iOS tester or you know someone who is, do them a...

Success is a journey, not a destination. So stop running. As a tester, one of my most repeated routines includes the following: downloading a new build from our CI server uninstalling the build from my test devices (note the plural) launching the app for a specific set of tests or investigations.  This happens many times a day, each day, all year. This routine also has many variations based on whether I just want to clear app data and relaunch it, whether I have a different method to launch the app in mind (different starting activity, different intent extras, etc). On top of all of this, there are details about the build which are relevant to any resultant actions I take in our task/bug tracking suite such as build number, device information, etc. This kind of jumble is cumbersome and time-consuming to manage manually. The whole reason to manage all of that process and information is to maintain a consistent practice of reliable reproducible tests and rich environment an...

"I told you. I wake up every day, right here, right in Punxsutawney, and it's always February 2nd, and there's nothing I can do about it." ~ Phil Connors, Groundhog Day The life of a tester involves a lot of repetition. In even ideal environments where you can focus mainly on exploration (rather than rote, unthinking execution of endless, soul-crushing test case minutia) you will still find the need to repeat some of the same tasks many times. Some of those tasks are mechanical and serve an operational purpose (e.g. deploying a fresh environment in which to play). These you can script away to great effect (and I'll detail some of how I do that on mobile devices in a different post). Other tasks are more for communication such as generating detailed, informative bug reports. This is typically viewed as the bread and butter of a tester's life. Whether you see it that way or not, there is value in recognizing the structure of those reports and automating a...

I love monkeys; the way they look like us, the way they act like naughty children, even the way they're misused as analogs for random-event-generators in statistical metaphors. When it comes to the Android platform, we have a very naughty monkey of our own: the UI Exerciser Monkey . On the one hand it is extremely useful to have an off-the-shelf option for random UI stress testing. On the other hand the monkey is kind of limited in some of its capabilities (e.g. it gets stuck if it hits a login/logout page and the app doesn't use the " isUserAMonkey() method wisely on click listeners) and scary in others (e.g. potentially dialing phone numbers, sending emails, cranking on music, taking unnecessary screenshots, etc). What the monkey DOES provide is a dumb event generator that flails with swipes, touches, and keypresses. What the monkey DOES NOT do is spider your app, carefully looking for dead-ends, traps, and unhandled touch states. In my typical dealings with the UI ...