Everybody Tests

Take a minute and read the following 2-pager entitled " Guest View: Why developers shouldn’t perform software testing ". Man, I tried to like this article because the author, Martin Mudge, clearly knows that businesses who undervalue quality by trying to eliminate testing through simply shuffling the traditional testing task to developers are making a huge mistake. Unfortunately he begins by making an assertion that testing overburdens a developer which at face value is complete nonsense. If you feel overburdened, it is a timeline issue and that is true no matter WHAT the nature of the tasking is.  So his assertion of “one the most important” contributing factors being overburdening the developers is massively flawed. It gets immediately worse from there because his second point is about time constraints.  Mr Mudge just gets so much wrong. What he really should be shooting down is the idea that testing, as a cost-center not as a task, can be eliminated by having your p...

UI automation is like a dumb computer user; a really, really, profoundly dumb computer user. It will fail, it will be obvious that it is failing. But it won't always know immediately why. Sometimes just observing it's failures by means of a screenshot of the failure is all it takes. As you recall from my last post on the topic , screenshots on failure are a potent way to quickly reduce the time to answer the question of why a UI test case failed. Google's Android Test Tools team likes screenshots enough that they've included the capability inside the UiAutomator framework . Square likes it enough to have included it in their Spoon framework . And as we all remember, Robotium's been doing this for a long, long time . The main advantage to the approach I described previously is that screenshots would happen automatically based on errors of any kind in the test lifecycle. As of now, you're all probably migrating or have already migrated to the newer Android...

When you go from building UiAutomator 1.0 tests in Eclipse as a stand-alone JAR over the last couple years to trying to use UiAutomator 2.0 within a product codebase with Gradle and nothing looks familiar... When you figure out why you were getting a null pointer exception after 2 days and it was all because of a copy-paste error... When you start trying to explain a tricky problem with a new test framework to the front-end developers on your project and it starts to feel like they're more spectators then collaborators... When you right-click on a test method and choose "Debug as..." and start debugging your UiAutomator test after all those months of needing to compile a JAR and edit a shell script prescribing which test case to run instead... When you can no longer use all those fantastically useful shell commands you were able to run in your UiAutomator 1.0 tests because 'adb shell uiautomator...' imparted sufficient permissions but 'adb...

When you keep ramping up the event total count on your stress test because it mocks you by not resulting in crashes... When you join a legacy project and upon looking through their Gordian-knot of build dependencies realize that 90% of it is obsolete, yet subtly interdependent experimental code no longer included in the app (EWWWW)... When the project planners lowball the QA estimates because all they're doing is "throwing on a fresh coat of paint"... When that other team throws yet another party celebrating a routine bug-fix release...

New Android build system and test runner, same goose chase using undocumented features and hacks Oh, you thought they were ready to support full enterprise scale CI just because they moved to Gradle? As I've posted before, I am a big fan of Jenkins. It is extremely flexible, open source, and supported by a staggering array of plugins actively developed by engineers running over 100,000 instances of the server worldwide. With it's distributed node model, you can even build your own device cloud for hosting enterprise-scale automation , economizing hardware investments by sharing resources across multiple projects as well as speeding up automation by parallelizing test runs. I had been using a Jenkins-based system in the past to support instrumentation automation with Robotium quite happily. For the last couple years however, my work hasn't required that as much and I've found myself doing a lot more manual testing and using UiAutomator which didn't require...

When you label a bug P1 and are asked to justify that by explaining the impact on the user: When they force you to run the sprint demo and a new bug shows up in the middle of it: When a developer says "show me" and the bug miraculously stops happening: When your bug report is rejected on the grounds of "design by defect" not on the merits of the issue itself: When you spend all day searching the spec documentation for a subtle requirement justifying the bug you're reporting only to refresh the page and see that requirement was just deprecated: When you find a bug so pernicious, it takes the senior developer a week and a half of steady work to fix it, then you immediately find another bug in the fix: When you see the build in production and immediately find a spleling error: When you're asked to estimate the amount of effort it will take to test a specific user story during sprint planning and someone on the team...

In 2013 at a Google-hosted event in New York called the " Google Test Automation Conference ", or GTAC for short, a team of Google engineers tried to sell the attendees the idea that a device lab consisting of real world devices is not as maintainable or optimal of a test bed as a vast server farm running emulators. That's right. Google said that in order to avoid the tyranny of dealing with the real world everyone should have a Google-scale server farm for testing their apps ( video , slides ). And then 2 months later they bought Appurify . Those are pretty epic mixed signals. Example of less paradoxical signaling. Now don't get me wrong, I would LOVE to have Google's server resources for my automated testing. But like many companies who do not make mobile apps or even software in general as their primary business, my current employer seems to prefer to invest more shall we say "modestly" in their app test automation environment resources. ...