A marshmallow shaped problem in my screenshot empire

"Never without my permission" ~ Leeloo (translated)

android.permission.WRITE_EXTERNAL_STORAGE

As a tester, you probably have a favorite app, maybe a favorite feature, possibly even a favorite bug. But you probably don't have a favorite permission.

I've been taking advantage of it for years to do things like gather code coverage on non-rooted devices, logcat output from an entire test run, and probably my favorite usage is screenshots triggered when tests fail. Yep, that little permission has been a big pal of mine for a while. So it should be no surprise that I felt a little betrayed when I recently discovered a problem with it on devices running Android 6.0 aka "Marshmallow".

Among the many changes brought about by the 6.0 release was a long-overdue overhaul to Android's permission scheme, notably, from an all-or-nothing approval of permissions at install-time model from Android 1.0-5.1.1 to the more nuanced permissions requested when required during run-time model now. The idea is to give the user more control over what they want to allow an application to do and give app developer the ability to request permissions in a context the user understands inside their running application. In other words, everybody wins. Kinda. Okay not really.

See, I really like to use screenshots to quickly determine what went wrong when a UI test case of mine has failed. Like the saying goes, "a picture is worth a thousand lines of logcat". My test harnesses have had a means of capturing those screenshots and saving them locally to the device's /sdcard/TEST_OUTPUT_DIR which my framework would specify. Since that's not an internal directory to the application under test, I need to make sure it's manifest includes the permission: android.permission.WRITE_EXTERNAL_STORAGE. For some apps I've tested that was added at build-time for my specific builds as the public version of the app didn't require it. For the rest, it was already there and simply provided extra utility for my automation. And up until Marshmallow, that was enough.

What I discovered after adding test devices to my cluster that were running Android 6.0 was that when their UI tests experienced failures, no screenshots were recovered. When I checked the logs, I found that the files could not be written owing to a permission violation exception: a little like this:

E UiAutomatorBridge: java.io.FileNotFoundException: /sdcard/test_output_dir/com.example.app.test.DummyTestClass.testMethod.png: open failed: EACCES (Permission denied)

So after double-checking the manifest to make sure I wasn't taking crazy pills, it occurred to me to check the permissions in the Settings App to see if it wasn't giving me access to Storage. Sure enough, the permission was in the list but the toggle was off. I flipped the toggle on and re-ran the test without clearing app data or reinstalling it. Then lo and behold a wild screenshot appeared!

While discussing this problem with my friend, Joe, he suggested I try calling "pm grant com.example.app android.permission.WRITE_EXTERNAL_STORAGE" before the test and see if that worked. My hopes were high. I had a clever check in my script (also by Joe's suggestion) to only run that command if "getprop ro.build.version.sdk" evaluated greater than or equal to 23. I had tested the "pm grant" command and observed that it showed the Storage toggle as enabled in the Settings App's permissions for my app under test. Everything should have worked beautifully.

E UiAutomatorBridge: java.io.FileNotFoundException: /sdcard/test_output_dir/com.example.app.test.DummyTestClass.testMethod.png: open failed: EACCES (Permission denied)

ARE YOU KIDDING ME?!?!

So for the next hour or two or three, Joe and I speculated, I retraced my steps over and over again. Then I re-read that error message: "open failed". That sounded an awful lot like READ_EXTERNAL_STORAGE. To look further into it, I did something crazy: I read the documentation. Tucked away in there was this beautiful little nugget: 

However, if your app uses the WRITE_EXTERNAL_STORAGE permission, then it implicitly has permission to read the external storage as well.

How lovely. That made me think that perhaps the script needed to explicitly grant the READ_EXTERNAL_STORAGE permission even though it was implicitly included with the write permission. After a quick edit to the script, I re-ran my example test scenario and it finally worked again. After beating my head against my monitor for half the day thinking I'd been wrong, or high it came down to setting read access in order to use write access. Intuitive, right? Not if you ask me. I was so taken aback by the problem, I actually opened a bug on this. The issue no longer really seems to be a bug per se, but it is at least a subtle and possibly unforeseen consequence of the new permissions model.

Here's a quick Gist demonstrating the basic approach I use inside my larger test execution script: