A large number of unit tests cause to get stuck and cause Binder Transaction to fail
I wrote about 400 unit tests for my Android application. If I run the tests package by package and everything works fine, all my tests pass. However, if I try to run them at the same time, eventually (about 360 tests), Android starts spitting !!! Failed Binder transaction error !!!. After about 10-20 such errors, the process of the application I was testing was terminated and the unit tests never even completed.
I should have noticed that during this time Eclipse reported Collecting test information in the console. I think this is strange because it shows that it is running tests, even though the JUnit UI does not reflect this. When everything is fine, in the Collecting test information phase I see the tests run, then the JUnit UI appears, and then all the tests run again (I know this by reading logcat). It’s (I think) a separate issue, but I feel like if anyone knew what that was, I would mention it.
Edited June 6, 2011
Based on Christopher’s answer below, I’ve confirmed that this will only happen when trying to run through Eclipse. If I run my test suite with ANT, all tests will eventually execute.
When I try to run my test suite now, Eclipse is basically stuck on “collecting test information”. I didn’t let it run for a long time, but I’ll try it soon and see if it gets done.
Solution
I investigated the issue
again, and as far as I can tell, it’s purely an issue with the Android Eclipse plugin. A problem that seems to be solved but still happens sometimes (probably machine-related).
As you mentioned, the Eclipse plugin seems to run all tests twice. The first time it actually did was collect the test suite and test names so that it could display all the test names in a nice hierarchical JUnit UI.
However, “running” tests like this seems to cause problems. As part of Android SDK Tools v8, a “workaround” was added puts a delay of 15 milliseconds between each test checked. This is done to prevent “Binder transaction failure… For large test suites”.
INDEED, I
WENT BACK TO THE PROJECT WITH THE “FAILED BINDER TRANSACTION” ERROR, AND I WAS UNABLE TO REPRODUCE IT ON THE COMMAND LINE. We also tried it in Eclipse, but it can no longer be replicated (although I’m sure we already had SDK Tools r8+ when we first saw it).
However, I can still reproduce it roughly in Eclipse. Try running your tests again from Eclipse using the latest Android tools, or try this minimal GitHub repository I created and see if you can reproduce it:
https://github.com/orrc/android-large-test-failures#readme