Since tests are not run in bin/compile, there is no reason they need to be resolved and download. The approach of modifying allDependencies works locally by filtering in the HerokuPlugin; however, for some reason, the plugin is not being picked up on Heroku.