add runTests and checkTests tasks to verify a release

[perNyfelt]

Initial version that, although a bit crude, works. It depends on the PR to add a test summary file i submitted to renjin a couple of minutes ago.

The initial content of the packages.mustpass is just meant for testing for now. Given its content:

# This is the list of test that must pass before a release is made
date
digest
- sha1Test.R: floating point values are different, but within tolerances
MASS
- epil.Rd
-lda.Rd: not sure about this one
- stepAIC.Rd: not sure about this one

It is now possible to run ./gradlew -Dprofile=mustpass checkTests which will run the tests in date, digest and MASS (if not up to date) and then parse the $submodule/build/renjin-test-reports/renjin-test-results.log for each package in the list and verify the result against the packages.mustpass list. The result is:

vagrant@ubuntu-xenial:/home/ubuntu/renjin-release/packages$ ./gradlew -Dprofile=mustpass checkTests
Configuration on demand is an incubating feature.
Running tests in date, digest, MASS, 

> Task :checkTests
date: all mandatory tests passed
digest: Failure detected but the test sha1Test.R is excluded in the mustpass list
digest: all mandatory tests passed
MASS: Failure detected but the test epil.Rd is excluded in the mustpass list
MASS: Failure detected but the test lda.Rd is excluded in the mustpass list
MASS: Failure detected but the test stepAIC.Rd is excluded in the mustpass list
MASS: all mandatory tests passed

All tests in the 3 packages of profile "mustpass" checked and verified, good to go!

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 7s
106 actionable tasks: 1 executed, 105 up-to-date

If I remove the last waiver in the packages.mustpass (- stepAIC.Rd: not sure about this one) we get the following instead:

vagrant@ubuntu-xenial:/home/ubuntu/renjin-release/packages$ ./gradlew -Dprofile=mustpass checkTests
Configuration on demand is an incubating feature.
Running tests in date, digest, MASS, 

> Task :checkTests FAILED
date: all mandatory tests passed
digest: Failure detected but the test sha1Test.R is excluded in the mustpass list
digest: all mandatory tests passed
MASS: Failure detected but the test epil.Rd is excluded in the mustpass list
MASS: Failure detected but the test lda.Rd is excluded in the mustpass list
MASS: Failure detected: stepAIC.Rd (root) fail

FAILURE: Build failed with an exception.

* Where:
Build file '/home/ubuntu/renjin-release/packages/build.gradle' line: 127

* What went wrong:
Execution failed for task ':checkTests'.
> 1 failure(s) detected, checkTests for profile "mustpass" was NOT successful

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 6s
106 actionable tasks: 1 executed, 105 up-to-date

I am not sure why those Groovy files from the renjin-gradle-plugin are there as when i diff against the master branch there are no differences - must be something about submodules I dont fully understand.

[akbertram]

Thanks Per, this looks great!
This pull request seems to be a bit borked, so I am going to merge it a bit piecemeal. The first bit is here:
8a431d9

[perNyfelt]

For tools:setupPackages to work we need a setting.gradle file. I suggested in this PR to solve this by adding tools as a subproject to the base in renjin-release. Another way would be to just add a settings.gradle to tools. Right now when I try to build with the latest i get

vagrant@ubuntu-xenial:/home/ubuntu/renjin-release/tools$ ./gradlew setupPackages

FAILURE: Build failed with an exception.

* What went wrong:
Project directory '/home/ubuntu/renjin-release/tools' is not part of the build defined by settings file '/home/ubuntu/renjin-release/settings.gradle'. If this is an unrelated build, it must have its own settings file.

[akbertram]

I think this only happens because somehow you've checked out renjin-gradle-plugin in the renjin release root. There should be no settings.gradle in the root of renjin-release

[perNyfelt]

Yes, you are right. Thanks and sorry for the noise. The vagrant file needs to be cranced up to allow for a 8G heap and I added a config to /home/vagrant/.gradle. I see the Vagrant file was not merged - was there some issues with the proposed changes to it or just an oversight?

[akbertram]

No, sorry, that wasn't intentional, I wanted to extract the Vagrant file as well as some changes to the readme's but didn't have a chance.

…

On Wed, Sep 2, 2020 at 10:15 PM Per Nyfelt ***@***.***> wrote: Yes, you are right. Thanks and sorry for the noise. The vagrant file needs to be cranced up to allow for a 8G heap and I added a config to /home/vagrant/.gradle. I see the Vagrant file was not merged - was there some issues with the proposed changes to it or just an oversight? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADO5Q6Q2X6MS7B2DLJON73SD2RVBANCNFSM4QPETBUA> .

-- Alexander Bertram Technical Director *BeDataDriven BV* Web: http://bedatadriven.com Email: alex@bedatadriven.com Tel. Nederlands: +31(0)647205388 Skype: akbertram

[perNyfelt]

Ok no worries, the pull request was not as clean as I would have wished but you made nice enhancements to it when merging it in. Dealing with submodules in a fork using branches turns out to be quite complex so I'll try to come up with another way to get it cleaner next time. I have some issues with the replacements i.e. I get a null for version in the packages that depend on a replacement package. I tried to verify this on a fresh clone of the repo but that did not build. I'll wait a few days until the merge is complete before attacking this again...