I have a test plan that creates a GPG key in before. I had upgraded GPG since the last time I'd run GPG, so the running gpg-agent didn't match /bin/gpg. This caused an error whenever it tried to run GPG. That is to say, I have a before function that started erroring for reasons unrelated to my test suite or the code being tested.
roundup handled this error in before by aborting, pretending that any pending tests or test plans did not exist.
This meant that the test plan with the failing before showed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.
Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!
This is a critical bug for a testing framework. Title is only a slight exaggeration (rm -rf on somewhere important would be worse).
I have a test plan that creates a GPG key in
before. I had upgraded GPG since the last time I'd run GPG, so the runninggpg-agentdidn't match/bin/gpg. This caused an error whenever it tried to run GPG. That is to say, I have abeforefunction that started erroring for reasons unrelated to my test suite or the code being tested.roundup handled this error in
beforeby aborting, pretending that any pending tests or test plans did not exist.This meant that the test plan with the failing
beforeshowed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!
This is a critical bug for a testing framework. Title is only a slight exaggeration (
rm -rfon somewhere important would be worse).