Frustration finally got the better of the dev team. With all the main projects now running from PHP 5.3 based boxes (or Mac OSX), relying on SimpleTest was proving too much. So the work to convert to PHPUnit has been brought forward.
The next release of Scorpio will have all tests running via PHPUnit. This has been a topic that has been discussed before, but to refresh: previously all test cases were run through SimpleTest which was itself hooked into Scorpio via the CLI tool testSuite.php and in a web based testing component in the baseAdminSite. SimpleTest was chosen because that was the environment most familiar.
However: development of SimpleTest seems to have slowed almost to a standstill, and with no new major release in over 2 years, it was obvious an alternative was required. Couple that with the shear volume of errors caused by deprecated notices and it makes using it under PHP 5.3 extremely difficult.
Fortunately there is already an alternative in PHPUnit. PHPUnit seems to now be the standard test framework in use in almost all major PHP projects. Further: it is nicely integrated into Netbeans, Eclipse and Zend Studio and is already PHP5.3 compliant - in fact it is a PHP 5 application.
Converting between frameworks is never easy, but fortunately in Scorpio an intermediary class (testCaseScorpio) was used for the tests. This allows an amount of work to be done to accommodate some of the API differences between SimpleTest and PHPUnit. In fact if straight forward testing has been done then most tests should still work largely OK without a lot of additional work.
What will cause issues are tests that used Mock::generate() from SimpleTest. The mock interface is very different in PHPUnit. Mock::generate() can be replaced with: $this->getMock(); and then the return types can be set on the mocked object by calling: expects()->method()->will(). The documentation can be found on PHPUnits website.
The other areas likely to cause issue are:
1) Testing boolean return types - PHPUnit requires that assertTrue() test for an actual true value - 1, the string "true" etc will fail these tests. And the same goes for assertFalse - it will only pass for actual false values, not for empty strings, null, false, 0 etc.
2) Skipping tests. SimpleTest provided an abstract method skip() that would allow an entire test class to be skipped. This no longer works. You should implement the setUp() method and then add any skip criteria to this and call $this->markTestSkipped(). Remember to call into the parent::setUp() method when implementing your own setUp() or tearDown().
The CLI tools have been updated to now run with PHPUnit and the testSuitePackages methods will now return PHPUnit_Framework_TestSuite objects. The formatting remains largely the same. The test case builder will be updated shortly to build better PHPUnit compliant stubs.
What does this mean for SimpleTest? Well simply put: it will be removed from the libraries folder of Scorpio: BUT, PHPUnit will NOT take its place. PHPUnit will not be bundled with Scorpio. You will be required to download, install and configure PHPUnit yourself. This can be done relatively easily under Linux and Mac OSX via PEAR. Full instructions are available on the PHPUnit website.
One last note: the test cases have been reworked so that they can be launched from within Zend Studio by simply right-clicking and selecting Run As -> PHPUnit Test. This can be done on folders or single files.
We hope that our users understand this change - and the need for it and we hope that this will not be too jarring a difference. It will make testing easier in the long term which is what is most important.