Testing With PHPUnit

This article covers how to build and use unit and functional tests to develop components and extensions to X2Engine. All automated tests in X2Engine use PHPUnit, and in the case of functional testing, Selenium. To understand more what these tools are, why it is important to know about them, how they can make testing easier and how they will strengthen your design and development methodology (in addition to helping you make your code more stable), the following is recommended reading:

• PHPUnit Manual, Ch. 1: Automating Tests
• PHPUnit Manual, Ch. 2: PHPUnit's Goals
• Selenium: Introduction
• Defining Fixtures, Unit Testing and Functional Testing in the Definitive Guide to Yii

While not all of X2Engine has been built using test-driven development, some tools and structure have been developed to make it easier to use unit and functional testing when developing new features. This article mainly covers them, and is intended as supplemental material (not a replacement) for the existing documentation on test-driven development in Yii using PHPUnit and Selenium.

Testing in a nutshell

What are tests?

Tests are organized into PHP classes called test cases that are stored within protected/tests/unit (unit tests) or protected/tests/functional (functional tests). Test cases extend one of the following classes:

CTestCase

For ordinary, non-databse unit tests; useful for defining expected behaviors of component classes that don't rely on tables or data that is specific to a HTTP request

CDbTestCase

For tests involving communication with a MySQL database, especially if that communication involves a change in any data.

WebTestCase (versions up to 2.7.1); X2WebTestCase (2.8 and later); extends CWebTestCase

For tests that can involve web browser actions via Selenium and change in the persistent data of the web app.

Anatomy of a typical test case

How tests work

All tests contain assertions, which are calls to methods in the case that define the conditions under which the test should pass. At the end of the test case or suite (group of tests), if any assertions failed, PHPUnit will tell you exactly which ones failed (by their line number in the case's source code, and the method they're called in). Think of them as a very, very quick and handy way of inserting a conditional statement and a print statement that tells you not only what went wrong but also where in the execution of the code things went wrong.

While PHPUnit runs a suite, you will see a line of output showing the progress of the testing that grows by one character as each test finishes. If a test passed, a period will be appended to the line. If a test failed, it will be a capital "f". If a non-fatal PHP error was encountered, you will see a capital "e". For example, in a test suite where the first five tests passed, the next failed, the next ran into an error and the final one passed you will see this progress:

.....FE.

After the test suite completes, you will see a summary of all the errors and failures encountered, and what lines they were encountered on.

Running test cases

Running a test case proceeds as follows: on the server where the testing environment is installed, inside of the protected/tests folder, run

phpunit path/to/TestCase.php

or, to run a group of test cases that exist in the same directory:

phpunit directory/

PHPUnit will recursively scan directories for test cases to run. In Yii (and X2Engine), unit tests are stored in protected/tests/unit and functional tests in protected/tests/functional. For the sake of clarity, in X2Engine, tests are organized into a directory structure that resembles that of the application under the protected/ folder.

Fortunately, in many shell environments, the previously-run command can be called up by hitting the up arrow on the keyboard. This allows you to quickly re-run an automated test by hitting up and then enter.

Installing PHPUnit

For the most up-to-date information on how to install PHPUnit, see Chapter 3: Installing PHPUnit in the official PHPUnit manual. In addition to PHPUnit, you will need each of the following PHPUnit extensions:

• DBUnit
• PHP_Invoker
• PHPUnit_Selenium
• PHPUnit_Story

See the PHPUnit manual for more information on how to obtain these extensions.

Preparing a testing database

After creating the test database, the following steps can be used to set it up for database testing:

1. Copy the installer files back into the root of the web application: index.php, initialize.php, requirements.php, and initialize_pro.php if on professional edition.
2. Re-run the installer, but enable "Testing Database".

A few very important points to note about tests involving a database connection:

• A different database than the live database should be used for testing
• The contents of each of the test database's tables will be fixed at the beginning of each test using fixtures and init scripts
• All database records generated during tests or through manual interaction will be purged when fixtures for the table in question are used

Fixtures and init scripts

When writing tests that extend CDbTestCase or X2WebTestCase, it is important to define fixtures if the test will change any data in the database and, if desired, init scripts defining how tables should be initialized at the beginning of a test suite and each test.

What are they?

Fixtures are PHP scripts stored in protected/tests/fixtures, each named after the table in the database to which it corresponds. Each fixture script contains a return statement; the returned value is an array that defines the contents of the table at the beginning of the test; each value of the array corresponds to a row in the table and is itself an array with [column name] => [value] entries. By default, no fixtures will be used for a test; the fixtures to use must be defined in the public fixtures attribute (array). For more information on fixtures and how to use them: see Testing: Defining Fixtures and Unit Testing in The Definitive Guide to Yii.

By default, at the beginning of a CDbTestCase test and before loading fixture data, the table in the test database will be completely emptied, its AUTO_INCREMENT value reset to 1.

Initscripts allow you to override that behavior; they are similar to fixtures, but define the initial state before loading fixture data.

As a rule of thumb, the more fixtures you define for a unit test, the more time the test will take to execute, as it means that more database operations will need to take place at the beginning of every test method. If you do not need to reset a particular table in a test case's tests, but need there to be preexisting data to test against, you should create an init script for the table(s) instead.

Note the following comparison between initscripts and fixtures:

Property	Fixtures	Init scripts
File path / name	protected/tests/fixtures/{table name}.php	protected/tests/fixtures/{table name}.init.php
When run/applied	At the beginning of tests in test cases that use them	At the beginning of tests in test cases that use them, before fixtures, and: At the very beginning of a test suite (before any tests run)
Structure of the returned array	array([row alias] => array( [column name] => [column value]))	array( array( [column name] => [column value]) )

To understand the behavior of init scripts, one need only look at X2FixtureManager::resetTable() which, in the case that an array is returned by the init script, inserts the data after truncating the table. Note that additionally, besides defining an initial database state (in its return value, as an array), an init script also may contain arbitrary PHP to be executed at the beginning of a test. This can include such things as filesystem and database schema operations, and thus extend the concept of the initial testing state beyond mere control over ephemeral database contents.

Automatically generate fixtures and init scripts: howto

Beginning in version 2.8, fixture scripts can be automatically generated from preexisting database contents using the "exportfixture" Yii console command. To use it:

1. Open a command shell
2. Change directory into the protected folder inside of the web application
3. Run ./yiic exportfixture

You will then be prompted to enter some details about the data to be exported. You can also execute the command in one fell swoop; run ./yiic help exportfixture to see its invocation syntax.

Note: this will take data from the live database and not from the testing database.

Reference Fixtures

If your test case extends X2DbTestCase, you can specify that certain fixtures be loaded at the beginning of a test class only, as "reference" data that will not be changed but will be referenced. This makes running tests faster by skipping re-load of the data, while making fixture data easily accessible as a fixtures property. To do this, declare the static referenceFixtures method of your test case such that it returns an array similar to the fixtures property. For example, to load all the user/profile data for reference:

    public static function referenceFixtures(){
        return array(
            'profile' => 'Profile',
            'users' => 'User',
        );
    }

Alternate Fixtures

Using X2FixtureManager allows you to have more than one fixture per table, and to switch between distinct tests that need different data. You can do this by, in the declaration of a fixture, using a two-element array in place of a string to specify the model or table name. In the array, the first element is assumed to be the name of the active record model or table, whereas the second is a suffix that when appended to the table name and concatenated with ".php" gives the full fixture filename.

For example, if you had a fixture file x2_users.extra.php, and wanted to use that file instead of x2_users.php, you could use the following fixture definition:

    public $fixtures=array(
        'users'=>array('User','.extra'),
    );

Functional Testing with Selenium

An advanced PHPUnit/Selenium configuration involving multiple network hosts, operating systems and web browsers. PHPUnit (1) begins a test by starting browser sessions on a remote host running Selenium RC (2). Selenium then commands the web browsers to open the web application in testing mode (3) and run automated instructions, sending results back to PHPUnit (i.e. whether a test has passed or failed). All logos are © their respective owners.

Selenium provides the means to control and automate browser actions, and hence, to run tests with full web-based sessions and HTTP request data. Test cases that use Selenium extend WebTestCase and are run in the same manner as unit tests, but require that the following environmental conditions are met:

1. The <selenium> section in the configuration file protected/tests/phpunit.xml is set properly. For information on how to configure this, see The PHPUnit Manual Appendix C: The XML Configuration File
2. Selenium RC on all the hosts defined in phpunit.xml can be accessed from the web server on the specified ports (note: if they are behind a firewall, it will be necessary to set up NAT in order to properly use them)
3. The URL defined as constant TEST_BASE_URL in protected/tests/WebTestConfig.php resolves properly (points to index-test.php/ on the web server).
4. PHPUnit and the necessary extensions are installed on the web server

Creating Selenium test cases

1. Create a class that extends WebTestCase in protected/tests/functional
2. Note that at the beginning of the execution of a test case's method, the method WebTestCase::setUp() will be called, and this will create a session as the admin user. If any additional operations are necessary to include at the beginning of every test, you may override the method, but should include a call to parent::setUp() at the beginning of it.
3. In the test method, open the URL where the test is to begin by calling WebTestCase::openX2(), passing it as an argument the URI[[wikipedia:Uniform Resource Identifier]]: The part of a URL that identifies the resource on the server to be accessed. In the context of the API, this refers to the relative path within the web server based in the web root of X2Engine, i.e. ''index.php/api2/Contacts/324.json'' as opposed to the full URL, which begins with the protocol (i.e. "http") and might also contain a path relative to the web site's document root relative to the base URL (what would come after index-test.php).
4. Note the availability of CWebTestCase's methods, especially those inherited from PHPUnit_Framework_TestCase, PHPUnit_Framework_Assert and PHPUnit_Extensions_SeleniumTestCase, in addition to WebTestCase::session() (which will ensure the session matches the user/password defined in WebTestCase::$login, logging the browser out/in as necessary) and the aforementioned methods that will be inherited from WebTestCase.

Running test cases built with Selenium IDE

Building functional tests involving browser automation with Selenium can be done with PHPUnit browser commands, i.e. assertTextPresent. However, it is far more easy, convenient and quick to build them using Selenium IDE. Selenium IDE is a web browser plugin for Mozilla Firefox that provides a means of generating Selenium commands by capturing mouse and keyboard actions. It also allows you to edit and re-position commands after creating them, and boasts a multitude of other very useful features. To obtain Selenium IDE, visit the Selenium HQ downloads page.

To run a native Selenese .html test case created in Selenium IDE:

1. Save the .html in the same folder as the PHP test case class (a subdirectory of protected/tests/functional)
2. In a test method, open the URL where the Selenese test case is to begin using WebTestCase::openX2().
3. Call the method WebTestCase::localSelenese() with the filename of the Selenese test case as the sole argument.

Selenium tests are easy to execute an arbitrary number of times; simply calling localSelenese() again will run the script a second time. Selenese test cases are thus optimal not only for typical tests due to how easy they are to generate, but also are good for tests that need to be repeated as different users with different permissions (in the case that similar behavior/display is desired for all sets of permissions).

Example

The following example method opens the "create" action in the contacts module, runs a Selenese test case called "TestCreate.html", logs out, logs back in as a different user ("user2", with password "password2") and runs the same test again:

class ContactsTest extends WebTestCase {
   // ...
   public function testCreate() {
      $this->openX2('contacts/create');
      $this->localSelenese("TestCreate.html");
      $this->login = array('username'=>'user2','password'=>'password2');
      $this->session();
      $this->openX2('contacts/create');
      $this->localSelenese("TestCreate.html");
   }
   // ...
}