© 2015 X2Engine Inc.

Difference between revisions of "Testing With PHPUnit"

From X2Engine
Jump to: navigation, search
(Running test cases)
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
[[Category:Development]]
 
[[Category:Development]]
This article covers how to build and use unit and functional tests to develop components and extensions to X2CRM. While not all of X2CRM has been built using test-driven development, some tools and structure have been developed to make it easier to use unit and functional testing when developing new features. All automated tests in X2CRM use PHPUnit, and in the case of functional testing, Selenium. To understand more what these tools are, why it is important to know about them, and how they will strengthen your design and development methodology (in addition to helping you make your code more stable), the following is recommended reading:
+
This article covers how to build and use unit and functional tests to develop components and extensions to X2Engine. All automated tests in X2Engine use PHPUnit, and in the case of functional testing, Selenium. To understand more what these tools are, why it is important to know about them, how they can make testing easier and how they will strengthen your design and development methodology (in addition to helping you make your code more stable), the following is recommended reading:
 
* [http://www.phpunit.de/manual/current/en/automating-tests.html PHPUnit Manual, Ch. 1: Automating Tests]
 
* [http://www.phpunit.de/manual/current/en/automating-tests.html PHPUnit Manual, Ch. 1: Automating Tests]
 
* [http://www.phpunit.de/manual/current/en/goals.html PHPUnit Manual, Ch. 2: PHPUnit's Goals]
 
* [http://www.phpunit.de/manual/current/en/goals.html PHPUnit Manual, Ch. 2: PHPUnit's Goals]
 
* [http://seleniumhq.org/docs/01_introducing_selenium.jsp Selenium: Introduction]
 
* [http://seleniumhq.org/docs/01_introducing_selenium.jsp Selenium: Introduction]
 +
* [[yiiguide:test.fixture|Defining Fixtures]], [[yiiguide:test.unit|Unit Testing]] and [[yiiguide:test.functional|Functional Testing]] in the Definitive Guide to Yii
  
Also useful are [[yiiguide:test.fixture|Defining Fixtures]], [[yiiguide:test.unit|Unit Testing]] and [[yiiguide:test.functional|Functional Testing]] in the Definitive Guide to Yii.
+
While not all of X2Engine has been built using test-driven development, some tools and structure have been developed to make it easier to use unit and functional testing when developing new features. This article mainly covers them, and is intended as supplemental material (not a replacement) for the existing documentation on test-driven development in Yii using PHPUnit and Selenium.
  
= Introduction =
+
= Testing in a nutshell =
  
 
== What are tests? ==
 
== What are tests? ==
Line 15: Line 16:
 
;[[yii:CDbTestCase|CDbTestCase]]
 
;[[yii:CDbTestCase|CDbTestCase]]
 
: For tests involving communication with a MySQL database, especially if that communication involves a change in any data.
 
: For tests involving communication with a MySQL database, especially if that communication involves a change in any data.
;[[x2doc:WebTestCase|WebTestCase]] (extends [[yii:CWebTestCase|CWebTestCase]])
+
;[[x2doc:WebTestCase|WebTestCase]] (versions up to 2.7.1); [[x2doc:X2WebTestCase|X2WebTestCase]] (2.8 and later); extends [[yii:CWebTestCase|CWebTestCase]]
 
: For tests that can involve web browser actions via Selenium and change in the persistent data of the web app.
 
: For tests that can involve web browser actions via Selenium and change in the persistent data of the web app.
 
[[File:TestCase.png|400px|thumb|right|Anatomy of a typical test case]]
 
[[File:TestCase.png|400px|thumb|right|Anatomy of a typical test case]]
 +
 +
== How tests work ==
 +
All tests contain ''assertions'', which are calls to methods in the case that define the conditions under which the test should pass. At the end of the test case or suite (group of tests), if any assertions failed, PHPUnit will tell you exactly which ones failed (by their line number in the case's source code, and the method they're called in). Think of them as a very, very quick and handy way of inserting a conditional statement ''and'' a print statement that tells you not only what went wrong but also where in the execution of the code things went wrong.
 +
 +
While PHPUnit runs a suite, you will see a line of output showing the progress of the testing that grows by one character as each test finishes. If a test passed, a period will be appended to the line. If a test failed, it will be a capital "f". If a non-fatal PHP error was encountered, you will see a capital "e". For example, in a test suite where the first five tests passed, the next failed, the next ran into an error and the final one passed you will see this progress:
 +
<pre>.....FE.</pre>
 +
After the test suite completes, you will see a summary of all the errors and failures encountered, and what lines they were encountered on.
  
 
== Running test cases ==
 
== Running test cases ==
Running a test case proceeds as follows: on the server where the testing environment is installed, inside of the <tt>protected/tests</tt> folder, run <pre>phpunit path/to/TestCase.php</pre> or, to run a group of test cases that exist in the same directory: <pre>phpunit directory/</pre> PHPUnit will recursively scan directories for test cases to run. In Yii (and X2CRM), unit tests are stored in <tt>protected/tests/unit</tt> and functional tests in <tt>protected/tests/functional</tt>. For the sake of clarity, in X2CRM, tests are organized into a directory structure that resembles that of the application under the <tt>protected/</tt> folder.
+
Running a test case proceeds as follows: on the server where the testing environment is installed, inside of the <tt>protected/tests</tt> folder, run <pre>phpunit path/to/TestCase.php</pre> or, to run a group of test cases that exist in the same directory: <pre>phpunit directory/</pre> PHPUnit will recursively scan directories for test cases to run. In Yii (and X2Engine), unit tests are stored in <tt>protected/tests/unit</tt> and functional tests in <tt>protected/tests/functional</tt>. For the sake of clarity, in X2Engine, tests are organized into a directory structure that resembles that of the application under the <tt>protected/</tt> folder.
  
 
Fortunately, in many shell environments, the previously-run command can be called up by hitting the up arrow on the keyboard. This allows you to quickly re-run an automated test by hitting up and then enter.
 
Fortunately, in many shell environments, the previously-run command can be called up by hitting the up arrow on the keyboard. This allows you to quickly re-run an automated test by hitting up and then enter.
Line 33: Line 41:
  
 
= Preparing a testing database =
 
= Preparing a testing database =
A few very important points to note about tests involving a database connection:
 
* A different database will be used than the live database
 
* The contents of each of the test database's tables will be fixed at the beginning of each test using ''fixtures''
 
* All database records generated during tests or through manual interaction will be purged when fixtures for the table in question are used
 
 
 
After creating the test database, the following steps can be used to set it up for database testing:
 
After creating the test database, the following steps can be used to set it up for database testing:
 
# Copy the installer files back into the root of the web application: index.php, initialize.php, requirements.php, and initialize_pro.php if on professional edition.
 
# Copy the installer files back into the root of the web application: index.php, initialize.php, requirements.php, and initialize_pro.php if on professional edition.
 
# Re-run the installer, but enable "Testing Database".
 
# Re-run the installer, but enable "Testing Database".
 +
 +
A few very important points to note about tests involving a database connection:
 +
* A different database than the live database should be used for testing
 +
* The contents of each of the test database's tables will be fixed at the beginning of each test using ''fixtures'' and ''init scripts''
 +
* All database records generated during tests or through manual interaction will be purged when fixtures for the table in question are used
  
 
= Fixtures and init scripts =
 
= Fixtures and init scripts =
When writing tests that extend <tt>CDbTestCase</tt> or <tt>WebTestCase</tt>, it is important to define ''fixtures'' if the test will change any data in the database and, if desired, ''init scripts'' defining how tables should be initialized at the beginning of a test suite and each test.
+
When writing tests that extend <tt>CDbTestCase</tt> or <tt>X2WebTestCase</tt>, it is important to define ''fixtures'' if the test will change any data in the database and, if desired, ''init scripts'' defining how tables should be initialized at the beginning of a test suite and each test.
  
 
== What are they? ==
 
== What are they? ==
 
'''Fixtures''' are PHP scripts stored in <tt>protected/tests/fixtures</tt>, each named after the table in the database to which it corresponds. Each fixture script contains a <tt>return</tt> statement; the returned value is an array that defines the contents of the table at the beginning of the test; each value of the array corresponds to a row in the table and is itself an array with <tt>[column name] => [value]</tt> entries. By default, no fixtures will be used for a test; the fixtures to use must be defined in the public <tt>fixtures</tt> attribute (array). For more information on fixtures and how to use them: see [[yiiguide:test.fixture|Testing: Defining Fixtures]] and [[yiiguide:test.unit|Unit Testing]] in ''The Definitive Guide to Yii''.
 
'''Fixtures''' are PHP scripts stored in <tt>protected/tests/fixtures</tt>, each named after the table in the database to which it corresponds. Each fixture script contains a <tt>return</tt> statement; the returned value is an array that defines the contents of the table at the beginning of the test; each value of the array corresponds to a row in the table and is itself an array with <tt>[column name] => [value]</tt> entries. By default, no fixtures will be used for a test; the fixtures to use must be defined in the public <tt>fixtures</tt> attribute (array). For more information on fixtures and how to use them: see [[yiiguide:test.fixture|Testing: Defining Fixtures]] and [[yiiguide:test.unit|Unit Testing]] in ''The Definitive Guide to Yii''.
  
By default, at the beginning of a <tt>CDbTestCase</tt> test and before loading fixture data, the table in the test database will be completely emptied, its <TT>AUTO_INCREMENT</tt> value reset to 1. '''Initscripts''' allow you to override that behavior; they are similar to fixtures, but define the initial state before loading fixture data.
+
By default, at the beginning of a <tt>CDbTestCase</tt> test and before loading fixture data, the table in the test database will be completely emptied, its <TT>AUTO_INCREMENT</tt> value reset to 1.  
 +
 
 +
'''Initscripts''' allow you to override that behavior; they are similar to fixtures, but define the initial state before loading fixture data.
  
As a rule of thumb, the more fixtures you define for a unit test, the more time the test will take to execute, as it means that more database operations will need to take place at the beginning of every test method. If you do not need to reset a particular table in a test case's tests, but need there to be preexisting data to test against, you should create an script for the table(s) instead.
+
As a rule of thumb, the more fixtures you define for a unit test, the more time the test will take to execute, as it means that more database operations will need to take place at the beginning of every test method. If you do not need to reset a particular table in a test case's tests, but need there to be preexisting data to test against, you should create an init script for the table(s) instead.
  
 
Note the following comparison between initscripts and fixtures:  
 
Note the following comparison between initscripts and fixtures:  
Line 77: Line 87:
 
To understand the behavior of init scripts, one need only look at [[x2propdoc:X2FixtureManager.html#_resetTable|X2FixtureManager::resetTable()]] which, in the case that an array is returned by the init script, inserts the data after truncating the table. Note that additionally, besides defining an initial database state (in its return value, as an array), an init script also may contain arbitrary PHP to be executed at the beginning of a test. This can include such things as filesystem and database schema operations, and thus extend the concept of the initial testing state beyond mere control over ephemeral database contents.
 
To understand the behavior of init scripts, one need only look at [[x2propdoc:X2FixtureManager.html#_resetTable|X2FixtureManager::resetTable()]] which, in the case that an array is returned by the init script, inserts the data after truncating the table. Note that additionally, besides defining an initial database state (in its return value, as an array), an init script also may contain arbitrary PHP to be executed at the beginning of a test. This can include such things as filesystem and database schema operations, and thus extend the concept of the initial testing state beyond mere control over ephemeral database contents.
  
== How to automatically generate fixtures and init scripts ==
+
== Automatically generate fixtures and init scripts: howto ==
 
Beginning in version 2.8, fixture scripts can be automatically generated from preexisting database contents using the "exportfixture" Yii console command. To use it:
 
Beginning in version 2.8, fixture scripts can be automatically generated from preexisting database contents using the "exportfixture" Yii console command. To use it:
 
# Open a command shell
 
# Open a command shell
 
# Change directory into the <tt>protected</tt> folder inside of the web application
 
# Change directory into the <tt>protected</tt> folder inside of the web application
 
# Run <tt>./yiic exportfixture</tt>
 
# Run <tt>./yiic exportfixture</tt>
You will then be prompted to enter some details about the data to be exported. You can also run the command in one fell swoop; run <tt>./yiic help exportfixture</tt> to see its invocation syntax.
+
You will then be prompted to enter some details about the data to be exported. You can also execute the command in one fell swoop; run <tt>./yiic help exportfixture</tt> to see its invocation syntax.
 +
 
 +
'''Note:''' this will take data from the ''live'' database and not from the testing database.
 +
 
 +
== Reference Fixtures ==
 +
If your test case extends X2DbTestCase, you can specify that certain fixtures be loaded at the beginning of a test class only, as "reference" data that will not be changed but will be referenced. This makes running tests faster by skipping re-load of the data, while making fixture data easily accessible as a fixtures property. To do this, declare the static <tt>referenceFixtures</tt> method of your test case such that it returns an array similar to the fixtures property. For example, to load all the user/profile data for reference:
 +
<syntaxhighlight lang="php">
 +
    public static function referenceFixtures(){
 +
        return array(
 +
            'profile' => 'Profile',
 +
            'users' => 'User',
 +
        );
 +
    }
 +
</syntaxhighlight>
 +
 
 +
== Alternate Fixtures ==
 +
Using <tt>X2FixtureManager</tt> allows you to have more than one fixture per table, and to switch between distinct tests that need different data. You can do this by, in the declaration of a fixture, using a two-element array in place of a string to specify the model or table name. In the array, the first element is assumed to be the name of the active record model or table, whereas the second is a suffix that when appended to the table name and concatenated with ".php" gives the full fixture filename.  
  
'''Note:''' that this will take data from the ''live'' database and not from the testing database.
+
For example, if you had a fixture file <tt>x2_users.extra.php</tt>, and wanted to use that file instead of <tt>x2_users.php</tt>, you could use the following fixture definition:
 +
<syntaxhighlight lang="php">
 +
    public $fixtures=array(
 +
        'users'=>array('User','.extra'),
 +
    );
 +
</syntaxhighlight>
  
 
= Functional Testing with Selenium =
 
= Functional Testing with Selenium =
 
[[File:PHPUnit.Selenium.png|400px|thumb|right|An advanced PHPUnit/Selenium configuration involving multiple network hosts, operating systems and web browsers. PHPUnit '''(1)''' begins a test by starting browser sessions on a remote host running Selenium RC '''(2)'''. Selenium then commands the web browsers to open the web application in testing mode '''(3)''' and run automated instructions, sending results back to PHPUnit (i.e. whether a test has passed or failed). All logos are &copy; their respective owners.]]
 
[[File:PHPUnit.Selenium.png|400px|thumb|right|An advanced PHPUnit/Selenium configuration involving multiple network hosts, operating systems and web browsers. PHPUnit '''(1)''' begins a test by starting browser sessions on a remote host running Selenium RC '''(2)'''. Selenium then commands the web browsers to open the web application in testing mode '''(3)''' and run automated instructions, sending results back to PHPUnit (i.e. whether a test has passed or failed). All logos are &copy; their respective owners.]]
Selenium provides the means to control and automate browser actions, and hence, to run tests with full browser sessions. Test cases that use Selenium extend <tt>WebTestCase</tt> and are run in the same manner as unit tests, but require that the following environmental conditions are met:
+
Selenium provides the means to control and automate browser actions, and hence, to run tests with full web-based sessions and HTTP request data. Test cases that use Selenium extend <tt>WebTestCase</tt> and are run in the same manner as unit tests, but require that the following environmental conditions are met:
 
# The <tt>&lt;selenium&gt;</tt> section in the configuration file <tt>protected/tests/phpunit.xml</tt> is set properly. For information on how to configure this, see [http://www.phpunit.de/manual/current/en/appendixes.configuration.html The PHPUnit Manual Appendix C: The XML Configuration File]
 
# The <tt>&lt;selenium&gt;</tt> section in the configuration file <tt>protected/tests/phpunit.xml</tt> is set properly. For information on how to configure this, see [http://www.phpunit.de/manual/current/en/appendixes.configuration.html The PHPUnit Manual Appendix C: The XML Configuration File]
 
# Selenium RC on all the hosts defined in phpunit.xml can be accessed from the web server on the specified ports (note: if they are behind a firewall, it will be necessary to set up NAT in order to properly use them)  
 
# Selenium RC on all the hosts defined in phpunit.xml can be accessed from the web server on the specified ports (note: if they are behind a firewall, it will be necessary to set up NAT in order to properly use them)  
Line 101: Line 132:
  
 
== Running test cases built with Selenium IDE ==
 
== Running test cases built with Selenium IDE ==
Building functional tests involving browser automation with Selenium can be done with PHPUnit browser commands, i.e. <tt>assertTextPresent</tt>. However, it is far more easy, convenient and quick to build them using Selenium IDE. Selenium IDE is a web browser plugin for Firefox that provides a means of generating Selenium commands by capturing mouse and keyboard actions. It also allows you to edit and re-position commands after creating them, and boasts a multitude of other very useful features. To obtain Selenium IDE, visit the [http://seleniumhq.org/download/ Selenium HQ downloads page].
+
Building functional tests involving browser automation with Selenium can be done with PHPUnit browser commands, i.e. <tt>assertTextPresent</tt>. However, it is far more easy, convenient and quick to build them using Selenium IDE. Selenium IDE is a web browser plugin for Mozilla Firefox that provides a means of generating Selenium commands by capturing mouse and keyboard actions. It also allows you to edit and re-position commands after creating them, and boasts a multitude of other very useful features. To obtain Selenium IDE, visit the [http://seleniumhq.org/download/ Selenium HQ downloads page].
  
 
To run a native Selenese <tt>.html</tt> test case created in Selenium IDE:
 
To run a native Selenese <tt>.html</tt> test case created in Selenium IDE:

Latest revision as of 23:54, 6 February 2014

This article covers how to build and use unit and functional tests to develop components and extensions to X2Engine. All automated tests in X2Engine use PHPUnit, and in the case of functional testing, Selenium. To understand more what these tools are, why it is important to know about them, how they can make testing easier and how they will strengthen your design and development methodology (in addition to helping you make your code more stable), the following is recommended reading:

While not all of X2Engine has been built using test-driven development, some tools and structure have been developed to make it easier to use unit and functional testing when developing new features. This article mainly covers them, and is intended as supplemental material (not a replacement) for the existing documentation on test-driven development in Yii using PHPUnit and Selenium.

Testing in a nutshell

What are tests?

Tests are organized into PHP classes called test cases that are stored within protected/tests/unit (unit tests) or protected/tests/functional (functional tests). Test cases extend one of the following classes:

CTestCase
For ordinary, non-databse unit tests; useful for defining expected behaviors of component classes that don't rely on tables or data that is specific to a HTTP request
CDbTestCase
For tests involving communication with a MySQL database, especially if that communication involves a change in any data.
WebTestCase (versions up to 2.7.1); X2WebTestCase (2.8 and later); extends CWebTestCase
For tests that can involve web browser actions via Selenium and change in the persistent data of the web app.
Anatomy of a typical test case

How tests work

All tests contain assertions, which are calls to methods in the case that define the conditions under which the test should pass. At the end of the test case or suite (group of tests), if any assertions failed, PHPUnit will tell you exactly which ones failed (by their line number in the case's source code, and the method they're called in). Think of them as a very, very quick and handy way of inserting a conditional statement and a print statement that tells you not only what went wrong but also where in the execution of the code things went wrong.

While PHPUnit runs a suite, you will see a line of output showing the progress of the testing that grows by one character as each test finishes. If a test passed, a period will be appended to the line. If a test failed, it will be a capital "f". If a non-fatal PHP error was encountered, you will see a capital "e". For example, in a test suite where the first five tests passed, the next failed, the next ran into an error and the final one passed you will see this progress:

.....FE.

After the test suite completes, you will see a summary of all the errors and failures encountered, and what lines they were encountered on.

Running test cases

Running a test case proceeds as follows: on the server where the testing environment is installed, inside of the protected/tests folder, run
phpunit path/to/TestCase.php
or, to run a group of test cases that exist in the same directory:
phpunit directory/
PHPUnit will recursively scan directories for test cases to run. In Yii (and X2Engine), unit tests are stored in protected/tests/unit and functional tests in protected/tests/functional. For the sake of clarity, in X2Engine, tests are organized into a directory structure that resembles that of the application under the protected/ folder.

Fortunately, in many shell environments, the previously-run command can be called up by hitting the up arrow on the keyboard. This allows you to quickly re-run an automated test by hitting up and then enter.

Installing PHPUnit

For the most up-to-date information on how to install PHPUnit, see Chapter 3: Installing PHPUnit in the official PHPUnit manual. In addition to PHPUnit, you will need each of the following PHPUnit extensions:

  • DBUnit
  • PHP_Invoker
  • PHPUnit_Selenium
  • PHPUnit_Story

See the PHPUnit manual for more information on how to obtain these extensions.

Preparing a testing database

After creating the test database, the following steps can be used to set it up for database testing:

  1. Copy the installer files back into the root of the web application: index.php, initialize.php, requirements.php, and initialize_pro.php if on professional edition.
  2. Re-run the installer, but enable "Testing Database".

A few very important points to note about tests involving a database connection:

  • A different database than the live database should be used for testing
  • The contents of each of the test database's tables will be fixed at the beginning of each test using fixtures and init scripts
  • All database records generated during tests or through manual interaction will be purged when fixtures for the table in question are used

Fixtures and init scripts

When writing tests that extend CDbTestCase or X2WebTestCase, it is important to define fixtures if the test will change any data in the database and, if desired, init scripts defining how tables should be initialized at the beginning of a test suite and each test.

What are they?

Fixtures are PHP scripts stored in protected/tests/fixtures, each named after the table in the database to which it corresponds. Each fixture script contains a return statement; the returned value is an array that defines the contents of the table at the beginning of the test; each value of the array corresponds to a row in the table and is itself an array with [column name] => [value] entries. By default, no fixtures will be used for a test; the fixtures to use must be defined in the public fixtures attribute (array). For more information on fixtures and how to use them: see Testing: Defining Fixtures and Unit Testing in The Definitive Guide to Yii.

By default, at the beginning of a CDbTestCase test and before loading fixture data, the table in the test database will be completely emptied, its AUTO_INCREMENT value reset to 1.

Initscripts allow you to override that behavior; they are similar to fixtures, but define the initial state before loading fixture data.

As a rule of thumb, the more fixtures you define for a unit test, the more time the test will take to execute, as it means that more database operations will need to take place at the beginning of every test method. If you do not need to reset a particular table in a test case's tests, but need there to be preexisting data to test against, you should create an init script for the table(s) instead.

Note the following comparison between initscripts and fixtures:

Property Fixtures Init scripts
File path / name protected/tests/fixtures/{table name}.php protected/tests/fixtures/{table name}.init.php
When run/applied At the beginning of tests in test cases that use them
  • At the beginning of tests in test cases that use them, before fixtures, and:
  • At the very beginning of a test suite (before any tests run)
Structure of the returned array array([row alias] => array( [column name] => [column value])) array( array( [column name] => [column value]) )

To understand the behavior of init scripts, one need only look at X2FixtureManager::resetTable() which, in the case that an array is returned by the init script, inserts the data after truncating the table. Note that additionally, besides defining an initial database state (in its return value, as an array), an init script also may contain arbitrary PHP to be executed at the beginning of a test. This can include such things as filesystem and database schema operations, and thus extend the concept of the initial testing state beyond mere control over ephemeral database contents.

Automatically generate fixtures and init scripts: howto

Beginning in version 2.8, fixture scripts can be automatically generated from preexisting database contents using the "exportfixture" Yii console command. To use it:

  1. Open a command shell
  2. Change directory into the protected folder inside of the web application
  3. Run ./yiic exportfixture

You will then be prompted to enter some details about the data to be exported. You can also execute the command in one fell swoop; run ./yiic help exportfixture to see its invocation syntax.

Note: this will take data from the live database and not from the testing database.

Reference Fixtures

If your test case extends X2DbTestCase, you can specify that certain fixtures be loaded at the beginning of a test class only, as "reference" data that will not be changed but will be referenced. This makes running tests faster by skipping re-load of the data, while making fixture data easily accessible as a fixtures property. To do this, declare the static referenceFixtures method of your test case such that it returns an array similar to the fixtures property. For example, to load all the user/profile data for reference:

    public static function referenceFixtures(){
        return array(
            'profile' => 'Profile',
            'users' => 'User',
        );
    }

Alternate Fixtures

Using X2FixtureManager allows you to have more than one fixture per table, and to switch between distinct tests that need different data. You can do this by, in the declaration of a fixture, using a two-element array in place of a string to specify the model or table name. In the array, the first element is assumed to be the name of the active record model or table, whereas the second is a suffix that when appended to the table name and concatenated with ".php" gives the full fixture filename.

For example, if you had a fixture file x2_users.extra.php, and wanted to use that file instead of x2_users.php, you could use the following fixture definition:

    public $fixtures=array(
        'users'=>array('User','.extra'),
    );

Functional Testing with Selenium

An advanced PHPUnit/Selenium configuration involving multiple network hosts, operating systems and web browsers. PHPUnit (1) begins a test by starting browser sessions on a remote host running Selenium RC (2). Selenium then commands the web browsers to open the web application in testing mode (3) and run automated instructions, sending results back to PHPUnit (i.e. whether a test has passed or failed). All logos are © their respective owners.

Selenium provides the means to control and automate browser actions, and hence, to run tests with full web-based sessions and HTTP request data. Test cases that use Selenium extend WebTestCase and are run in the same manner as unit tests, but require that the following environmental conditions are met:

  1. The <selenium> section in the configuration file protected/tests/phpunit.xml is set properly. For information on how to configure this, see The PHPUnit Manual Appendix C: The XML Configuration File
  2. Selenium RC on all the hosts defined in phpunit.xml can be accessed from the web server on the specified ports (note: if they are behind a firewall, it will be necessary to set up NAT in order to properly use them)
  3. The URL defined as constant TEST_BASE_URL in protected/tests/WebTestConfig.php resolves properly (points to index-test.php/ on the web server).
  4. PHPUnit and the necessary extensions are installed on the web server

Creating Selenium test cases

  1. Create a class that extends WebTestCase in protected/tests/functional
  2. Note that at the beginning of the execution of a test case's method, the method WebTestCase::setUp() will be called, and this will create a session as the admin user. If any additional operations are necessary to include at the beginning of every test, you may override the method, but should include a call to parent::setUp() at the beginning of it.
  3. In the test method, open the URL where the test is to begin by calling WebTestCase::openX2(), passing it as an argument the URI[[wikipedia:Uniform Resource Identifier]]: The part of a URL that identifies the resource on the server to be accessed. In the context of the API, this refers to the relative path within the web server based in the web root of X2Engine, i.e. ''index.php/api2/Contacts/324.json'' as opposed to the full URL, which begins with the protocol (i.e. "http") and might also contain a path relative to the web site's document root relative to the base URL (what would come after index-test.php).
  4. Note the availability of CWebTestCase's methods, especially those inherited from PHPUnit_Framework_TestCase, PHPUnit_Framework_Assert and PHPUnit_Extensions_SeleniumTestCase, in addition to WebTestCase::session() (which will ensure the session matches the user/password defined in WebTestCase::$login, logging the browser out/in as necessary) and the aforementioned methods that will be inherited from WebTestCase.

Running test cases built with Selenium IDE

Building functional tests involving browser automation with Selenium can be done with PHPUnit browser commands, i.e. assertTextPresent. However, it is far more easy, convenient and quick to build them using Selenium IDE. Selenium IDE is a web browser plugin for Mozilla Firefox that provides a means of generating Selenium commands by capturing mouse and keyboard actions. It also allows you to edit and re-position commands after creating them, and boasts a multitude of other very useful features. To obtain Selenium IDE, visit the Selenium HQ downloads page.

To run a native Selenese .html test case created in Selenium IDE:

  1. Save the .html in the same folder as the PHP test case class (a subdirectory of protected/tests/functional)
  2. In a test method, open the URL where the Selenese test case is to begin using WebTestCase::openX2().
  3. Call the method WebTestCase::localSelenese() with the filename of the Selenese test case as the sole argument.

Selenium tests are easy to execute an arbitrary number of times; simply calling localSelenese() again will run the script a second time. Selenese test cases are thus optimal not only for typical tests due to how easy they are to generate, but also are good for tests that need to be repeated as different users with different permissions (in the case that similar behavior/display is desired for all sets of permissions).

Example

The following example method opens the "create" action in the contacts module, runs a Selenese test case called "TestCreate.html", logs out, logs back in as a different user ("user2", with password "password2") and runs the same test again:

class ContactsTest extends WebTestCase {
   // ...
   public function testCreate() {
      $this->openX2('contacts/create');
      $this->localSelenese("TestCreate.html");
      $this->login = array('username'=>'user2','password'=>'password2');
      $this->session();
      $this->openX2('contacts/create');
      $this->localSelenese("TestCreate.html");
   }
   // ...
}