Prev Next

Chapter 10. Incomplete and Skipped Tests

Incomplete Tests

When you are working on a new test case class, you might want to begin by writing empty test methods such as:

public function testSomething()
{
}

to keep track of the tests that you have to write. The problem with empty test methods is that they are interpreted as a success by the PHPUnit framework. This misinterpretation leads to the test reports being useless -- you cannot see whether a test is actually successful or just not yet implemented. Calling $this->fail() in the unimplemented test method does not help either, since then the test will be interpreted as a failure. This would be just as wrong as interpreting an unimplemented test as a success.

If we think of a successful test as a green light and a test failure as a red light, we need an additional yellow light to mark a test as being incomplete or not yet implemented. PHPUnit_Framework_IncompleteTest is a marker interface for marking an exception that is raised by a test method as the result of the test being incomplete or currently not implemented. PHPUnit_Framework_IncompleteTestError is the standard implementation of this interface.

Example 10.1 shows a test case class, SampleTest, that contains one test method, testSomething(). By calling the convenience method markTestIncomplete() (which automatically raises an PHPUnit_Framework_IncompleteTestError exception) in the test method, we mark the test as being incomplete.

Example 10.1: Marking a test as incomplete

<?php
require_once 'PHPUnit/Framework.php';
 
class SampleTest extends PHPUnit_Framework_TestCase
{
    public function testSomething()
    {
        // Optional: Test anything here, if you want.
        $this->assertTrue(TRUE, 'This should already work.');
 
        // Stop here and mark this test as incomplete.
        $this->markTestIncomplete(
          'This test has not been implemented yet.'
        );
    }
}
?>


An incomplete test is denoted by an I in the output of the PHPUnit command-line test runner, as shown in the following example:

phpunit --verbose SampleTest
PHPUnit 3.4.2 by Sebastian Bergmann.

SampleTest
I


Time: 0 seconds

There was 1 incomplete test:

1) testSomething(SampleTest)
This test has not been implemented yet.
/home/sb/SampleTest.php:14

OK, but incomplete or skipped tests!
Tests: 1, Assertions: 0, Incomplete: 1.

Table 10.1 shows the API for marking tests as incomplete.

Table 10.1. API for Incomplete Tests

MethodMeaning
void markTestIncomplete()Marks the current test as incomplete.
void markTestIncomplete(string $message)Marks the current test as incomplete using $message as an explanatory message.


Skipping Tests

Not all tests can be run in every environment. Consider, for instance, a database abstraction layer that has several drivers for the different database systems it supports. The tests for the MySQL driver can of course only be run if a MySQL server is available.

Example 10.2 shows a test case class, DatabaseTest, that contains one test method, testConnection(). In the test case class' setUp() template method we check whether the MySQLi extension is available and use the markTestSkipped() method to skip the test if it is not.

Example 10.2: Skipping a test

<?php
require_once 'PHPUnit/Framework.php';
 
class DatabaseTest extends PHPUnit_Framework_TestCase
{
    protected function setUp()
    {
        if (!extension_loaded('mysqli')) {
            $this->markTestSkipped(
              'The MySQLi extension is not available.'
            );
        }
    }
 
    public function testConnection()
    {
        // ...
    }
}
?>


A test that has been skipped is denoted by an S in the output of the PHPUnit command-line test runner, as shown in the following example:

phpunit --verbose DatabaseTest
PHPUnit 3.4.2 by Sebastian Bergmann.

DatabaseTest
S


Time: 0 seconds

There was 1 skipped test:

1) testConnection(DatabaseTest)
The MySQLi extension is not available.
/home/sb/DatabaseTest.php:11

OK, but incomplete or skipped tests!
Tests: 1, Assertions: 0, Skipped: 1.

Table 10.2 shows the API for skipping tests.

Table 10.2. API for Skipping Tests

MethodMeaning
void markTestSkipped()Marks the current test as skipped.
void markTestSkipped(string $message)Marks the current test as skipped using $message as an explanatory message.


Prev Next
1. Automating Tests
2. PHPUnit's Goals
3. Installing PHPUnit
4. Writing Tests for PHPUnit
Test Dependencies
Data Providers
Testing Exceptions
Testing PHP Errors
5. The Command-Line Test Runner
6. Fixtures
More setUp() than tearDown()
Variations
Sharing Fixture
Global State
7. Organizing Tests
Composing a Test Suite Using the Filesystem
Composing a Test Suite Using XML Configuration
Using the TestSuite Class
8. TestCase Extensions
Testing Output
9. Database Testing
Data Sets
Flat XML Data Set
XML Data Set
CSV Data Set
Replacement Data Set
Operations
Database Testing Best Practices
10. Incomplete and Skipped Tests
Incomplete Tests
Skipping Tests
11. Test Doubles
Stubs
Mock Objects
Stubbing and Mocking Web Services
Mocking the Filesystem
12. Testing Practices
During Development
During Debugging
13. Test-Driven Development
BankAccount Example
14. Behaviour-Driven Development
BowlingGame Example
15. Code Coverage Analysis
Specifying Covered Methods
Ignoring Code Blocks
Including and Excluding Files
16. Other Uses for Tests
Agile Documentation
Cross-Team Tests
17. Skeleton Generator
Generating a Test Case Class Skeleton
Generating a Class Skeleton from a Test Case Class
18. PHPUnit and Selenium
Selenium RC
PHPUnit_Extensions_SeleniumTestCase
19. Logging
Test Results (XML)
Test Results (TAP)
Test Results (JSON)
Code Coverage (XML)
20. Build Automation
Apache Ant
Apache Maven
Phing
21. Continuous Integration
Atlassian Bamboo
CruiseControl
phpUnderControl
22. PHPUnit API
Overview
PHPUnit_Framework_Assert
assertArrayHasKey()
assertClassHasAttribute()
assertClassHasStaticAttribute()
assertContains()
assertContainsOnly()
assertEqualXMLStructure()
assertEquals()
assertFalse()
assertFileEquals()
assertFileExists()
assertGreaterThan()
assertGreaterThanOrEqual()
assertLessThan()
assertLessThanOrEqual()
assertNotNull()
assertObjectHasAttribute()
assertRegExp()
assertSame()
assertSelectCount()
assertSelectEquals()
assertSelectRegExp()
assertStringEndsWith()
assertStringEqualsFile()
assertStringStartsWith()
assertTag()
assertThat()
assertTrue()
assertType()
assertXmlFileEqualsXmlFile()
assertXmlStringEqualsXmlFile()
assertXmlStringEqualsXmlString()
PHPUnit_Framework_Test
PHPUnit_Framework_TestCase
PHPUnit_Framework_TestSuite
PHPUnit_Framework_TestResult
Package Structure
23. Extending PHPUnit
Subclass PHPUnit_Framework_TestCase
Assert Classes
Subclass PHPUnit_Extensions_TestDecorator
Implement PHPUnit_Framework_Test
Subclass PHPUnit_Framework_TestResult
Implement PHPUnit_Framework_TestListener
New Test Runner
A. Assertions
B. Annotations
@assert
@backupGlobals
@backupStaticAttributes
@covers
@dataProvider
@depends
@expectedException
@group
@outputBuffering
@runTestsInSeparateProcesses
@runInSeparateProcess
@test
@testdox
@ticket
C. The XML Configuration File
PHPUnit
Test Suites
Groups
Including and Excluding Files for Code Coverage
Logging
Test Listeners
Setting PHP INI settings, Constants and Global Variables
Configuring Browsers for Selenium RC
D. Index
E. Bibliography
F. Copyright