Testing our SemanticHacker WordPress plugin has some similarities to testing foof, our Firefox extension, in that we are testing within another application. As with testing Firefox extensions, WordPress plugin testing must include testing on multiple operating systems and multiple versions of Firefox, and it adds the need to test on additional browsers. Because WordPress has been releasing frequent updates we’ve had to focus attention on how to quickly verify our plugin on each WordPress upgrade. As a result, we have two major types of testing for our WordPress plugin: testing a new release of the plugin and verifying our plugin in a new WordPress release.
Regardless of which type of test sequence we’re on, there are some things that we always have to test. We need to validate all supported browser and OS combinations and we need to test all functionality of the SemanticHacker plugin. This functionality includes the ability to use text in a blog post to find relevant content links, tags, webpage links, and products.
When testing a new release of our WordPress plugin, we have two user paths we need to test: An update of an older plugin release and a fresh install of the new version of the plugin. We run our tests on all versions for WordPress that we are supporting following both paths. Of course, if there is new functionality or bug fixes, we need to add test cases to cover those cases.
When there is a new WordPress release, we also consider two paths in which our plugin can appear in that version of WordPress: One is an existing instance of WordPress with the Semantic Hacker plugin is upgraded to the new version. The other is that our plugin is installed fresh on the version being tested. All tests are run on the new version of WordPress following both possible paths. Assuming the new WordPress release passes our tests, we add that version to our list of supported WordPress releases. At the same time we determine if there are older versions on the list for which it is no longer worthwhile to continue testing because they are too little used.
Tags: qa, quality assurance, WordPress