Testing Symfony 5 applications - Test automation

Details
Category: Programming internals

Article Index

Page 6 of 6

Test automation

Having tools like automatic tests and code checks is great. But it is even better, when these are used automatically. Because of that, I use a Jenkins continuous integration server to automatically run whenever I check in a new version of the source code. I have even two Jenkins servers running: one on my laptop and one on my server. The first one helps me even when I am not at home, the latter one runs all the time and is the "final instance" to judge if the software also runs on a different system, which maybe has a different php and tool versions. Here is picture of one of the jenkins instances:

This is based on a template made by Sebastian Bergmann, the man behind some of the quality tools in php like phpunit. On the website for this, he explains how to setup jenkins and the project to be used by php projects. Using jenkins to automate the quality checks is done in a 2-step-process:

  1. Create all the information files on the project by running tools like phpunit, php mess detector, style checks, etc.
  2. Visualise this information on the jenkins server.

Automating all build steps

I use the build system ant, which is a project maintained by the Apache community (most know by the Apache HTTP/web server). In this system, you create a file called build.xml, in which all build steps are defined. By calling ant you can run one or more of these steps automatically. My build.xml is located in the root directory of the project. I defined two different build runs, one running all the tools except the coverage run of phpunit (build-fast), and one including this (build-coverage). Background was that the coverage run took more than an hour, while the normal run took 30 minutes (on my quite slow Intel Pentium Silver J5005 CPU). Since upgrading to php 7.4, phpunit 9.3 and symfony 5, these runs only take 7 and 8 minutes. I will probably remove the coverage run in the future.

Here are the steps done in my build:

Tool / target name build-fast build-coverage Purpose
install-tools x x This installs the current version of all needed tools. I moved from using composer to phive to install tools recently, because managing the dependencies in composer for the tools was a real pain. In phive, the tools are managed independently of the versions of the projects libraries, which makes sense. With phive I can manage all but one tool: php code browser unfortunately is not yet downloadable by phive. Here I download it directly from the internet page.
 prepare x  
  • Clear all files generated in the previous run, except those for the test coverage.
  • Set the time stamps of the test coverage files to the current time, so they can be used without warnings.
  • Create all directories needed to store the files created by the tools to be run.
  • Add the coding style (symfony2) to the php checkstyle tool.
prepare-coverage   x
  • All steps from prepare.
  • Remove coverage files and create needed directories
  • Run a special command to generate a list of files for which the coverage has to be generated. This filters out for example all the third-party-code, which makes a coverage run more than twice as fast.
db-init x x
  • Clear all sqlite cache files.
  • Drop the database (in case, we do not use sqlite for the tests).
  • Create the database and the database schema.
  • Load the fixture data.
lint x x Check the php code for any syntax errors.
pdepend x x Create a lot of metrics on the source code.
phpmd x x PHP mess detector is a static code analysis, which checks for typical programming errors and code smells like code which is too complex (cyclomatic complexity).
phpcpd x x PHP copy/paste detector checks if identical pieces of code are found in different parts of the software. Copy and pasting pieces of code instead of creating methods / services for that is a code smell, because it reduces the maintainability of the code.
phpcs x x PHP code sniffer checks if the code style complies to the defined code style (symfony2 in my project).
phploc x x This measures the size and structure of the project, e.g. it shows how many lines of code the project has.
phpcb x x PHP code browser generates an html structure for the source code.
phpdoc x x Generate the doxygen documentation for the entire project.
phpunit x   Run the unit and integration tests without generating coverage information.
phpunit-coverage   x Run the unit and integration tests including the generation of coverage information.

 The nice thing about ant is that all these targets can be run in one step, e.g. ant build-fast. But it is also possible to run one of the steps by its own or a block of steps, e.g. ant db-init to do all steps to initialise the database.

Visualising the results

This page describes which jenkins plugins have to be installed to visualise the results of the build. In Jenkins, there are a lot of so call post-build actions, which run the plugins to visualise the result. By using the template made by Sebastian Bergmann, this is quite easy. By now, there are several plugins marked as depricated. I haven't found the time yet to check if these already have been updated in the template.

Deployment

I run two production instances of this project: the one on this page and one for the Dutch-German Society. To update these, I created two jenkins jobs to do the deployment. They use the deployment script, which automates all the steps needed for deploying to a web space without ssh-access and git command line tool. Now I can just press a button at any time and the software including the database are updated on these two production servers.