Performance of PHP versions

The arrival of Facebook HipHop machine (HHVM) awakened the PHP project and introduced an healthy competition between the different projects. So you will see that the performance is now an important objective to the extent that the execution times were divided by three between PHP 5.3.10 and PHP 5.6.4.

However, the negative effect of this competition is the publication of biased or unrealistic tests. Indeed, many tests compare the performance of different PHP runtimes with the calculation of Fibonacci Sequence, the encoding json or the reading of an WordPress article (without plugin). This may interest some projects, but most people want to know what the behaviour of the different alternatives with a real composite application. Then we found it necessary to qualify the execution speeds announced with the latest versions.

Méthodology

We have created a JMeter script that you can find on the project repository. In order to exclude a part of the network time, static components (JS, CSS, images, etc.) are not downloaded because we are only interested in the execution time of the various PHP engines. Similarly, this test doesn't write in database.

The database was loaded with 2,000 users and 7,000 leave requests (see the Excel file in the given previous link).

We used a VPS hosted by OVH with one VCPU and 1GB of RAM. We limited the test to 30 concurrent users and pauses of less than 10 seconds between steps. These limits were calculated so as to remain below 60% CPU time, ie the server has never been in a situation of saturation. PHP machines are started with their default configuration and as a FastCGI server.

The test takes about 30 minutes and the compared response times are those of the 90th percentile. Put simply, we compare what happens in 90% of cases by excluding the most extreme samples.

You can read the conclusion of this article for more technical information.

Summary of results

There is a significant performance improvement starting from version 5.4 of PHP. On one hand, the response times of previous versions are poor and, on the other hand, the latest versions consume less CPU power for a faster result.

We recommend installing Jorani with version 5.6 of PHP. We strongly encourage you to update your environment in the case of an earlier version.

Version 7 is still being developed and its performance could be improved before its release (scheduled for year-end 2015). However, do not expect a significant compression of the response time.

Transaction	PHP 5.3.10	PHP 5.4.41	PHP 5.5.25	PHP 5.6.4	PHP 7-dev	HHVM
	-331%	-105%	-47%	base	-5%	-31%
Login Form	3188	2431	2369	2117	2102	2919
Login Action	3179	1254	908	376	383	344
Home	893	406	204	136	155	175
List of leaves	1399	714	506	298	326	402
Export leaves	2652	998	492	173	286	188
View counters	1393	623	238	144	173	192
View calendar	1252	624	313	201	228	300
Leaves feed for FullCalendar	1146	471	182	110	146	133
Show day offs in calendar	1037	389	188	118	140	152
tabular calendar	2292	1104	647	415	434	476
Show organization	1376	635	371	230	251	323
Root node of organization	1629	508	199	142	138	166
List of employees in a node	1256	397	180	110	133	138
Supervisor of a node	977	359	172	106	135	134
Request List	2166	1440	1417	1027	986	1429
View a leave	1198	588	215	144	177	200
ICS of day offs	2417	756	625	290	293	317
ICS of global calendar	1945	749	604	399	376	371
Logout	1600	734	396	245	298	317
API get access token	701	395	203	139	175	154
API get list of employees	2014	1371	1715	1365	1385	1985

Matrix of technical components

As explained above, the existing benchmarks rarely involve a complete application (and composed of several external dependencies). If it helps you in your own project, we list here the various third party libraries used by the different transactions.

Transaction	BCRYPT	phpseclib	PHPExcel	Oauth2	VObject	(json)
Login Action
Export leaves
Root node of organization
List of employees in a node
ICS of day offs
ICS of global calendar
API get access token
API get list of employees

Conclusion

About load testing

Beware that the performance test used for writing this article does not reflect the actual conditions of Jorani usage. The script puts the application under stress and calls features not necessarily used in your organization (eg API) and with high call frequencies. This is a benchmark — or a technical comparison.

However, you might adapt this JMeter script in order to create your own load test. Testing the application performance is important — among other objectives — if you want to validate the sizing of your hosting environment.

Runtime Settings

This did not matter in the interpretation of results, but the web server was nginx and the database server was MariaDB. Both of them in their latest version and on the same server (the overhead of these two components was around 5%).

HHVM was launched in server mode.

hhvm --mode server -vServer.Type=fastcgi -vServer.Port=9000

PHP was started in server mode with 4 workers for versions 5.3 to 5.6, while the version 7 required 8 workers (otherwise it would crash).

PHP_FCGI_CHILDREN=4
export PHP_FCGI_CHILDREN
/usr/bin/php-cgi  -b 127.0.0.1:9000

It is possible to optimize the various parameters and the runtime environment (for example by using a socket), but it is difficult to find documentation. Especially for HHVM where tuning environments is not very well explained by the project. For PHP, the situation is different because the gap in documentation comes from significant changes in the latest versions. Indeed, many papers and articles have been made obsolete by the latest versions.

Do not hesitate to contribute by sharing your optimization tricks.

— Benjamin BALET

Tags : configure