Web Development

How we found slow API requests on a WordPress site

and got more than x40 speed boost after making fixes

Background

We are developing an educational platform that allows you to:

Сreate lesson presentations with slides with different types of content

Share them with other teachers

Clone and edit content to adapt to different age groups

Show these presentations to students and receive feedback from them in the form of answers to questions

All actions with content involve large amounts of data and active communication with the database.

Sometime after the test launch of the project, we found that the loading, saving, and cloning of lessons began to take too much time. This prompted us to study the problem in detail and search for solutions based on the identified bottlenecks.

Platform technical stack

Code

We used WordPress as the engine and the famous LearnDash plugin to implement the Learning Management System (LMS) functionality. This combination allowed the client to develop the necessary functionality and launch a demo to test the perception by end users in a short period of time. In the process of changing the requirements, we had to abandon most of the features that LearnDash provides and restructure the approach to creating content – replacing courses with lessons and lessons with slides.

We chose Vue.js as the front-end framework, which allowed us to make the UI of the main functionality of the project responsive. That is to say, the functionality of the editor and the presentation of lessons. The peculiarity of the implementation lay in the construction of communication between the client and the server in the form of API requests. The requests are sent in large numbers due to the approach to managing posts in the selected CMS (WordPress), so reducing the number of requests was not possible. However, parallelization and asynchronous execution made it possible to achieve the desired results in performance.

Server

The scalable architecture based on AWS allowed us to make sure that the server capacity was sufficient to process a large flow of users and guarantee decent performance at peak loads. However, this time the situation turned out to be so unpredictable that horizontal and vertical scaling did not give any better results.

Finding the problem and fixing it

Problem

The initial description of the problem explained that the developed front-end lesson editor as well as the pages for editing records in the administrative panel were taking too long to load. The first thing that came to mind was that the admin panel was written in React and used API requests to fetch categories, taxonomies and tags. Our lesson editor also used API requests to read and write lesson content.

After exploring the “network” tab in the browser dev tools, we found that all asynchronous requests to the server worked as expected, with the exception of API requests that took at least 3 seconds each. A detailed view of the request showed a high TTFB (time to first byte), indicating slowness on the backend or server-side.

It was empirically calculated that no matter what actions the API endpoint performed, the result would always be the same – at least 3 seconds for processing, increasing to 5-6 seconds with the growth of the database.

Of particular note, production turned out to be the slowest phase because it had the largest amount of data – posts and metadata for the posts. In order to verify that the load speed decreased as the amount of data increased, the production base was deployed in a test environment where we received confirmation of this theory.

We constantly asked ourselves the same question:

Why is the speed of an API request that does not use the database when processing internal logic being affected by the size of the database?

Solution

So, it was decided to turn off the plugins used one by one in order to find out their impact on the load speed. The first result came after the deactivation of the LearnDash plugin, the content of which is the basis of the entire platform. The processing speed of API requests was instantly 40 times faster! However, based on the impossibility of abandoning the plugin and resorting to the search for analogs, the investigation continued.

Going through the LearnDash plugin code, starting with initialization and including files, we commented on large blocks of code and checked their impact on the speed. Particular attention was paid to the REST API functionality, where the first hook appeared. By fully commenting out the use of the REST API by the plugin, the issue was resolved. However, this was also not a viable solution. Moving along the chain, we did find the reason for this behavior.

The LearnDash plugin was using the rest_api_init WordPress hook not only to register its own API endpoints and fields to them but also to include additional controllers with logic, one of which contained a huge unoptimized sequential selection from several tables in the database. Thus, regardless of the route API function, this selection was performed every time we sent an API request to the server (and, as mentioned above, a lot of requests are sent). The reason for this approach was not clear, however, a request for clarification was sent to the LearnDash plugin developers with the hope of a fix in future releases in mind.

The presence of a filter in the plugin code (provided by the LearnDash support team in response to our ticket) helped to fix the situation without problems and allow the exclusion of controllers loaded with logic before they were loaded. This is what we did for the problematic controller.

Results:

First of all, the problem with the slow operation of API requests was resolved. Now they load even faster than before on the minimum data set. The overall increase in load speed on the current amount of data is 40 times!

Thanks to getting rid of the “heavy” fetch with each API request, we managed to reduce the overall load on the database server, which is a nice bonus and justified the time spent looking for the bottleneck.

Based on information provided by CitrusDev WordPress Team Lead, the LearnDash support team responded to our ticket and a plugin update was provided. These improvements were integrated into the basic plugin source code. As a result, users around the world of the LearnDash plugin and WordPress REST API obtained a great speedup on their websites. That is why it is so critical to update WordPress and all plugins in a timely manner.