Day 16 - Performance Automation

Do you know why most developers don't manage the performance of their applications? Many will tell you they don't have the time, but the real reason is probably that they don't have the right tools. Blackfire fills this gap. Being able to easily profile applications in development and diagnose problems in production is great, but if the process is manual, developers will stop doing it after a while.

Automation is key to continuously manage application performance.

Automation is also key to avoid performance regressions in production. Blackfire provides a feature that helps developers trigger performance tests on an application and be alerted whenever a problem occurs.

Writing Scenarios

Manually triggering profiles on individual URLs like we did previously works well on development machines, but it does not scale well for production monitoring.

Blackfire scenarios let you run a set of profiles on your application's main URLs or API endpoints. Scenarios are defined in .blackfire.yml under the scenarios section.

GitList scenarios could look like the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
scenarios:
    Homepage:
        - /
    Twig Homepage:
        - /Twig
    Twig Commits:
        - /Twig/commits
    Twig Stats:
        - /Twig/stats
    Twig Network:
        - /Twig/network
    Twig Network Ajax
        - /Twig/network/1.x
    Search:
        - /Twig
        - path: /Twig/tree/1.x/search?query=loader
          method: POST
          samples: 10
          warmup: true

Each scenario has a name and a list of URLs to profile; but only the last URL of each scenario is profiled.

In the example, all scenarios, but the last one, trigger profiles on HTTP GET requests. For each one of them, Blackfire warms up the URL by hitting it a few times before generating a profile out of several iterations.

The "Search" scenario is more interesting as it requests a POST request, but a safe one. By default, scenarios for non-GET requests have no warmup and profiles are generated from only one iteration. But as GitList search is idempotent, the scenario is explicitly configured to enable warmup (warmup: true) and to generate 10 requests (samples: 10) for the profile.

Note

Blackfire scenarios have more options as described in the scenarios documentation.

Triggering Scenarios

Now that the GitList scenarios are defined, we need a way to trigger them. Blackfire has several ways to trigger them, but the most flexible one is to use a webhook.

Go to the Dashboard's Builds tab of your environment and click the "Start a build" button:

/docs/blackfire-triggers.png

Then enter the endpoint (http://gitlist.demo.blackfire.io/) and trigger the build by submitting the form:

/docs/blackfire-triggers-submit.png

Then wait for the profiles to finish.

Alternatively, you can also copy and paste the generated URL at the bottom of the form in a console:

1
2
3
4
curl -i -X POST https://blackfire.io/api/v1/build/env/__UUID__/webhook \
--user 'CLIENT-ID:CLIENT-TOKEN' \
-d 'endpoint=http://gitlist.demo.blackfire.io/' \
-d 'title=My First Scenario!'

Note

Scenarios and triggers are available for any environment. To use the command above, simply replace the __UUID__ placeholder with the UUID of one of your Blackfire environments.

If the trigger fired correctly, the JSON output should contain "A new build has been started". Go to the Blackfire dashboard, then to the "Builds" tab and select the newest build:

/docs/gitlist-builds.png

With just one simple request, we were able to automatically generate 7 profiles in parallel for the main GitList URLs.

The build report displays all profile results, highlights failed scenarios, and provides details for any failed assertions:

/docs/gitlist-build-master.png

As expected, some scenarios fail. Next execute the scenarios on the fix2 branch (from the web or from the console), where our performance patches have been applied:

1
2
3
4
curl -i -X POST https://blackfire.io/api/v1/build/env/__UUID__/webhook \
--user 'CLIENT-ID:CLIENT-TOKEN' \
-d 'endpoint=http://fix2-ijtxpsladv67o.eu.platform.sh/' \
-d 'title=Scenario on the fix2 branch'

The results are definitely better, but not as good as we could have hoped. Have a closer look and you will realize that some pages are slower than expected. This is the time to dig into the root causes and try to find more optimizations.

Note

Remember that the main benefits of storing scenarios in a .blackfire.yml file alongside your code is to make them specific to your current work: a pull request, a branch, a specific version of your code, etc. Whenever you add a new feature, don't forget to update the scenarios.

Being notified

Web hooks are a great way to integrate Blackfire into any tool. However, once your checks are automated, you will need a way to be alerted when performance degrades.

Blackfire notification channels alert you when a build fails or when a project's status changes. Blackfire comes with many built-in notification channel, but the simplest one is the email notification channel.

On the dashboard Builds view, add an email notification channel. Configure the email notification channel to receive an email whenever there is a failure or when the build status changes:

/docs/blackfire-notifiers-email.png

Conclusion

In development, update your application scenarios whenever you make significant changes.

Configure your test environment to run the scenarios via the webhook and use the email notification channel to receive a notification whenever a build fails.

For production, configure the Blackfire "Periodic builds" to automatically profile your application on a periodic basis and use the Slack, Hipchat or Microsoft Teams notification channel to get fast feedback.

But there is more. Builds are also available in the PHP SDK, which opens the door to dynamically building scenarios. The SDK is the best way to leverage Blackfire powerful features, and in the next chapter we will study some advanced usages.