Blackfire gathers a lot of data about how your code behaves at runtime.
Tests allow to write assertions on this data. You can write
performance tests with Blackfire by writing assertions on time dimensions
(wall-clock time, I/O time, and CPU time), but whenever possible, we recommend
you to write assertions that do not depend on time.
The main reason is that time is always a consequence, a symptom of a deeper
issue. In addition, time is not stable from one profile to another, which
makes your tests volatile.
Instead, find what is slowing down your application and write assertions on
it, i.e. the number of SQL queries, the number of times a function is
called, or the memory usage.
Moreover, Blackfire tests are a great way to test legacy applications.
To get started, create a .blackfire.yaml file in the root directory of an
"Pages should be fast enough":
path: "/.*" # run the assertions for all HTTP requests
- "main.wall_time < 100ms" # wall clock time is less than 100ms
"Commands should be fast enough":
command: ".*" # run the assertions for all CLI commands
- "main.wall_time < 2s" # wall clock time is less than 2s
You can bootstrap a new .blackfire.yaml file with the following
command: blackfire bootstrap-tests
Validate .blackfire.yaml files via the Blackfire online
.blackfire.yaml is a YAML file where tests, metrics, and scenarios can be defined.
A test must located under the tests key and is composed of the following
Here is another example with several assertions limited to some API calls of the
"Homepage should not hit the DB":
methods: [GET, HEAD]
- "metrics.sql.queries.count == 0" # no SQL statements executed
- "main.peak_memory < 10mb" # memory does not exceed 10mb
- "metrics.output.network_out < 100kb" # the response size is less than 100kb
When a profile is made on a project that contains a .blackfire.yaml file,
Blackfire automatically runs all tests matching the HTTP request path. The
result of the tests is displayed as a green or red icon in the dashboard and
the full report is available on the profile page. The same goes when
profiling a CLI script via blackfire run.
Note that assertions in the report contain the actual metric and variable
values so that you know if you are close to the target or not
(metrics.sql.queries.count 5 == 0; 0 is the target, 5 is the actual number
of SQL statements executed).
metrics.sql.queries.count 5 == 0
Assertions support profile comparison as well
to assert the performance evolution of your code:
"Pages should not become slower":
- "percent(main.wall_time) < 10%" # time does not increase by more than 10%
- "diff(metrics.sql.queries.count) < 2" # less than 2 additional SQL statements
Comparison assertions are only evaluated when running builds. Values are
compared between the current build and a reference build.
The reference can be either the last successful build, or a build that
has been referenced in the webhook command with
Read the assertion reference guide to
learn more about the Blackfire assertion syntax.
A condition is an expression similar to an assertion. If the condition is fulfilled, the test is
There are two kinds of conditions: when and unless.
They can be used separately or combined for a more complex conditions.
A when expression acts like an if condition. The corresponding test is
evaluated if the expression returns true.
'A database connection should be opened only when queries are made':
when: "metrics.sql.connections.count > 0"
- 'metrics.sql.queries.count > 0'
The example above means “For any HTTP request, if a database connection is
used, at least 1 SQL query must be run”.
An unless expression acts like an if not condition. The corresponding
test is evaluated unless the expression returns true.
'Twig template cache should be enabled in production':
- 'metrics.twig.compile.count == 0'
The example above means “For any HTTP request, unless the profile is run in
a development environment, Twig template path should be enabled”.
# A path or an array of paths, when in web context;
# Regular expression the web path must match to trigger the assertions.
# A path or an array of paths to exclude, when in web context;
# A command, when in CLI context;
# Regular expression the command must match to trigger the assertions.
command: bin/console project:cron
# A collection of HTTP Methods.
# Use ANY for accepting anything (it's the default value).
methods: # Example: [ GET, POST ]
# A condition. If not valid, the test will not be used.
when: ~ # Example: is_dev() == true
# A condition. If valid, the test will not be used.
unless: ~ # Example: is_dev() == true
# A collection of assertions
label: ~ # Example: No more than 5 SQL query
expression: ~ # Required, Example: metrics.sql.queries.count < 5