An assertion is an expression that must return a Boolean (
true if the
It always relies on metric values, operators, units, functions, and variables and must be written in the tests section of the .blackfire.yaml file:
1 2 3 4 5 6 7 8 9 10 11 12
tests: 'All pages are fast': path: '/.*' assertions: - 'main.wall_time < 50ms' - 'main.peak_memory < 10mb' - 'main.network_out < 10kb' 'Not too many SQL queries on the homepage': path: '/(en|fr)/blog/' assertions: - 'metrics.sql.queries.count < 5'
In the above example,
main.peak_memory are metric values. A unit like in
10mb can be set
explicitly to override default units.
Whenever possible, we recommend you write assertions that do not depend on time. The main reason is that time is always a consequence, a symptom of a deeper issue.
Assertions support profile comparison as well to assert the performance evolution of your code:
1 2 3 4 5 6
tests: "Pages should not become slower": path: "/.*" assertions: - "percent(main.wall_time) < 10%" # time does not increase by more than 10% - "diff(metrics.sql.queries.count) < 2" # less than 2 additional SQL statements
Comparison assertions are only evaluated when running builds.
Values are compared between the current build and a reference build.
The reference can be either the last successful periodic build or a build that has been referenced in
the webhook command with
Blackfire exposes metrics that are associated with the current
profile. The value of one dimension of a metric can be used in assertions;
for instance, the
count value of the
sql.queries metric is stored in
Learn more about all built-in metrics exposed in assertions by default and how to create your own.
The available dimensions for metrics are the following ones:
For each metric (count, time, or memory), a default unit is defined:
10 in an assertion for a time value, that evaluates to 10
milliseconds. It is possible to add an explicit unit to a metric, which will
override the default unit.
The following time units are supported:
The following memory units are supported:
The following generic units are supported:
The following operators are supported in assertions:
<=less than or equal to;
>=greater than or equal to.
Variables are useful when you configure several Blackfire environments to run performance tests on various machines
hosting the same application. Variables make it possible to use the same
.blackfire.yaml configuration on several servers that
have different purposes (e.g. development, staging, production).
Variables can be defined in the environment configuration and used in an
expression by passing the metric key to the
Variable values may also use units.
Example: You have configured two different environments in Blackfire:
Integrationhits your integration server, where your application runs with debug mode on;
Productionhits your production servers, where your application runs with debug mode off.
The same application runs on all servers, but the debug mode increases memory usage a lot. As such, you cannot set the same maximum value for your assertions on memory.
.blackfire.yaml would look like this when using variables:
1 2 3 4 5
tests: "Pages shouldn't use too much memory": path: "/.*" assertions: - "main.peak_memory < 10mb * var('memory_coeff')"
Now in both your
Production Blackfire environments,
you have to create the
memory_coeff == 2
memory_coeff == 1
var() function may also receive a default value as a second
This default value is used whenever the variable is not defined in one of your
1 2 3 4 5
tests: "Pages shouldn't use too much memory": path: "/.*" assertions: - "main.peak_memory < 10mb * var('memory_coeff', 1)"
vars.xxx notation, supported in older versions of Blackfire, is now
deprecated. We strongly advise you to migrate your expression to use
var() function instead.
An optional description can be added to an assertion. This could provide a context or an educational note left for the other delevopers of your team.
This note can help understand the performance challenges of some parts of the application.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
tests: "The autoloader classmap should be dumped": path: "/.*" assertions: - "metrics.composer.autoload.find_file.count == 0" description: | By default, the Composer autoloader runs relatively fast. However, due to the way PSR-4 and PSR-0 autoloading rules are set up, it needs to check the filesystem before resolving a classname conclusively. This slows things down quite a bit, but it is convenient in development environments because when you add a new class it can immediately be discovered/used without having to rebuild the autoloader configuration. https://getcomposer.org/doc/articles/autoloader-optimization.md
The description is displayed on the Assertions tab of a Profile:
And within Build reports:
When an assertion is run from an environment, the
is_dev() function returns
false when the environment is configured for production usage.
When using builds, it is possible to compare one build to another. This is useful when you want to validate a code merge (e.g. a pull-request) by triggering a build webhook or a Blackfire Player collection of scenarios. It can also ensure that your code doesn’t have performance regression over time using periodic builds.
Comparison can be made within assertion expressions thanks to the
If you want to assert that a metric value does not increase by more than a
given percentage between two builds, use the
"percent(main.wall_time) < 10%"
You can also test the evolution of a metric value in absolute terms with the
"diff(metrics.sql.queries.count) < 2"
In this example, the assertion checks that the profile from the newer build has less than 2 additional SQL statements compared to the previous one.