I regularly bump into the Magento 2 integration test procedure failing, before it even reaches my own tests. Here are a couple of reasons why things might be failing, partially related to the setup of the testing framework, partially related to the extension code.
The why of integration tests
Running Magento 2 integration tests for your own extension is very valuable. Instead of only looking for unit tests or testing the entire output using functional tests or end-to-end tests, integration tests allow you to look at the parts where your extension is hooking into the rest of Magento.
Instead of seeing the HTML output, you would be looking at the Response object to see if it holds the right headers. Instead of seeing empty output due to unknown database issues, you can check to see if your repository is able to fetch data at all. Even better, all of this takes place in a controlled environment, with an empty database and data fixtures.
Setting up integration tests
The setup of integration tests is kind of cumbersome: You will need an empty database, you will need a clean Magento environment (where bad extensions that will not work with integration tests are removed) and you will need to follow a procedure to set things up.
This blog is not about this procedure of setting up integration tests. Instead, it hopefully helps you troubleshoot things when you bump into issues.
Database is simply not there yet
While starting the integration tests, your
install-config-mysql.php file will be pointing to a database. If that database is not there, you can expect the
setup:upgrade command to fail with message like
Connection refused or
No tables found or something similar.
What I usually do is connect to the same MySQL instance using a command like
mysqladmin ping. In a CI/CD environment, such a command is definitely handy, for instance when both your tests and the database are run in Docker containers, but you want to make sure that the MySQL container is up first.
Database is empty while TESTS_CLEANUP is disabled
It might also be that the database is there but is empty, while the
TESTS_CLEANUP flag in your
phpunit.xml file is disabled. When the flag is enabled, Magento will run a setup on that empty database, but when the flag is disabled, it assumes that the setup has already been run.
I find this pretty annoying, I'm still hoping to provide some fix so that the flag can be disabled (for improved performance) but that the setup is simply run anyway when the database requires so.
SQLSTATE[42S02]: Base table or view not found
The most common issue that you might encounter is actually not due to your database setup, even though it tells you something is wrong with your database. For instance, a SQL error like
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'magento2.eav_entity_type' doesn't exist might be given.
A common reason why this is not working is because your module is not mentioning its dependencies properly. For instance, as soon as your module is active in either frontend or backend, chances are that your module needs to declare
Magento_Store as a dependency in your
etc/module.xml file. Similarly, if you're using dependencies from the catalog module, you should declare
Magento_Catalog as a dependency.
Check your dependencies with Yireo ExtensionChecker
To help you track dependencies, I have developed an open source module Yireo ExtensionChecker that helps you with this. You can simply run the command
bin/magento yireo_extensionchecker:scan Foo_Bar to see if the dependencies of
Foo_Bar match up with its
Not using proxies in your CLI commands
Another pitfall is that your module might contain a couple of CLI commands, with yet other dependencies. However, if those commands contain yet other dependencies and those dependencies require a working database, then it might be that the
bin/magento command stops working. And because the integration tests require a command like
bin/magento setup:upgrade to be working, the tests fail early again.
As a rule of thumb, it is best to create a
di.xml file that creates a
type for each command class, so that the original constructor arguments are replaced with proxy objects. If you are picky, you can try to do this only for those dependencies that require the database (like a repository), leaving out the others. Also, make sure not to use these dependencies in your
configure() method (which will break the
bin/magento command again) but only within the
tasklist.exe not found
Perhaps you have bumped into the
tasklist.exe not found error as well, like I did. It is not an error that is run before your integration tests, but actually after. But in a CI/CD environment, it might still break the tests. The error is generated by a method
Magento\TestFramework\Helper\Memory::getRealMemoryUsage() that tries to sum up the consumed memory and report it to the bottom of your tests output.
Unfortunately, the flags of
ps differ per platform. And depending on the environment where you're running these tests under, it might break things or not. Even stranger, when the UNIX command
ps fails, the PHP code simply tries again with a Windows command
tasklist.exe which is definitely not present in my Linux-based Docker containers. I usually patch the method by returning
42 or by letting it
die() when the exception occurs. A bit strange, but it helped me to get going in various CI environments.
Hopefully I documented a couple of ways of troubleshooting your integration testing setup that are useful to you. Feel free to comment on additional issues below.