"X test fails intermittently on Solano. This isn't a problem locally, so what's going on?"
Unfortunately, unexpected failures sometimes occur within Solano CI. There are two main reasons: Solano CI’s parallelism, and your test’s interaction with asynchronous behavior; below are some frequent offenders related to each reason. (For more fleshed out descriptions, please see our documentation.)
- Solano CI Environment Variable
- Hardcoded Ports
- Hardcoded Paths
- File Upload Tools
- FileStore Caching
- OpenID (e.g., with Omniauth)
- Analytics Agents (e.g. Kissmetrics)
Other Causes of Failures:
- Selenium and Asynchronous Behavior
- Disabling Garbage Collection
To properly identify the cause of the failures, we recommend three main troubleshooting steps.
- Isolate the test
- Check for dependency issues
- Check for concurrency issues
You can learn more about these steps and more from our troubleshooting guide.