Tests pass file by file but not when run at once

Development
1

Hi, I am interested in contributing in PyMC’s development.
I cloned the repository and installed the environment with

conda env create -f conda-envs/environment-dev.yml
conda activate pymc-dev
pip install -e .

Trying to run the tests for a first time, I got issues relative to numpyro and blackjax, which I installed with

pip install numpyro blackjax

Only then I could run the tests, which failed with this result, which is reproducible (I ran the whole test suite twice, with the same errors).

However, running the failing test files individually with pytest pymc/tests/test_distributions.py (for example) cannot reproduce the failures, and all of the tests seem to pass in this way.

Is there something that I am doing wrong?

2

It’s probably just a seeding issue, I have no good advice beyond that. I usually would never run that file all at once as it’s quite lengthy.

3

Thank you for your answer. I found the github action file, which helped me understand how PyMC is tested. I will see if I can investigate the seeding issue when running all the test suite at once: it only takes 1/2 hour on my laptop and I like to make sure that I am not missing anything.

I also found out that test_printing.py is referenced nowhere in the github action file, and hence probably never tested. Is this intentional? Or would you like me to add it somewhere?

1 Like
4

Test files that are not mentioned explicitly are included automatically in the first job IIRC.

Could be wrong though, would need to check the logs of a recent job to confirm.

5

I just downloaded the logs of one of the last jobs and grepped for test_printing, which didn’t bring up anything relevant, so I think that it is not run. All of the other test files are tested at least on one of the runners.

6

There is a helper script in the scripts folder that checks if any test files are missing from the jobs or duplicated. What does it say about the printing file?

7

It says that it is indeed not tested by anything :slight_smile:

1 Like
8

I thought that would cause the workflow to fail… well looking forward to your PR then :wink:

CC @michaelosthege

1 Like