Documentation format

Development
1

I was putting together what I hope is a pretty simple PR to add the MultiTrace documentation to the
API documentation (see below for related question). One thing confuses me, as a relative Python novice: what’s the right format for docstrings? The format I see in the code for MultiTrace looks right to me, and it looks good when it is displayed in a console. But when I look at the documentation it produces in HTML, it doesn’t look good at all. For example, the formatting of parameters and their type information looks bad (the : is dropped).
As I said, I’m a python novice, so what’s the right way to deal with this? I have tried looking at the ReStructured Text documentation, and IIUC, what they propose would look terrible when read in a console – it’s full of distracting markup.
Am I misunderstanding something here? Is there some happy medium that I don’t know about?

Also, a side question – it seems like the MultiTrace docs could go either in the “inference” chapter or in the backends chapter. TBH, although by code it belongs in the latter, to understand inference, I believe it belongs in the former. Suggestions? Requests? Demands?

2

I was just asking for advice on #python and was pointed at this sphinx extension: https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html

It claims to take the kind of docstrings used in PyMC3 and massage them for better processing by RST.

Would it be appropriate to experiment with this extension?

3

@colcarroll?

4

@lucianopaz @colcarroll – I have looked and the documentation on line looks better than what I see if I build it locally. I suspect that this means that there’s something about my configuration and the canonical one that fails to correctly invoke this extension (or that invokes it with the wrong settings).
This could be something about my local machine, but I think it’s probably some way in which the sphinx configuration in the repository is “leaky,” and doesn’t force my machine to do The Right Thing.

5

Interesting! Do you get any warnings when you build? pip install -r requirements-dev.txt should get you set up to run make html from the docs/source directory.

Sphinx configuration is in docs/source/conf.py, and we used to use Napolean, but no longer do. To be honest, the sphinx stuff is still pretty arcane – there is some reckless metaprogramming that converts the example directory to a gallery, but you should not be hitting that.

If it would be helpful, I keep a static site for hosting docs, and can build and push pull requests to there – please ping me on the PR!

6

@colcarroll I’m afraid my problems building documentation persist.

I installed the dependencies in my pymc3 virtual environment using the requirements-dev.txt file. But when the gallery generator tries to run I get this error:

Exception occurred:
  File "/Users/rpg/src/pymc3/docs/source/sphinxext/gallery_generator.py", line 79, in __init__
    with open(filename, "r") as fid:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/rpg/src/pymc3/notebooks/GP-slice-sampling.ipynb'

I believe that the make script should be looking for that notebook file in the directory /Users/rpg/src/pymc3/docs/source/notebooks/GP-slice-sampling.ipynb instead. But I don’t know how it’s searching.

One possibility is that it is searching relative to PYTHONPATH, because the failing pathname is what you get from ${PYTHONPATH}/notebooks/GP-slice-sampling.ipynb, but that is just a guess.

However, if I unset PYTHONPATH, then the make script cannot find pymc3 itself, leading to this error:

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/Users/rpg/.virtualenvs/pymc3/lib/python3.7/site-packages/sphinx/config.py", line 361, in eval_config_file
    execfile_(filename, namespace)
  File "/Users/rpg/.virtualenvs/pymc3/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 86, in execfile_
    exec(code, _globals)
  File "/Users/rpg/src/pymc3/docs/source/conf.py", line 19, in <module>
    import pymc3
ModuleNotFoundError: No module named 'pymc3'

I’m not sure what is the proper way to have the document make script find the pymc3 sources. Any help would be much appreciated.

P.S. I noticed that the SMC sampling method is not included in the API docs; that’s what I am working on now.
P.P.S. As an aside, yes, things have somehow gotten worse since I tried to configure the virtual environment properly for development.

7

That’s funny – it is eventually getting that from https://github.com/pymc-devs/pymc3/blob/master/docs/source/sphinxext/gallery_generator.py#L208, which calls into the Sphinx machinery for app.builder.srcdir.

Are you getting this from running make html in pymc3/docs/source?

8

Yes, this is what happens when I run make html in pymc3/docs/source. I get different issues when I try the top-level build script (including warnings about trying to write to github.io).

I’m looking at that source line in gallery_generator.py, and it seems like it’s getting a relative pathname only in the NotebookGenerator constructor.

I looked further in gallery_generator.py and I believe I have found the culprit at line 141, where it defines source_dir by splicing the srcdir with os.path.dirname.

I still don’t have the full answer, because if I unset PYTHONPATH then make html gives me the No module named pymc3 error. So somehow I need to get sphinx to find pymc3 without setting PYTHONPATH.

On a happier note, it seems like there are a lot of good documentation strings that only need inclusion directions to make the API section much more extensive.

9

Oooh, from that same spot you’re making the docs, can you run

python -c 'import pymc3; print(pymc3)'

If you installed editable (pip install -e ~/path/to/pymc3), you’ll get the path to your local copy: I get <module 'pymc3' from '/Users/colin/projects/pymc3/pymc3/__init__.py'>. If you didn’t, you’ll get something funny.

You can either install editable, or build docs with

PYTHONPATH=~/path/to/pymc3 make html
10

I had just been using PYTHONPATH before, so I installed editable, and this is what I see when checking for pymc3:

$ python -c 'import pymc3; print(pymc3)'
<module 'pymc3' from '/Users/rpg/src/pymc3/pymc3/__init__.py'>

But when I do make html, it fails as before:

 $ make html
sphinx-build -b html -d _build/doctrees   . _build/html
Running Sphinx v2.0.1
/Users/rpg/.virtualenvs/pymc3/lib/python3.7/site-packages/sphinx/util/docutils.py:311: RemovedInSphinx30Warning: function based directive support is now deprecated. Use class based directive instead.
  RemovedInSphinx30Warning)
WARNING: The config value `html_theme_path' has type `str', defaults to `list'.
/Users/rpg/.virtualenvs/pymc3/lib/python3.7/site-packages/sphinx/util/compat.py:33: RemovedInSphinx30Warning: The config variable "source_parsers" is deprecated. Please use app.add_source_parser() API instead.
  RemovedInSphinx30Warning)
/Users/rpg/.virtualenvs/pymc3/lib/python3.7/site-packages/sphinx/util/compat.py:37: RemovedInSphinx30Warning: app.add_source_parser() does not support suffix argument. Use app.add_source_suffix() instead.
  app.add_source_parser(suffix, parser)

Exception occurred:
  File "/Users/rpg/src/pymc3/docs/source/sphinxext/gallery_generator.py", line 79, in __init__
    with open(filename, "r") as fid:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/rpg/src/pymc3/notebooks/GP-slice-sampling.ipynb'
The full traceback has been saved in /var/folders/ys/9xhjsh8x2pq8j66_jy4r940c0000gn/T/sphinx-err-ycvc_rb0.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [html] Error 2