New structure for documentation

[TeresiaOlsson]

We have previously discussed to restructure the documentation. I have promised to take the task but it would be useful with input so I have a better idea of what new structure to build.

So far we have discussed:

• Move all documentation which requires several packages into it's own repository. This would be examples, tutorials etc. Currently it is all inside the pyaml repository.
• Keep only documentation for a specific package in it's own repository and provide a common entry point to it, for example from the https://python-accelerator-middle-layer.github.io/ webpage.

I have looked at some other projects to get ideas for how they structure their documentation and would like input on what you like/don't like:

• PreCICE: https://precice.org/
• Bluesky project: https://blueskyproject.io/

In PreCICE they have one common webpage for all documentation. They have the menu options Quickstart, Docs and Tutorials.
In the Bluesky project they have separated the documentation for different packages. They have the options Documentation where you can click on the package you are interested in and Tutorial for ecosystem wide (I guess...) tutorials.

Currently I'm leaning towards doing it the PreCICE way. My experience of the Bluesky documentation is that you go to a specific package, find a package specific tutorial in there and after a while you are confused about how the ecosystem fits together because there is no joint documentation. The idea would then be to still have package specific documentation in each repository but the main purpose of it is to link all the pieces together into one joint documentation.

[TeresiaOlsson]

I found this documentation for how PreCICE manage their documentation: https://precice.org/docs-meta-overview. It looks like they have the documentation as part of their main webpage but then import content from other repositories using git submodules.

[TeresiaOlsson]

What I don't like about that approach is that they have the ecosystem level documentation directly inside the repository for the webpage. That seems a bit messy to me because you want to rebuild the website when there is some community news and rebuild the docs when there is a new code version. And also probably not use the same template. The reponsibilites for it is also different. The steering committee is responsible for the webpage and the maintainers for the docs.

Maybe better to make a separate repository for the ecosystem docs which will publish it's own page? That page can then be linked on the organisation webpage.

[TeresiaOlsson]

For the content I think it was from @simoneliuzzo I learned about diataxis. That feels like a good approach to follow. We already tried to do it for the last workshop.

[TeresiaOlsson]

@gubaidulinvadim @GamelinAl @kparasch @JeanLucPons Do you have some opinions on this? If not, I will go ahead based on what I think and make a first implementation for people to look at.

[kparasch]

I like diataxis. A separate single repository (without submodules) for the documentation I think is the most convenient option for the "officially supported" parts of pyaml.

[JeanLucPons]

I agree diataxis is good.
It would be nice to keep in pyaml-core only the API reference.
And to move example / tutorials / getting_started to an other repo.
IMHO, it is more urgent to freeze pyaml-core (mainly CS and pyaml devices abstraction) in order to avoid big refurbishment each time otherwise the progression will be slower and slower.

[TeresiaOlsson]

I agree diataxis is good. It would be nice to keep in pyaml-core only the API reference. And to move example / tutorials / getting_started to an other repo. IMHO, it is more urgent to freeze pyaml-core (mainly CS and pyaml devices abstraction) in order to avoid big refurbishment each time otherwise the progression will be slower and slower.

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

[JeanLucPons]

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

The CS part of the configuration layer is dynamic. As i already said we cannot consider all use cases of all control systems and try to build a definitive schema for all. The important thing here is to fix the CS interface and keep it as simple as possible. I would say that all the configuration part under the control: field belongs to various CS backends.

Concerning the pyaml level:

• A BPM configuration simplification (for offset) is required. We have to discuss if we want to keep a model (to handle electrode to position conversion for instance). Today it is not possible to implement BBA on internal simulator.
• Configuration of additional corrector that affect a same multipole (2 power supply for 1 strength on 1 element) , i already proposed few solutions but here at esrf we are in favor of having a correction strength file (It is not my choice) we would prefer an additional field in the lattice file. We have to take a decision.
• Configuration for multi combined function magnet (n power supplies , m strengths, k elements) A very tricky point for pole face strip.

[TeresiaOlsson]

According to the roadmap the hardware abstraction layer doesn't have to be frozen until the end of the year (which I think includes the CS and the pyaml devices abstraction?) but the configuration layer should be done before the summer vacations and part of that is user documentation for how to write the configuration. My thinking was to refactor the documentation now so we can add the configuration part, tell the steering committee that milestone is done and then focus on the hardware abstraction layer.

The CS part of the configuration layer is dynamic. As i already said we cannot consider all use cases of all control systems and try to build a definitive schema for all. The important thing here is to fix the CS interface and keep it as simple as possible. I would say that all the configuration part under the control: field belongs to various CS backends.

Concerning the pyaml level:

* A BPM configuration simplification (for offset) is required. We have to discuss if we want to keep a model (to handle electrode to position conversion for instance). Today it is not possible to implement BBA on internal simulator.

* Configuration of additional corrector that affect a same multipole (2 power supply for 1 strength on 1 element) , i already proposed few solutions but here at esrf we are in favor of having a correction strength file (It is not my choice) we would prefer an additional field in the lattice file. We have to take a decision.

* Configuration for multi combined function magnet (n power supplies , m strengths, k elements) A very tricky point for pole face strip.

I agree. For me those things are part of the hardware abstraction layer and not the configuration layer. For me the configuration layer is limited to the architecture for it, how the user would use the different parts and make it sufficiently flexible to read in schemas from external packages so objects from external packages can be validated and built within the same framework as the standard pyAML implementation. I think those parts can be frozen and the configuration layer considered done while still working on settling the things on your list. Maybe @gubaidulinvadim would disagree since he needs to rewrite the configuration file all the time but at least for me the exact fields in the configuration file isn't part of the configuration layer since they depend on how the CS, devices, models etc are implemented and which attributes they need.

[TeresiaOlsson]

Then I will create a new repository called pyaml-documentation, copy the existing docs folder from pyaml there and make the necessary changes so only the API reference is built in pyaml, tango-pyaml and pyaml-cs-oa and linked into the main documentation.