Fix more links in the online versions of manuals

[wilfwilson]

For online HTML versions of manuals, the release-gap-package script already seds any relative references to the GAP reference manual to instead be hyperlinks pointing at the appropriate absolute location at https://www.gap-system.org/Manuals/doc/ref/ instead. That's great.

For references to other packages, it doesn't currently work so well.

For example (I assume that this package was released by ReleaseTools, but I could be wrong, sorry if so), the online version of HAPcryst here refers to polymaking, but the relative link

../../../pkg/polymaking-0.8.2/doc/chap3_mj.html#X7C07B16B873BA46D

resolves to https://gap-packages.github.io/pkg/polymaking-0.8.2/doc/chap3_mj.html#X7C07B16B873BA46D, which gives a 404 error. The unwanted pkg/ and the -0.8.2 are the problems.

Similarly, in cvec, the reference to the orb package here does not resolve, since it points to
https://gap-packages.github.io/pkg/orb-4.8.2/doc/chap4_mj.html#X7ACED4FB7C971A5A.
For it to work, the pkg/orb-4.8.2/ should instead just be orb/.

It would be great to have all such links not give a 404.

For packages hosted at gap-packages and referring to another gap-packages packages with a gh-pages branch deployed, this is hopefully not too hard to fix (finding a way to remove the pkg/ and removing any -0.0.0 version numbers from the names of packages).

But for packages that are not hosted at gap-packages (e.g. Digraphs), or for any package referring to a non gap-packages package, this approach won't work, since we'll also need to work out the host. Perhaps this information could be automatically extracted from wherever https://gap-packages.github.io gets its data?

[fingolfin]

First off, those relative links are formatted with https://www.gap-system.org/ in mind -- thus the cvec link to orb in https://www.gap-system.org/Manuals/pkg/cvec-2.7.4/doc/chap5.html#X875B076C8559CAF0 works perfectly.

In the end, I am afraid this is an unsolvable problem in the current setup, because we simply can't now where (to say with the example) polymaking will be on the server: it might be in a directory named polymaking/ but it could also be in polymaking-0.8.2/ or in the future even polymaking/0.8.2/ or even PoLyMaKiNg (the latter is of course made up but several packages do use such names for their package directories).

One thing we could do is to try and harmonize the URLs between https://gap-packages.github.io/pkg and https://www.gap-system.org/Manuals/pkg/ in the following sense: we could change the scripts which generate that to normalize directory names to the package name in lowercase. (Actually, my "dream" would be to use pkgname/VERSION/, and have pkgname/ redirect to the latest version, and keep old versions around; and for the finite set of existing pkgname-VERSION dirs, just turn them into redirects to the new scheme). Then, we can use URLs as you suggest.

Except that of course, as you already pointed out, this still won't work on https://gap-packages.github.io/pkgname/... if there is a link to a package hosted elsewhere (such as Semigroups, Digraphs, ...). We could perhaps work around that by adding fake package subdirs (i.e., add subdirs to the https://github.com/gap-packages/gap-packages.github.io repository with the names of those packages), and have those redirect to the "real" URLs (except that in order to be able to do proper redirects, we couldn't host this on GitHub Pages anymore, we'd need to host it on a "full" server -- but that could be arranged).

But then those external packages would still have the same problem if they try to link to other packages.

Taking a step back, the only "real" fix then seems to be to use absolute URLs everywhere (just like we do for the GAP manuals); but unfortunately, to the best of my knowledge, there is no proper way to get this information! We'd have to add a "CanonicalDocumentationURL" field to PackageInfo for that, I guess (probably on a per-book basis); and then ideally GAPDoc would know about that. (That said... if I build package documentation locally, I really prefer local links...). I guess if CanonicalDocumentationURL is missing, a suitable "guess" could be made (e.g. if the PackageWWWHome has https://gap-packages.github.io as prefix, point there; otherwise point to gap-system.org).

Taking another step backwards, perhaps we should also reconsider the whole model of how we distribute documentation: Perhaps we should not include the generated manual in package anymore at all, and instead rely on PackageManager / the user to build it. Now that GAPDoc allows building the text & HTML version of the manual even if the user has no LaTeX installed, that might be actually viable (well, assuming issue #74 can be resolved cleanly, which I am pretty hopeful about, though). Some thoughts

• It would simplify ReleaseTools greatly. We could add a way for a package to signal that it doesn't want the manual to be built. (Although it probably would be a good idea to still build it once anyway, just to test that building the manual works -- just not do it again the second time when creating the tarball).
• for the upload to the package's gh-pages branch, we can use one of the techniques discussed above
• for the GAP website, we already rebuild all manuals anyway, I think, and then rereate tarballs, which I always disliked... Anyway: since the final use is known, when building the manuals for this usecase, suitable relative (or absolute) links can be made
• it would also essentially codify what some Linux distro packagers (e.g. for Debian) always have been doing (I think)
• on the user's machine, links would be completely local, which can be useful, too (e.g. works offline).

The main drawback of course is that building the manual still can go wrong, and also people wouldn't automatically get the PDF (but IMHO, not so many people will want that, and those who do want it can still download it)

[wilfwilson]

Thanks for all of these thoughts @fingolfin and for the information. It's interesting and I support a lot of what you say, I'll think about it some more in the future.

[fingolfin]

I think we "resolved" this for now by renaming package directories in the package distro and on the website to not have the versions again.

[fingolfin]

However this only resolves if the user making the release is making the release with the latest GAP release installed. Which is not always the case. The relase-pkg action helps with that (as it always installs a specific GAP version).

However, it doesn't help if a package A has refs and package B and it, and when it was released, its documentation was built against, say, GAP 4.14.0 and package B 1.0. Then a a year later, GAP 4.15.0 and B 2.0 are released, and the referenced items have moved. Then the links in A's manual will be broken, with the only possible fix in the current system being to make a fresh release of A, even if there are no code changes :-(.

(well, or the GAP server admins can manually set up redirects... but for that one has to be aware of the issue in the first place)

Ah well. Luckily this kind of moving things around is very rare, so all in all I think there are worse problems with easier solutions that we should look into first :-).