Bug 470381 - Julia backend does not allow symlinked executable
Summary: Julia backend does not allow symlinked executable
Status: RESOLVED FIXED
Alias: None
Product: cantor
Classification: Applications
Component: julia-backend (show other bugs)
Version: 23.04.1
Platform: NixOS Linux
: NOR normal
Target Milestone: ---
Assignee: Alexander Semke
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-05-28 23:27 UTC by hqurve
Modified: 2023-09-16 13:09 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 23.12


Attachments
Screenshot showing that the julia backend works with symlink (96.95 KB, image/png)
2023-08-01 23:50 UTC, hqurve
Details

Note You need to log in before you can comment on or make changes to this bug.
Description hqurve 2023-05-28 23:27:28 UTC
SUMMARY

If the julia executable on your path is a symlink, the julia backend cannot be chosen. 

STEPS TO REPRODUCE
1. Have the julia executable on your path be a symlink (eg install through nix)
2. Launch cantor
3. Attempt to open with julia backend

OBSERVED RESULT

The following message is displayed:

```
Some requirements are not fulfilled: The path to Julia specified in the application settings must point directly to the executable. Symlinks are not allowed. Please provide the correct path in the application settings and try again.
```


EXPECTED RESULT
Julia backend can be selected

SOFTWARE/OS VERSIONS
Linux/KDE Plasma: nixos unstable
(available in About System)
KDE Plasma Version: 5.27.5
KDE Frameworks Version: 5.106.0
Qt Version: 5.15.9

ADDITIONAL INFORMATION

By searching the code, symlinks are only prevented in the julia backend. The logic exists in this file https://invent.kde.orghttps://invent.kde.org/education/cantor/-/blob/d5121b245973f1e70949828d6a98e4bf0cfd6ff9/src/backends/julia/juliabackend.cpp#L84 and (I think) the logic was first introduced in https://invent.kde.org/education/cantor/-/commit/1e144559c034ff70b23b3d6b5c1b7b28dbf8af18

By building cantor with the logic removed, the julia backend works. However, I am unsure if this is suitable since I do not know why symlinks are prevented. 

Also, I am asking since I wish to also patch out this logic in the nixpkgs package for cantor in order for the cantor julia backend to work. https://github.com/NixOS/nixpkgs/pull/234479/files#diff-b1d95328883e905223e5671936ca77425773b2f4a2f7293311a5cb1a37c76e95
Comment 1 Alexander Semke 2023-08-01 10:49:46 UTC
(In reply to hqurve from comment #0)
> By building cantor with the logic removed, the julia backend works. However,
> I am unsure if this is suitable since I do not know why symlinks are
> prevented. 
This was implemented to make sure Julia's shared libraries are found at runtime. Does Julia backend really work for you if you remove that if check like in your path? Can you try to do a simple evaluation, something like 5+5 is enough, with your patch? It doesn't work for me...
Comment 2 hqurve 2023-08-01 23:50:14 UTC
Created attachment 160673 [details]
Screenshot showing that the julia backend works with symlink

I attached a screenshot showing that the julia backend is really working and that the executable is a symlink.

However, if I do the following, I get an error `Julia session can't login due internal julia problem with missing internal file - "/home/joshua/kde/src/lib/julia/sys.so"`

1. link julia to the current directory `ln -s $(which julia) ./julia`
2. add current directory to path `PATH=$(pwd):$PATH`
3. run cantor and select julia backend
4. try 1+1

From https://invent.kde.org/education/cantor/-/blob/670993d7bf38afcacd2914509d621761dd8e4e60/src/backends/julia/juliaserver/juliaserver.cpp#L27, the error occurs since the .so file cannot be found and the fallback path is `../lib/julia/sys.so`. 

So, I guess that the issue with symlinks depends on whether you also correctly symlink the lib/julia/sys.so file. However, the official installation instructions (https://julialang.org/downloads/platform/#linux_and_freebsd) for julia only recommends linking the julia binary and has no suggestions for also linking the shared library.

Perhaps the restriction for symlinks could be removed and canonicalPath() could be used instead of dir().absolutePath() on line 32. (documentation for canonicalPath https://doc.qt.io/qt-5/qfileinfo.html#canonicalPath). This replacement allows the julia symlink in my current directory (which is on my path) to work.
Comment 3 Alexander Semke 2023-08-02 18:14:49 UTC
(In reply to hqurve from comment #2)
> Created attachment 160673 [details]
> Screenshot showing that the julia backend works with symlink
> 
> I attached a screenshot showing that the julia backend is really working and
> that the executable is a symlink.
> 
> However, if I do the following, I get an error `Julia session can't login
> due internal julia problem with missing internal file -
> "/home/joshua/kde/src/lib/julia/sys.so"`
> 
> 1. link julia to the current directory `ln -s $(which julia) ./julia`
> 2. add current directory to path `PATH=$(pwd):$PATH`
> 3. run cantor and select julia backend
> 4. try 1+1
> 
> From
> https://invent.kde.org/education/cantor/-/blob/
> 670993d7bf38afcacd2914509d621761dd8e4e60/src/backends/julia/juliaserver/
> juliaserver.cpp#L27, the error occurs since the .so file cannot be found and
> the fallback path is `../lib/julia/sys.so`. 
> 
> So, I guess that the issue with symlinks depends on whether you also
> correctly symlink the lib/julia/sys.so file. However, the official
> installation instructions
> (https://julialang.org/downloads/platform/#linux_and_freebsd) for julia only
> recommends linking the julia binary and has no suggestions for also linking
> the shared library.
> 
> Perhaps the restriction for symlinks could be removed and canonicalPath()
> could be used instead of dir().absolutePath() on line 32. (documentation for
> canonicalPath https://doc.qt.io/qt-5/qfileinfo.html#canonicalPath). This
> replacement allows the julia symlink in my current directory (which is on my
> path) to work.
Thank you for your valuable input here. I really appreciate it!

Switching to QFileInfo::canonicalPath() only doesn't seem to work for me. I'm looking at the logic implemented in JuliaServer::login()
https://bugs.kde.org/show_bug.cgi?id=425695 and don't understand the complexity introduced here (I'm not the original author of this code)...

Everything works for me fine, also with symlinks, if I replaced that whole logic with the simple  jl_init(). With this we don't even need to ask the user to provide the path to the executable. Cantor's binaries cantor_juliabackend.so and cantor_juliaserver are linked against libjulia.so that was used when compiling&linking Cantor and everything else won't work anyway. 

For python we don't ask to provide the path to the executable. Yes, we can work with one single version of python only that was used during the linking step but this is also true for Julia at the moment - if you select a wrong julia executable it won't work. In future we should be able to support multiple runtimes (different versions of python, julia, etc.) but this is not possible at the moment since we more or less fix the version during the link time and this needs to be implemented differently in future.

So, why to ask the user to provide the path to Julia's executable at all and all that complexity with jl_init_with_image()?
Comment 4 Alexander Semke 2023-09-16 13:09:42 UTC
Git commit 5c038e422c8fe2fcc14ddca2f8ec8d8c6fbca592 by Alexander Semke.
Committed on 16/09/2023 at 15:09.
Pushed by asemke into branch 'master'.

[Julia] removed the dependency on Julia's executable and set the recommended
version to the current LTS version 1.6.7.

We don't communicate via CLI with julia's executable and load the interpreter
into Cantor's runtime in the "server" directly and similarly to how it's
done also for Python and R. cantor_juliabackend.so and and cantor_juliaserverjulia
are linked to julia's libraries. We nowhere depend on the executable.
Allowing the user to specify the path to the executable and loading the libraries
from there made some edge cases possible but introduced additional complexity
in the code and such severe limitation as not being able to work with symlinks
in the PATH. So, we remove this complexity and limitations now and the more important
option to work with different packages and version of Python, R and Julia will be
solved in future differently.
FIXED-IN: 23.12

M  +3    -0    CHANGELOG.md
M  +8    -62   src/backends/julia/juliabackend.cpp
M  +0    -6    src/backends/julia/juliabackend.kcfg
M  +9    -29   src/backends/julia/juliaserver/juliaserver.cpp
M  +2    -3    src/backends/julia/juliaserver/juliaserver.h
M  +1    -5    src/backends/julia/juliasession.cpp
M  +1    -3    src/backends/julia/juliasettingswidget.cpp
M  +69   -85   src/backends/julia/settings.ui

https://invent.kde.org/education/cantor/-/commit/5c038e422c8fe2fcc14ddca2f8ec8d8c6fbca592