I'd like to use python3 (and its packages, etc) in cantor within my conda environment, but that doesn't seem to be possible. It is somehow hardcoded to use system's python version: /usr/lib/python3
The only way to achieve this at the moment is to recompile Cantor with python headers and libs from the conda environment. So, cantor has to be linked against this environment...
(In reply to Alexander Semke from comment #1) > The only way to achieve this at the moment is to recompile Cantor with > python headers and libs from the conda environment. So, cantor has to be > linked against this environment... It would be great if this could be easily modified from settings (as it can be done for other backends). I thought that just replacing which python is used could be enough, but I guess I was too optimistic. This is a workaround I could use as I'm actually compiling Cantor myself because of bug https://bugs.kde.org/show_bug.cgi?id=406712 Could you please let me know how to build linking to Conda instead? I tried compiling Cantor from scratch inside Conda's environment but this is creating lots of cmake errors because of things that are not python related.
(In reply to avlas from comment #2) > (In reply to Alexander Semke from comment #1) > > The only way to achieve this at the moment is to recompile Cantor with > > python headers and libs from the conda environment. So, cantor has to be > > linked against this environment... > > It would be great if this could be easily modified from settings (as it can > be done for other backends). I thought that just replacing which python is > used could be enough, but I guess I was too optimistic. For other backends like Maxima for example we communicate with the executable via stdin/stdout. For this the user have to or can specify the path for the executable. For scripting languages like python and julia we embedd the interpreter into our process at runtime. Which interpreter to use is determined during the compilation and linking to the python libraries. > This is a workaround I could use as I'm actually compiling Cantor myself > because of bug https://bugs.kde.org/show_bug.cgi?id=406712 I'll check this bug report. > Could you please let me know how to build linking to Conda instead? I tried > compiling Cantor from scratch inside Conda's environment but this is > creating lots of cmake errors because of things that are not python related. I'm not familiar with Conda but this "environment" also has to store the files somewhere. I think what you're looking for can achieved if you manage cmake to find the python installation from this environment. Please have a look at cmake/FindPythonLibs3.cmake. The variables PYTHONLIBS3_LIBRARIES and PYTHONLIBS3_INCLUDE_DIRS are used at compile and linking time and determine the python environment to use. This cmake script is looking for Python.h and for python3.2m. You can provide hints here by setting the environment variables PYTHONLIBS3_INCLUDEDIR and PYTHONLIBS3_LIBDIR in your shell. Just point these variables to your conda installation and it should work.
(In reply to Alexander Semke from comment #3) > You can provide hints here by setting the environment > variables PYTHONLIBS3_INCLUDEDIR and PYTHONLIBS3_LIBDIR in your shell. Just > point these variables to your conda installation and it should work. I tried but it doesn't seem to work
Yes, environment variables seems don't work, so try something like this: cmake -DCMAKE_BUILD_TYPE=release -DPYTHONLIBS3_LIBRARY=/mnt/Red/progs_for_cantor/Python-3.7.2/libpython3.7m.a -DPYTHONLIBS3_INCLUDE_DIR=/mnt/Red/progs_for_cantor/Python-3.7.2/Include .. But with your paths for Anaconda.
With the static library ".a" I get a compilation error. With the dynamic library ".so", cmake warns about Qt5 libraries may be hidden in miniconda lib. An example: "runtime library [libQt5Widgets.so.5] in /usr/lib/x86_64-linux-gnu may be hidden by files in: /home/miniconda/envs/cenv1/lib" But I can build cantor with no error. However, the backend does not seem to work. I get the following error: The name org.kde.Cantor.Python3-0 was not provided by any .service files
Created attachment 119924 [details] Cantor builded with Anaconda3
You actually need .so library, not .a library. Also, after build with nonsystem python you need specify path to this library, that Cantor don't try link with system python. So, you need run cantor something like this (there are LD_LIBRARY_PATH special environment variable for cases like this in Linux): LD_LIBRARY_PATH=/home/mmmm1998/anaconda3/lib/ cantor -b python3 I had run some tests on my machine, all looks working, except matplotlib (failed to import, some problems with _posixsubprocess, don't know why).
About hidden library, I solved this in ugly, but effective, way: moving all libQt5 shared libraries ("mv /home/mmmm1998/anaconda3/lib/libQt5* /home/mmmm1998/anaconda3", for example) and cmake directory ("mv /home/mmmm1998/anaconda3/lib/cmake /home/mmmm1998/anaconda3/") before building (and return all files back after build).
Will give a try, thank you
(In reply to Nikita Sirgienko from comment #9) > About hidden library, I solved this in ugly, but effective, way: moving all > libQt5 shared libraries ("mv /home/mmmm1998/anaconda3/lib/libQt5* > /home/mmmm1998/anaconda3", for example) and cmake directory ("mv > /home/mmmm1998/anaconda3/lib/cmake /home/mmmm1998/anaconda3/") before > building (and return all files back after build). Weird. Even after doing this, I get the error: "cantor: /home/miniconda/envs/cenv1/lib/libQt5Core.so.5: version `Qt_5.12' not found (required by /usr/x86_64-linux-gnu/qt5/libcantorlibs.so.21)"
Some clarification: after moving qt libs from anaconda3/lib directory, you need clean up your build directory and rerun cmake.
(In reply to Nikita Sirgienko from comment #12) > Some clarification: after moving qt libs from anaconda3/lib directory, you > need clean up your build directory and rerun cmake. I did. I reran cmake and make, and then get those libraries back. The error appeared anyway. However, if I keep the libraries away, cantor and its python3 backend in the conda environment work. Unfortunately, permanently removing those libraries is not an option for having conda working properly...
It's actually strange: for using the libraries in cantor, cmake need found them first. Could you cleanup build directory with 'rm -rf *' (or something like that) before cmake run? Is it solve the problem? And did you install new cantor build? I mean, cantor could work only installed, so you need somethink like 'make && make install && cantor -b python3'.
(In reply to Nikita Sirgienko from comment #14) > It's actually strange: for using the libraries in cantor, cmake need found > them first. Could you cleanup build directory with 'rm -rf *' (or something > like that) before cmake run? Is it solve the problem? > And did you install new cantor build? I mean, cantor could work only > installed, so you need somethink like 'make && make install && cantor -b > python3'. That's exactly what I did. I think it's because "LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/" sets the path for the Qt libraries as well, and them being different than Qt 5.12 trigger the error...
(In reply to avlas from comment #15) > (In reply to Nikita Sirgienko from comment #14) > > It's actually strange: for using the libraries in cantor, cmake need found > > them first. Could you cleanup build directory with 'rm -rf *' (or something > > like that) before cmake run? Is it solve the problem? > > And did you install new cantor build? I mean, cantor could work only > > installed, so you need somethink like 'make && make install && cantor -b > > python3'. > > That's exactly what I did. > > I think it's because "LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/" sets > the path for the Qt libraries as well, and them being different than Qt 5.12 > trigger the error... If you want me to be more explicit, I did: 0. cd /home/miniconda/envs/cenv1/lib/ 1. mv ../libQt5* . 2. cd to cantor build directory 3. rm -rf * 4. cmakekde5 -DPYTHONLIBS3_LIBRARY=/home/miniconda/envs/cenv1/lib/libpython3.7m.so -DPYTHONLIBS3_INCLUDE_DIR=/home/miniconda/envs/cenv1/include/python3.7m 5. make -j9 6. sudo make install 7. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/ cantor -b python3 If libQt5* libs are moved back to /home/miniconda/envs/cenv1/lib/ before executing point 7 above, the error is triggered
Sorry, point 1 above should read "mv libQt5* ../" (I copied "mv ../libQt5* .", which is how I put them back, by mistake)
I don't understand correctly, you make a mistake only in the message, or also when build cantor?
(In reply to Nikita Sirgienko from comment #18) > I don't understand correctly, you make a mistake only in the message, or > also when build cantor? Only in what I pasted here. The procedure was correct. In summary, this recipe only works if Qt libraries are not present in conda's library. Once they are set back in their folder, cantor crashes because it does not like Qt libs < 5.12
Okey, I got it, maybe moving libraries was not good idea. We actually could ignore cmake warning them, we use LD_LIBRARY_PATH to point python library (and also to all libraries in this directory). So, we could not do steps 0 and 1, but in this case, we loose any system settings (like icons theme, fonts) as I see.
Created attachment 119967 [details] Cantor with Anaconda3 Qt libraries
(In reply to Nikita Sirgienko from comment #20) > Okey, I got it, maybe moving libraries was not good idea. > We actually could ignore cmake warning them, we use LD_LIBRARY_PATH to point > python library (and also to all libraries in this directory). > So, we could not do steps 0 and 1, but in this case, we loose any system > settings (like icons theme, fonts) as I see. Unfortunately there is no difference, same thing happens even if one omits steps 0 and 1.
Could you try this variant, with proper paths? cmake -DCMAKE_PREFIX_PATH=/home/mmmm1998/anaconda3 -DPYTHONLIBS3_LIBRARY=/home/mmmm1998/anaconda3/lib/libpython3.7m.so -DPYTHONLIBS3_INCLUDE_DIR=/home/mmmm1998/anaconda3/include/python3.7m ..
(In reply to Nikita Sirgienko from comment #23) > Could you try this variant, with proper paths? > > cmake -DCMAKE_PREFIX_PATH=/home/mmmm1998/anaconda3 > -DPYTHONLIBS3_LIBRARY=/home/mmmm1998/anaconda3/lib/libpython3.7m.so > -DPYTHONLIBS3_INCLUDE_DIR=/home/mmmm1998/anaconda3/include/python3.7m .. there are lots of cmake warnings, and then there are make errors complaining about Qt 5.12 undefined references: /usr/lib/x86_64-linux-gnu/libQt5DBus.so.5.12.0: undefined reference to `qt_version_tag@Qt_5.12' /usr/lib/x86_64-linux-gnu/libQt5DBus.so.5.12.0: undefined reference to `QString::arg(QLatin1String, int, QChar) const@Qt_5' collect2: error: ld returned 1 exit status . . . /usr/include/KF5/KTextEditor/ktexteditor/editor.h:255:46: error: ‘QStringView’ does not name a type; did you mean ‘QStringList’? I recognized that this is trickier than I anticipated. I guess I don't fully understand what makes:/
Sorry last sentence didn't make much sense. I meant, I guess I don't fully understand what makes python so special in comparison to other cantor backends, but it is how it is :/
I think it would work for me if there would be a way to explicitly indicate the location of the Qt5 library, similarly to what we do with python3 libs
I freaked out of it, if we want to working with conda qt libraries, we also need build QtDbus 5.12 and build all cantor KF5 dependency for this qt version.(In reply to avlas from comment #26) > I think it would work for me if there would be a way to explicitly indicate > the location of the Qt5 library, similarly to what we do with python3 libs I don't think so: as I see, if we want use conda qt library, we need also build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. Well, that about this? 0. cd /home/miniconda/envs/cenv1/lib/ 1. mkdir cantor-python-dir 2. cd cantor-python-dir 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 libpython3.7m.so.1.0 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor -b python3
(In reply to Nikita Sirgienko from comment #27) > > I think it would work for me if there would be a way to explicitly indicate > > the location of the Qt5 library, similarly to what we do with python3 libs > > I don't think so: as I see, if we want use conda qt library, we need also > build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. I meant systems Qt5 library, because the error comes from cantor using conda's library instead. But it may not be possible, don't know... > Well, that about this? > 0. cd /home/miniconda/envs/cenv1/lib/ > 1. mkdir cantor-python-dir > 2. cd cantor-python-dir > 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 > libpython3.7m.so.1.0 > 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor > -b python3 I'll check it out, but I think it won't fix the issue with the Qt5 libraries (conda's Qt5 libs are < 5.12)
(In reply to avlas from comment #28) > (In reply to Nikita Sirgienko from comment #27) > > > I think it would work for me if there would be a way to explicitly indicate > > > the location of the Qt5 library, similarly to what we do with python3 libs > > > > I don't think so: as I see, if we want use conda qt library, we need also > > build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. > > I meant systems Qt5 library, because the error comes from cantor using > conda's library instead. But it may not be possible, don't know... > > > > Well, that about this? > > 0. cd /home/miniconda/envs/cenv1/lib/ > > 1. mkdir cantor-python-dir > > 2. cd cantor-python-dir > > 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 > > libpython3.7m.so.1.0 > > 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor > > -b python3 > > I'll check it out, but I think it won't fix the issue with the Qt5 libraries > (conda's Qt5 libs are < 5.12) Oh, I see the logic now, will check
(In reply to avlas from comment #28) > (In reply to Nikita Sirgienko from comment #27) > > > I think it would work for me if there would be a way to explicitly indicate > > > the location of the Qt5 library, similarly to what we do with python3 libs > > > > I don't think so: as I see, if we want use conda qt library, we need also > > build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. > > I meant systems Qt5 library, because the error comes from cantor using > conda's library instead. But it may not be possible, don't know... > > > > Well, that about this? > > 0. cd /home/miniconda/envs/cenv1/lib/ > > 1. mkdir cantor-python-dir > > 2. cd cantor-python-dir > > 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 > > libpython3.7m.so.1.0 > > 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor > > -b python3 > > I'll check it out, but I think it won't fix the issue with the Qt5 libraries > (conda's Qt5 libs are < 5.12) In `cantor-python-dir` directory you need have Cantor builded with system qt libraries (cmake without CMAKE_PREFIX_PATH). This variant actually fix the problem, that when we use LD_LIBRARY_PATH, we add path to python library, but also path to conda qt libraries, which conflicts with system qt libraries.
(In reply to Nikita Sirgienko from comment #30) > (In reply to avlas from comment #28) > > (In reply to Nikita Sirgienko from comment #27) > > > > I think it would work for me if there would be a way to explicitly indicate > > > > the location of the Qt5 library, similarly to what we do with python3 libs > > > > > > I don't think so: as I see, if we want use conda qt library, we need also > > > build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. > > > > I meant systems Qt5 library, because the error comes from cantor using > > conda's library instead. But it may not be possible, don't know... > > > > > > > Well, that about this? > > > 0. cd /home/miniconda/envs/cenv1/lib/ > > > 1. mkdir cantor-python-dir > > > 2. cd cantor-python-dir > > > 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 > > > libpython3.7m.so.1.0 > > > 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor > > > -b python3 > > > > I'll check it out, but I think it won't fix the issue with the Qt5 libraries > > (conda's Qt5 libs are < 5.12) > > In `cantor-python-dir` directory you need have Cantor builded with system qt > libraries (cmake without CMAKE_PREFIX_PATH). > This variant actually fix the problem, that when we use LD_LIBRARY_PATH, we > add path to python library, but also path to conda qt libraries, which > conflicts with system qt libraries. Only with libpython3.7m.so* does not work as it may be internally relying on other libs of the same folder, so I linked all libraries and then removed the link to libQt5* libs in that folder and worked :) By the way, that folder can be anywhere, so I created it within cantor folder instead of in conda's.
(In reply to Nikita Sirgienko from comment #27) > I freaked out of it, if we want to working with conda qt libraries, we also > need build QtDbus 5.12 and build all cantor KF5 dependency for this qt > version.(In reply to avlas from comment #26) > > I think it would work for me if there would be a way to explicitly indicate > > the location of the Qt5 library, similarly to what we do with python3 libs > > I don't think so: as I see, if we want use conda qt library, we need also > build QtDbus 5.12 and build all cantor KF5 dependency. It a lot of work. > > Well, that about this? > 0. cd /home/miniconda/envs/cenv1/lib/ > 1. mkdir cantor-python-dir > 2. cd cantor-python-dir > 3. ln -s /home/miniconda/envs/cenv1/lib/libpython3.7m.so.1.0 > libpython3.7m.so.1.0 > 4. LD_LIBRARY_PATH=/home/miniconda/envs/cenv1/lib/cantor-python-dir cantor > -b python3 By the way, thanks a lot for the workaround :)
Nice, that we solve this problem, should I close this bugreport?
(In reply to Nikita Sirgienko from comment #33) > Nice, that we solve this problem, should I close this bugreport? I would say so. [ Ideally, one would modify this in Cantor's settings, but this doesn't seem possible at all, so I guess there are no easier options than this solution ]