This is not kdesvn-build specific, but due to the Python plasma stuff in kdebase.

Ideally it should just install in the chosen installation prefix, and in case it is different than the Python one, give a message saying to setup PYTHONPATH properly.

*** Bug 183291 has been marked as a duplicate of this bug. ***

Severity=>major though for me it's even a blocker cause it does break my automated build-process and hence "Blocks development and/or testing work" :-/

Created attachment 32108 [details]
Patch to honour the PYTHON_SITE_PACKAGES_DIR path passed in the command line.

This patch for kdelibs honours the PYTHON_SITE_PACKAGES_DIR value passed in the command line with the -D parameter and only sets it to the value returned by distutils.sysconfig.get_python_lib(plat_specific=1) (which is usually /usr/lib/python2.N/site-packages) if a value is not specified in the command line.

It may be useful to do the same to PYTHON_INCLUDE_PATH too.

Additional comments about the previous patch to kdelibs:

If you're using kdesvn-build, adding 'cmake-options -DPYTHON_SITE_PACKAGES_DIR:PATH=/path/to/local/directory' to the 'module kdebase' directive should work. If you're compiling trunk manually, just pass the -D switch to cmake when compiling kdebase.

As I said, there may be some case in which PYTHON_INCLUDE_PATH should be modified too (it defaults to the global Python include path, something like /usr/include/python2.6), even though this has not happened to me yet.

I've got a patch ready to submit. I just need a moment to get it ready.

Created attachment 32117 [details]
This patches up kdelibs to provide the new CMake variables and improved python helper script.

I hope the comment I wrote is already attached to this. If not, see below.

Created attachment 32118 [details]
This updates kdebase to make use of the changes in kdelibs, so that python modules are handled better

PLEASE SEE MY ABOVE PATCH FOR kdelibs AS WELL. THEY BOTH APPLY TO revision 939041

I've got *2* patch that tries to make this whole enterprise as transparent as possible.

The crux of the problem is that KDE developers like to configure CMAKE_INSTALL_PREFIX so that it points to some directory under ${HOME}, while python has traditinally dictated that modules be installed to a system-wide site-packages directory. This causes a clash of permissions: `make install' can't install to this system-wide site-packages directory without special priveleges (for instance, `sudo make install').

For those with python>=2.6, the attached patches provides a complete fix. For those with python<2.6 installed, the attached patches provide help via cmake control.

Python>2.6 implements PEP 370, which provides for a per-user site-packages directory. Here's how the patches make use of this new functionality:

If the user has configured CMAKE_INSTALL_PREFIX such that it points to a directory that is not underneath ${HOME}, then cmake selects the system-wide site-packages directories for installing the python modules; this behavior is the same as it has always been.

If the user has configured CMAKE_INSTALL_PREFIX such that it points to a directory underneath the user's ${HOME}, then cmake assumes the user is a KDE developer and therefore intends to run `make install' without special priveleges, so cmake asks python for a path to the user's site-packages directory. If this succeeds, then everything is installed as expected, and the whole process is transparent.

However, if cmake finds that python doesn't know about user-specific site-packages directories, then cmake will message the user to explain the situation and to offer a partial solution: The user can force a directory to be used as the user's site-packages directory by specifying -DPYTHON_SITE_PACKAGES_INSTALL_DIR=$some_path on the command line.

The attached patches preserve PYTHON_SITE_PACKAGES_DIR and add 2 new variables: PYTHON_USER_SITE_PACKAGES_DIR to hold the user-specific site-packages directory of the user configuring KDE, and PYTHON_SITE_PACKAGES_INSTALL_DIR to multiplex between those 2 variables (it can always be overridden on the command line).

I have tested every scenario but 2: (1) systems without python and (2) systems with python<2.6. However, I'm fairly sure the code will work perfectly ;-)

Michael, note that this patch is now being discussed on the build-system mailinglist. You might want to comment there on questions or remarks from Alexander, our build system expert.

http://lists.kde.org/?t=123712361900003&r=1&w=2

Created attachment 32379 [details]
Patches for kdelibs, kdebase, and kdebindings.

Sorry for getting back WAY too late on this one. I've got excuses, but who cares... ;-)

I've attached a tarball that contains 3 pataches: kdelibs.patch, kdebase.patch, kdebindings.patch.

The main patch is kdelibs.patch, which cleans up some cruft and adds 3 new CMake commands: PARSE_ARGS(), ADD_PYTHON(), and INSTALL_PYTHON().

If I may boast, these are very cool and they do The Right Thing.

ADD_PYTHON() creates a target for compiling python modules (as well as whole packages) and INSTALL_PYTHON() allows for easily installing the results. Whereas before, there was just one always-out-of-date target to which all compilation commands (everywhere) were appended, it is now possible to create any number of named targets with outputs that only need to be rebuilt when necessary. More interesting is that the handling of packages and the preservation of relative paths might allow for a more pythonic organization of the python source code. In short, It's Awesome.

The patches to kdebase and kdebindings simply make use of the new commands where CURRENTLY appropriate, which---unfortunately---is not that many places: Both kdebase and kdelibs (and probably a lot of others) handle python modules improperly; python byte-code is specific to each version of the python interpreter. This is why python has interpreter-specific site-packages directories. Until many of the python modules are converted to modules that can be properly installed in versioned site-packages directories, you can't byte-compile the python and feel proud about it (and more importantly, you can't make use of the shiny new commands provided by kdelibs.patch ;-) As it is now, the misused compilation has been converted to direct installation of the .py files only.

Because of the large additions, the patches are kind of hard to read. I recommend applying them and then reading the affected files. The comments/documentation are pretty darn thorough. I would particularly point people to read about PARSE_ARGS(), which I think could be very helpful for further cmakery.

Also, as a very small footnote that can hopefully be disregarded, I didn't actually test my changes to kdebindings. I've compiled my qt-copy out of source, and my guess is that kdebindings makes a lot of egregious assumptions to the contrary (written in lurid tab-and-space-mixed perl, no less! Probably some emacs user). Needless to say, I couldn't get kdebindings to compile and install, but at least cmake ran and it's output looks suitable. Furthermore---and perhaps worse---I imagine that there are many more projects out there that will break, because I have thrown out the old PYTHON_INSTALL(). This is a good thing, for the path to the future is laid on the ruins of the past!

For me, after applying the patch, it tries to install in /PyKDE4 but can't of course.

The following patch has been applied to KDE trunk (rev. 965738): 
http://reviewboard.kde.org/r/553/diff/#index_header

So you should try setting -DPYTHON_SITE_PACKAGES_DIR=~/.local/lib/python2.6/site-packages

Worked fine here with python 2.6.0

So by default kde isn't installable into a custom prefix anymore, one needs to find this bug report in order to find out how to do that.... That's not really great for developers.

Could the cmake stuff detect that the location is not writable and use ~/.local in that case? [Not sure how this interacts with DESTDIR though]. Or at least telling the user what to do in such a case. Currently all a user sees is:
  file INSTALL cannot copy file
  "[...]"
  to "/usr/lib/python2.6/dist-packages/PyKDE4/plasmascript.py".

Which looks very buggy when the chosen prefix is not /usr....

There was a heated discussion on the kde-buildsystem mailing list some time ago about informing the user, or automatically choosing a local path (and how that only plays nice with Python 2.6+).

IIRC, Michael Witten's patch does something like that, but it needs some work so it hasn't been integrated.

Currently, I don't really know what to tell the user: if he's using Python <= 2.5 he needs to install somewhere and export PYTHONPATH to include his installation path (and then someone in the thread said exporting PYTHONPATH was not good, and someone else argued it was or something ;), if the user is using Python 2.6+ he should install to ~/.local/lib/python2.6/site-packages. The lowest common denominator here is just telling the user to pass -DPYTHON_SITE_PACKAGES_DIR to cmake, with which I agree, but I don't know if it's usual.

*** Bug 198927 has been marked as a duplicate of this bug. ***

Michael's patches look good overall indeed, it's a shame they're not in svn ;)
I do like the autodetection solution, and I think this should definitely go in.
(and if not, we should at least tell people which var to set in order to fix this, currently it's unclear to everyone but those who find comment 13 in this report)

Has this been posted to kde-buildsystem@kde.org so that Alex (well, let's cc him) and the other cmake guys can have a look?

In particular, ParseArgs.cmake looks very generic, maybe it should go to cmake itself?

The patches have been discussed in kde-buildsystem (there has been some discussion over k-c-d as well), Alex asked for some changes and then the discussion stalled ;)

The main thread can be seen here: http://lists.kde.org/?l=kde-buildsystem&m=123722881329874&w=2

I just stumbled over this bug yesterday and today over the bugreport. I hope this patch gets into svn, at least for 4.4

Michael: ping?

Hmm, re-reading http://lists.kde.org/?l=kde-buildsystem&m=123958336819732&w=2
I'm afraid that Thiago alienated Michael a bit :{

It looks like the small unrelated changes made "on the way" are the reason for this big fix not being committed, which is a bit sad.

IMHO the stuff should be committed as is and then we can talk about polishing  and/or detect breakages if the needs be, because that would be better than the current breakage going on forever and nothing happening for lack of an actual fix in svn. But I don't know anything about python so I certainly don't want to commit it myself, since I wouldn't really be able to make fixes if needed (nor fully explain what the patch does).

Michael: please commit your fix. You can blame me if someone says you should have posted a Nth version of the patch first.

I see benefits in reviews, but not when the result is discouragement and an important fix not going in at all.

I just wanted to state: Still not committed, and quite annoying for automated building :-(.

Michael? You're still on CC, are you around as well?

Currently, this problem also fails kdesvn-build if you don't run it with sudo (and risk killing your stable copy).

I'll try to get fix the kdesvn-build problem meaning at least in a sandbox it would be safe (which is related to pathing most likely), but my python knowledge is much more limited (does not really exist) than my perl knowledge, so i'd really appreciate this if it was fixed.

(In reply to comment #22)
> Currently, this problem also fails kdesvn-build if you don't run it with sudo
> (and risk killing your stable copy).
> 
> I'll try to get fix the kdesvn-build problem meaning at least in a sandbox it
> would be safe (which is related to pathing most likely), but my python
> knowledge is much more limited (does not really exist) than my perl knowledge,
> so i'd really appreciate this if it was fixed.

Erm, you just have to pass "cmake-options -DPYTHON_SITE_PACKAGES_DIR:PATH=/foo/bar" inside the "module kdebase" section inside kdesvn-build...

Created attachment 39030 [details]
add some comments about setting PYTHON_SITE_PACKAGES_DIR

meanwhile we're still deciding whether comitting the patch or not, would some documentation at least mitigate the problem?

Yes, a documentation would be good. I e.g. do not know where to set the path best. Why don't you set it to $KDEDIRS/lib/python2.6/site-packages per default?

Raising priority. Recent changes in kdebase mean that it *ALWAYS* tries to install the Plasma bindings now, regardless of the presence of PyQt4 or PyKDE4 ( which I don't have installed ). 

Requesting that the Plasma bindings be kicked out of kdebase to kdebindings, or the appropriate checks put in place.

Ugh. One shouldn't do these things late at night...

kicked out -> moved from.

So, I'm confused, *why* aren't we simply using CMAKE_INSTALL_PREFIX with a big warning to set PYTHONPATH? After all that's exactly how *everything else* works; if CMAKE_INSTALL_PREFIX is not / or /usr, you have to set PATH, LD_LIBRARY_PATH, QT_PLUGIN_PATH, etc. in order to use the installation. What's wrong with adding PYTHONPATH to that list? Why is Python being given special treatment?

Python stuff goes to $CMAKE_INSTALL_PREFIX for quite some time. The only trick is that one has to set SIP_DEFAULT_SIP_DIR.