We cannot fix Klipper because there doesn't seem to be anything broken in it. NEdit seems to have trouble with sending clipboard contents.

Why am I not surprised about this kind of response?

I get a lot of benefit from nedit, not surprisingly as it's my primary editor, but I have yet to see one benefit from klipper. Klipper has a long history of causing trouble with other applications, which isn't surprising as it aggressively interferes. Since it does nothing useful and causes trouble, the most obvious course of action which presents itself is rm /opt/kde3/bin/klipper. That's until it's fixed. It is not the responsibility of multiplatform applications to take care of some obscure and not very useful KDE component.

Btw you'd be standing much better to acknowledge bugs and say you haven't got resources to fix them, than to deny them.

You get this kind of response because that's the way it is. Klipper doesn't seem to be broken and NEdit seems to be broken, simple as that. It is the responsibility of even multiplatform applications to fix their bugs. If you don't like Klipper because (unlike many others) you fail to see its value or value it less than the broken application, just don't use it.

As for your confused view on Klipper and its accused "bugs", http://www.kdedevelopers.org/node/1787 might be an interesting read if you get bored one day.

I'm one of the NEdit developers and would like to look into this problem a bit. First problem here is that I don't see the effect with my NEdit:
Enhanced NEdit 5.5
Nov 24, 2005

     Built on: Linux, 486, GNU C
     Built at: Jan  2 2006, 20:46:10
   With Motif: 2.2.3 [@(#)Motif Version 2.2.3]
Running Motif: 2.2 [@(#)Motif Version 2.2.3]
       Server: The XFree86 Project, Inc 40399902
       Visual: 16-bit TrueColor (ID 0x22, Default)
       Locale: de_DE

Is there some special option I should activate in Klipper?

Second is a question to Lubos: Your statement sounds like you looked into NEdit's code a bit. Could you give me some pointer as to what's broken in NEdit?

Thanks!

I actually cannot now reproduce either, so I guess you'll have to sort this out with the reporter or whoever else runs into the problem. IIRC the problem was that NEdit sometimes failed to send clipboard (ok, PRIMARY actually) contents, which Klipper saw as empty clipboard and the "prevent empty clipboard" option caused Klipper to set the contents again to the last known contents. Which in turn unset the selection in NEdit.

Klipper in KDE3.5.2 got XFixes support which makes Klipper 100% reliably detect clipboard changes and might be a reason why it cannot be reproduced anymore. Even though I actually tried even with it explicitly turned off (adding #undef HAVE_XFIXES before the first test for it in kdebase/klipper/clipboardpoll.cpp and recompiling kdebase/klipper) and still couldn't reproduce *shrug*. Apparently a race condition somewhere.

As for checking NEdit's code, no, I didn't look at it, I checked only from Klipper's side and there it seemed correct while sometimes not getting the right reply from NEdit. If you manage to reproduce the problem, I suggest first checking that you check how NEdit replies to every single clipboard request and find out why it sometimes doesn't send back the contents (it might even be that I made a mistake and the request from Klipper is incorrect, but it looked fine to me when I checked ... I don't really remember much now).

If you don't get anybody to confirm the problem with at least KDE3.5.2 it might be probably simpler just to shrug it off though.

*** Bug 140180 has been marked as a duplicate of this bug. ***

I'm using KDE 3.5.2, and I experience this problem, easily and consistently.

Note that the problem appears with NEdit more reliability if it's linked with Lesstif than if it's linked with OpenMotif.  I would suggest looking for bad interaction between Klipper and Lesstif.

But I would strongly discourage blaming Lesstif completely.  From what I read, Lesstif works just fine with GNOME.  It is only when Klipper and Lesstif get together that we see this problem, suggesting that they're both to blame.

There's no need to go looking for bad interaction between Klipper and Lesstif, since it's there for sure and it's Lesstif's fault. It works just fine with GNOME because it has no Klipper-like functionality so there's nothing to trigger Lesstif bugs.

Has anyone from KDE been in contact with the Lesstif folks to see about fixing Lesstif?

While the bad behavior of Lesstif may not be your fault, it is nevertheless causing you trouble.  Perhaps some diplomatic relations with Lesstif developers will help everyone.

I always use nedit with openmotif because it gives me less trouble than lesstif. The klipper problem persists there too, although a workaround is to disable one specific klipper option.

The response "it's everyone else's fault but not klipper's" from KDE/klipper maintainer(s) is not very original. I would favour a more integrating approach.

It may not be very original, but the truth is usually rather dull (http://www.kdedevelopers.org/node/1787). If you have a good reason to make me believe the problem is in Klipper, I might reconsider, but the time I've already spent on this says otherwise. We have enough of our own bugs and don't have resources to work on Motif bugs, sorry.

I like to take up this bug report, since I think it is valid.
Moreover, the problem isn't likely to be cured with XFix, because it
actually is a time dependent issue, which is awful because it means
that some folks may easily reproduce this and others don't --
depending on their systems.

The problem is that at some point of time, probably only a millisecond
or so, klipper prevents other X clients to gain ownership of a
selection, and this isn't ok IMHO, especially so for a program running
in the background like a daemon.

To understand what happens, I must ask you to drop the idea that the
selection (primary or secondary) would just be a second clipboard.
The clipboard is the clipboard, and the selection is the selection,
that's why things are called like they are.  Please think of the
selection (the primary, let's forget about the secondary for this)
just as a region in console edtiors like (x)emacs.

What happens is as follows: You select a region in nedit to perform an
editing action on this.  When you perform this operation (doesn't
matter which one, let it be shifting or filling or replace all foo
with bar in the selection), the text in the selection/region is to be
replaced with a new text.  When this happens, the selection is lost,
and nedit creates a new selection adjusted to the new text, so that
you can go on with what you do, eg, shifting the new region again.

In a bit more detail, "the selection is lost" means that there is an
empty selection, and nedit disowns empty selections.  So, a rude way
to avoid interference with klipper would be to keep owning empty
selections, but then klipper would never see any changing selections
inside nedit, that is, you only ever would get the very first
selection into klipper, as long as you stay inside nedit -- I don't
know if this would be in klipper's interest, and checking gedit, it
also disowns empty selections.  Unfortunately, when klipper sees this
change in ownership, it feels disturbed by this unowned empty
selection, and if the option "prevent empty clipboard" is on, takes an
action that just for a millisecond prevents other X clients to gain
ownership of a selection.  If this coincides with the time nedit tries
to do a new selection (to adjust the region to the new text) it ends
up with no selection.  It seems that the action klipper takes is
setting the middle mouse button to paste a previous selection, but in
this case with nedit, klipper shouldn't fall back but it should
actually fall forward to the next selection which would just be a
millisecond away.  However, it probably can't know so, because the
future is always hard to predict ;-)

Now, is it really necessary to prevent X clients to get ownership of a
new selection at any point of time in order to achieve what you like
to do?

Are you sure that klipper actually prevents anyone from taking ownership? It sound more like a race to me, with the undesired path goes like this:
1. nedit announces new, empty selection.
2. nedit sets a new selection
3. klipper restores the selection from before 1.

Obviously, the nedit author would prefer 2 and 3 to be swapped. However, due to the asynchronous nature of X, I'm not sure how this would be achieved.

One thing that strikes me as odd in the nedit behaviour is why nedit first disowns the clipboard and then immediately reacquires it. Can't nedit predict this and set the selection to the new value directly? If nothing else, we would save some X-messages, which is always nice if ssh -X is involved.

> Are you sure that klipper actually prevents anyone from taking
> ownership?

No, I wasn't sure - it was just one of two possibilities, but I can see
clearly now that klipper sets the content of the primary selection,
which at least shouldn't happen if there is already one.

> Obviously, the nedit author would prefer 2 and 3 to be swapped.

I tend to doubt that the nedit author knew anything about klipper, KDE
or QT when he started with nedit in 1992, but I suspect he had to learn
many things about X and Motif (including the undocumented things).

> One thing that strikes me as odd in the nedit behaviour is why nedit
> first disowns the clipboard and then immediately reacquires it.
> Can't nedit predict this

No, nedit on its own can't predict in general whether new selections
are created after old ones are lost.  Just for example, someone could
program a macro which changes selections - selections are transient in
nature and are not nedit's clipboard but rather its regions, so it's
rather common to have macros that alter selections just like editing
alters selections/regions.  Now, how to predict at which points the
user does new selections or not after previous ones are lost, ie,
should we keep empty selections or is it ok to disown them[1]?  If one
could teach innocent users about the crucial question of whether or not
to keep ownership of empty selections, and if one keeps track of all
situations, yes sure everything is possible, simply because one can
program anything what one likes - provided one has enough time and
passion, but this would get really odd.

The idea there would be anything odd in using selections in the way
nedit does comes just from the odd idea the selection would be the
clipboard ;-) although I must admit that lots of applications claiming
to be X don't seem to know of doing anything more with the primary
selection than letting it be pasted[2], which in turn could lead to the
selection = clipboard idea.

But we don't need to get into a discussion of what's odd.  As I
understand now, you'd like to get the selection to act more like the
clipboard, but I assume that you actually don't intend to set the
primary selection to some previous content while there is a valid
selection already on the screen.  However, as proven by example, this
happens. (So, by assumption a valid bug report IMO.)


Therefore, the adjusted question would be if klipper really needs to
set the primary selection to let the middle mouse button paste content
from its history, in case there seems to be no primary?


In any case I would appreciate if you would document in a footnote at
least that the option prevent empty clipboard actually sets the primary
selection to the last entry in the klipper history.  This is all
confusing, since the real clipboard atom can become empty, btw.  From
the klipper documentation I rather expected that the clipboard atom
wouldn't be allowed to be emtpy and/or the klipper history wouldn't get
empty strings (of course, there are never empty strings making it into
klipper's history).



[1] So it comes just down to either disowning or keeping empty selections 
    always.  In the latter case nedit would own the selection(s) all the 
    time.  Whether it is some X convention or policy, or if nedit just 
    disowns empty selections by politeness (and since there is no point in
    owning them) isn't clear at the moment.
    
[2] So, perhaps it is odd under X, but surely not under Windows, at
    least I don't know any Windows editor that wouldn't use selections
    as regions and I think for Mac (Aqua) this holds, too.
    
    
    

Well, for sure it would be nice if all cases were caught, but I am not sure it is possible. As I understands, what happens is probably this (note that "sets selection" really is "announces ownership", if I recall correctly).

1. nedit sets empty, disowned clipboard
2. X tells klipper that clipboard is empty
3a. nedit sets new selection (possible overriding what was in the selection)
3b. klipper sets old selection (possible overriding what was in the selection)

If you do not want klipper to change the selection on it's own, prevent empty clipboard is not for you, I think. I cannot see how klipper can detect that someone has set the selection between querying the content and setting it. 

The idea with "prevent empty clipboard" is mainly to avoid the primary+selection from being unset when an application quits, thus taking potential user data with it. Possibly, the prevent empty clipboard feature could be enabled and disabled separately for selection and primary.

With the information provided by the recent comments it seems it is a problem caused by Klipper after all. I don't remember anymore why I originally dismissed it as NEdit bug, maybe there was an unrelated problem.

As for avoiding the problem, I don't think it can be really fixed. The option enabled by default has a quite some value, probably even higher than this actual problem, especially given that the option can be deactivated by the user in this case.

Klipper also really cannot detect that the application disowned and again claimed the selection, I expect that NEdit disowns the selection with its X timestamp, reacquires it with the same X timestamp and eventually its output queue is flushed. Even if Klipper used the same X timestamp to avoid claiming the selection if another claimed with a newer timestamp occurs, it would still come after nedit's claim with the same timestamp and win.

XFixes has support for reporting the reason why a selection is no longer owned, so Klipper could not set the previous contents again if the selection has been disowned intentionally, but that'd depend on XFixes, this is not possible to detect otherwise.

Can you confirm that NEdit uses the same X timestamp for reacquiring the selection (i.e. no X timestamp updates, no delays)? In such case I think Klipper could wait a little bit with preventing the empty contents and check again if it's actually still empty. With some care it could be even without race conditions I think.

> I don't remember anymore why I originally dismissed it as NEdit bug,
> maybe there was an unrelated problem.

Yes, you confuse(d) everyone by calling the selection clipboard, and
there is a bug in lesstif leaving a locked clipboard behind after
accessing it (nedit calls motif functions to handle the clipboard).
Please notice, I don't say cat to your dog because dogs and cats are all
the same for me.


> Can you confirm that NEdit uses the same X timestamp for reacquiring
> the selection (i.e. no X timestamp updates, no delays)?

No, this isn't what happens and I think it isn't the real problem.  The
thing is simply, if you don't want to set the primary selection while
there possibly is already a valid one on the screen, you simply can't set
the primary at all, because nobody can predict when the next selection
will be (after you detected an empty one).  Even if you would or could
lock the state (like for the clipboard), you would only achieve that
nobody else could get ownership of a selection in this time (just the
second possibility for failure)

(NEdit simply updates its buffer.  If it sees an empty selection there it
owns, it calls the XtDisown to disown it - sort of making clean.  It has
no idea at this point whehter or not a new selection will follow - it
simply can't know [in general], since no one really can.  Notice, if one
would have very quick fingers, one might trigger the same effect in gedit
by changing selections by hand.  Ok, I doubt it, but the difference is
only that with nedit you can automate editing tasks which is much
faster.)

I only see two possibilities:

1. Write a documentation about the "prevent empty clipboard" (this is
   already a misnomer), so that one can understand what goes on and that
   this could possibly lead (under rare conditions I admit) to the
   problem we discuss.

or, perhaps (not sure if it could work)

2. Instead of setting the primary, perhaps KDE could handle mouse events
   before anyone else.  So, on a middle click event one would check if
   there is a valid selection.  If so, let the event go through (so that
   X pastes the primary).  If not and X could paste nothing, let klipper
   handle it by pasting from its clipboard history.

I should add that the problem can of course only arise if klipper acts on its own, ie, w/o user interaction.  If the user clicks on the klipper symbol to pop up klipper and then clicks on an entry of its history, then there is no problem to set the primary to this content. 

The problem is if a user sits in front of some X client doing something there and all of a sudden the primary is taken away by someone running in the background.  So, at the very least the user should be able to clearly understand what the "prevent empty clipboard" option actually does or sometimes rather tries to do.

(I presume that Qt automatically prevents zero selections by falling back on the previously selected thing, so this klipper option would really be effective only for non-Qt applications?)

I'm pretty sure that after having spent so much time with Klipper and QClipboard I know the difference between the PRIMARY and CLIPBOARD selections, clipboard is simply a lazy way to name both/either of them. Sorry if I confused anybody, although from the comments it seems I confused only you.

>> Can you confirm that NEdit uses the same X timestamp for reacquiring 
>> the selection (i.e. no X timestamp updates, no delays)? 
> 
> No, this isn't what happens

 So what happens then, NEdit updates the X timestamp meanwhile or can have delays when disowning and reacquiring selection as a result of one user action?

> and I think it isn't the real problem.  The 
> thing is simply, if you don't want to set the primary selection while 
> there possibly is already a valid one on the screen, you simply can't set 
> the primary at all, because nobody can predict when the next selection 
> will be (after you detected an empty one).

 I believe X timestamps are there to exactly avoid such problems.

Ok, I'd just like to repeat (because perhaps it didn't get through) that it only ever matters what is in the clipboard (to adopt your redefinitions of language) when the user gives the command "paste it".  So, to avoid an empty clipboard (since pasting nothing wouldn't make sense), it isn't necessary to reset the clipboard every time it gets empty - this also doesn't work in general, btw.  Therefore my proposal would be to prevent the clipboard from being empty only when there is a middle-click event.  (This is also safe, because at this time the user just likes to paste the clipboard, and isn't in the middle of doing anything else.)

I'm pretty sure that wouldn't work. Klipper would not be asked to provide the contents of the clipboard unless Klipper had claimed the selection for itself. Unless XFixes provide a way to intercept such clipboard requests, I don't see how this would be achieved.

And could you stop this bitching about clipboard, selection and primary? Everyone in here are well aware of the differences, whatever you might think.

I agree with Lubos here.  A slight delay, say 10ms in the acquisition of the primary, might mean that a program would have a time to reacquire it if it drops the selection temporarily.  But a human wouldn't be able to do anything in that short of a time frame to notice.

However, I see no reason for nedit to drop selection ownership temporarily.  Most example toolkit code I see drops the selection ownership when the widget is destroyed.  

Perhaps both fixes would help users.

I agree that this needs fixing to help users, absolutely.  However, I
think the two simple example NEdit macros

deselect_all()
select(0, 10)

and

deselect_all()
for(i = 0; i < 10000; i++)
{}
select(0, 10)

show that neither a mere time shift nor a synchronization attempt
could help.  Moreover, the problem isn't related to NEdit, that only
uses the primary selection as the general purpose thing it is.(*)

Anyway, after some investigations, I must correct myself a bit.
For a fix I propose now to change the default of the "Prevent empty
clipboard" option to off.  Moreover, the option should really(*) be
renamed to "Prevent empty selections" and along the same lines the
documentation of this option should read

  Prevent empty selections:
  If this option is turned on and Klipper detects an empty selection
  it will take over the selection and set it to the most recent item
  of its clipboard history. (Notice that by the current implementation
  it can't be excluded that Klipper might overwrite an existing
  selection.)


IMHO something like this is required, especially since Klipper runs in
the background and takes the selection over without any connection to
user actions if that option is turned on. (Only adding to this is that
Klipper provides no visible feedback of what happens.)


(*)
Please recall that the atoms PRIMARY, SECONDARY, and CLIPBOARD are
defined in the ICCCM.  I can't help wondering how the PRIMARY could be
called (or seen as) clipboard, since in fact it misses all the necessary
conventions to behave clipboard-like.  Actually, these conventions are
all there in the ICCCM, but coming just a little bit after subsection
2.6.1.1 (The PRIMARY selection) in the somewhat lengthier subsection
2.6.1.3 (The CLIPBOARD selection).

If you think I'm misinterpreting, you can verify the preceding paragraph
in section 4 of the paper "The X Selection Mechanism" written by Keith
Packard trying to uncover Phil Karlton's (one of the X fathers) intent.  

http://keithp.com/~keithp/talks/

To give some indication, here are only two quotes:

"PRIMARY is sort of the default selection; ...  This is the selection
which is supposed to be represented graphically on the display; for
example a text editor would assert this selection whenever the user
selected a range of text and would highlight the region ..."

Does Klipper highlight the selection - well, it's just a clipboard, so
why to highlight it?  And indeed:

"The CLIPBOARD selection is designed to provide a cut/copy/paste
system by allowing a client to hold a selection which needn't be
displayed on the screen, and which should last longer than the
lifetime of a user activity."

Just recall that there is no convention that the PRIMARY should last
longer than this - or what should the option "Prevent empty clipboard"
be needed for?

Nice discussion; Jörg, good job at explaining the problem!

Some comments:
1. I think the basic problem is indeed a mixup of PRIMARY and Clipboard (!= CLIPBOARD):
- Fact: These two are very often confused. The bitching might be seen as inappropriate, but it's perfectly understandable: Seeing sloppy use of language like before does not make me think "Gee, these are smart guys, they certainly know that they are redefining the language a bit; let's assume they know what they talk about.", but, sadly, "Ah, this would be the 7,334,454th case of someone confusing the two." So, I am sure that you know the matter, but don't be surprised that someone assumes different.
- Fact: PRIMARY is for interactive use. If you munge it from the background it's your job that you don't step on someone's toes.

2. The root of the problem is that KDE, instead of educating its users, let them keep windowsy assumptions about Clipboard, which are not compatible with proper use of PRIMARY and CLIPBOARD.

3. Why is this still UNCONFIRMED?

I really fail to see what Klipper can do about this. Deselecting the clipboard and immediately reclaiming it seems pretty broken to me, and Klipper do what it advertises in this situation.

Unless Mr. Lunak disagrees, I think we should INVALID this particular report.

So whenever this is a lesstiff, motif or klipper problem: this bug is 3.5 years old, still open and it disturbs nedit users.

Is there a way to fix this issue or is there a workaround? For me, even with deactivated "prevent empty clipboard", nedit sometimes is loosing his selection and i can't copy&paste text.

AFAICR the CVS version of NEdit doesn't disown the selection any more, so simply close this bug. (I wouldn't consider it invalid as such, but it is probably more related to the Qt handling of the clipboard, which doesn't seem to be compliant to X conventions. The problem didn't occur with Xfce's Clipman, which relies on the GTK handling of the clipboard.)

@Andreas: There is only one PRIMARY at a time for X, so if you have a selection in NEdit (which is a PRIMARY), switching to another X client and creating a PRIMARY selection there loses the existing selection. (The selection highlighted in kate for instance is not necessarily a PRIMARY selection.)

Closing as INVALID as requested, with INVALID=fixed in Nedit cvs. The arch bug (race conditions possible with selection (well, primary too for that matter)) cannot be resolved without resorting to ugly hacks as far as I can see.

Ok, i'd like to reopen this bug: yesterday i compiled and installed nedit HEAD on my ubuntu 9.10 (running kde 4.3.2) and this bug still exists. After roughly half a day i can't copy&paste anymore.

There is still nothing in this bugreport that points to anything that could be fixed in Klipper (besides flipping the "prevent empty clibpoard" default, which we won't do simply because the current way is a better default on the whole), so I don't see any point in reopening this unless you can say what actually should be fixed in KDE.

Ok, then i should probably open another bug for nedit.

*** Bug 285084 has been marked as a duplicate of this bug. ***