Bug 370260 - No support for tweets with >140 characters
Summary: No support for tweets with >140 characters
Status: RESOLVED FIXED
Alias: None
Product: choqok
Classification: Unclassified
Component: Twitter (show other bugs)
Version: Git
Platform: Other Linux
: NOR major (vote)
Target Milestone: ---
Assignee: Mehrdad Momeny
URL:
Keywords:
: 386960 417841 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-10-07 20:08 UTC by Antonio Rojas
Modified: 2020-02-18 21:12 UTC (History)
11 users (show)

See Also:
Latest Commit:
Version Fixed In: 1.7


Attachments
Screenshot (56.99 KB, image/png)
2016-10-09 20:25 UTC, Antonio Rojas
Details
Possible patch (1.39 KB, patch)
2017-11-08 19:17 UTC, Simon St James
Details
Patch for v1.4 (3.63 KB, patch)
2017-11-09 21:06 UTC, Simon St James
Details
[PATCH] Don't overwrite original contents of retweets (1.47 KB, patch)
2018-04-22 19:01 UTC, Frédéric Brière
Details
[PATCH] Don't overwrite original contents of retweets [v2] (1.48 KB, patch)
2018-04-22 19:15 UTC, Frédéric Brière
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Antonio Rojas 2016-10-07 20:08:26 UTC
For a while the links, pictures etc don't count towards the 140 character limit in Twitter. Choqok doesn't seem to be aware of it and still ellides these tweets if the total length including links is over 140 characters.

Reproducible: Always
Comment 1 Andrea Scarpino 2016-10-08 15:47:51 UTC
Hi Antonio, I'm sorry but I don't get what you mean here.

However, note that links longer than `short_url_length` doesn't influence the maximum length of the tweet (see https://dev.twitter.com/overview/t.co).
Comment 2 Antonio Rojas 2016-10-09 20:25:45 UTC
Created attachment 101496 [details]
Screenshot

Links and mentions now don't count at all towards the 140 limit, see https://blog.twitter.com/2016/doing-more-with-140-characters

For instance, this tweet:
https://twitter.com/i/web/status/784121721301438465
is ellided in choqok, see attached screenshot
Comment 3 Mario Hommel 2016-10-16 10:39:47 UTC
I can confirm, this is about how the client deals with the new extended Tweets, see https://dev.twitter.com/overview/api/upcoming-changes-to-tweets for informations.
Comment 4 Mathias Homann 2017-11-08 10:50:18 UTC
Twitter now has officially increased the length allowed for tweets to 280 characters. Please upgrade this in choqok.
Comment 5 Christoph Feck 2017-11-08 15:57:52 UTC
Raising severity based on comment #4.
Comment 6 Simon St James 2017-11-08 19:17:28 UTC
Created attachment 108752 [details]
Possible patch

I'm using KDE4 and Choqok 1.4 at the moment so I can't test this patch against master (I can't even test if it compiles!), but the corresponding patch against 1.4 seems to work OK based on my limited testing.

Can someone who is using Choqok master give it a try?
Comment 7 Simon St James 2017-11-09 21:06:51 UTC
Created attachment 108768 [details]
Patch for v1.4

Looks like I was a little premature with that last patch :) I've attached my patch against v1.4 in the hopes that it will give some inspiration to people working on master.

Why it works: Choqok doesn't actually do the truncation itself; rather, by default, twitter will actually do it *on the server side*+ - this patch simply adds the "tweet_mode=extended" query, and reads the "full_text" of the returned JSON instead of the "text".

+ I spent about half an hour yesterday trying to figure out just *where* Choqok was performing the truncation - kicked myself when I figure out what was actually going on! XD
Comment 8 Mathias Homann 2017-11-09 22:55:26 UTC
(In reply to Simon St James from comment #7)
> Created attachment 108768 [details]
> Patch for v1.4
> 
> Looks like I was a little premature with that last patch :) I've attached my
> patch against v1.4 in the hopes that it will give some inspiration to people
> working on master.
> 
> Why it works: Choqok doesn't actually do the truncation itself; rather, by
> default, twitter will actually do it *on the server side*+ - this patch
> simply adds the "tweet_mode=extended" query, and reads the "full_text" of
> the returned JSON instead of the "text".
> 
> + I spent about half an hour yesterday trying to figure out just *where*
> Choqok was performing the truncation - kicked myself when I figure out what
> was actually going on! XD

Does this work for posting as well?
Comment 9 Christoph Feck 2017-11-16 00:31:00 UTC
*** Bug 386960 has been marked as a duplicate of this bug. ***
Comment 10 avlas 2017-11-16 06:44:42 UTC
> Does this work for posting as well?

Posting is just fine even without the patch, I tested it in Choqok 1.6. The only issue is the countdown that starts at 140.
Comment 11 avlas 2017-11-16 06:51:25 UTC
> Why it works: Choqok doesn't actually do the truncation itself; rather, by default, twitter will actually do it *on the server side*+ - this patch simply adds the "tweet_mode=extended" query, and reads the "full_text" of the returned JSON instead of the "text".

IMO, this patch will increase the functionality a lot. Even before the change to 280 ch, I frequently had to open the browser to read complete messages that were cut.

It is great that you realize how to do it properly.

Thanks!
Comment 12 Andrea Scarpino 2017-11-18 16:34:45 UTC
Git commit 0ecc05c1d261977eb3131c5d05cd1d414962c7f7 by Andrea Scarpino.
Committed on 18/11/2017 at 16:33.
Pushed by scarpino into branch 'master'.

[twitter] support for extended tweet
FIXED-IN: 1.7

M  +106  -0    microblogs/twitter/twittermicroblog.cpp
M  +4    -0    microblogs/twitter/twittermicroblog.h
M  +9    -0    microblogs/twitter/twittersearch.cpp

https://commits.kde.org/choqok/0ecc05c1d261977eb3131c5d05cd1d414962c7f7
Comment 13 Andrea Scarpino 2017-11-18 16:35:46 UTC
(In reply to Simon St James from comment #7)
> Created attachment 108768 [details]
> Patch for v1.4
> Why it works: Choqok doesn't actually do the truncation itself; rather, by
> default, twitter will actually do it *on the server side*+ - this patch
> simply adds the "tweet_mode=extended" query, and reads the "full_text" of
> the returned JSON instead of the "text".

Thank you Simon!

I rebased your work and pushed to master. Mainly because twitterapi is used by Twitter and GNU Social code, while this is only a Twitter thing.
Comment 14 avlas 2017-11-18 16:45:14 UTC
> I rebased your work and pushed to master.

Hi Andrea,

Could you please release version 1.7 so final users can benefit from this enhancement?

Many thanks in advance!
Comment 15 avlas 2017-11-18 17:25:28 UTC
I compiled choqok with the fix and I still see something weird going on:

Look for example at this message:

https://twitter.com/i/web/status/931891495212404738

The text reads: Network Theory: It's used by scientists around the world studying the human genome, the efficiencies of green power grids, the economics of world trade and a hundred other applications.

That is 186 characters in total (and an image)

This message gets cut in choqok to: Network Theory: It's used by scientists around the world studying the human genome, the efficiencies of green power… https://t.co/KFowfJeOuN

Can this be fixed somehow?
Comment 16 avlas 2017-11-18 17:28:58 UTC
(In reply to avlas from comment #15)
> I compiled choqok with the fix and I still see something weird going on:
> 
> Look for example at this message:
> 
> https://twitter.com/i/web/status/931891495212404738
> 
> The text reads: Network Theory: It's used by scientists around the world
> studying the human genome, the efficiencies of green power grids, the
> economics of world trade and a hundred other applications.
> 
> That is 186 characters in total (and an image)
> 
> This message gets cut in choqok to: Network Theory: It's used by scientists
> around the world studying the human genome, the efficiencies of green power…
> https://t.co/KFowfJeOuN
> 
> Can this be fixed somehow?

And actually the image does not appear embedded in choqok, I would say this is because the message got cut, but not sure...
Comment 17 Mathias Homann 2017-11-19 08:44:17 UTC
I built a rpm from git master, and when I tried to send a tweet to twitter I got an error message saying "Could not authorize you"...
Comment 18 Mathias Homann 2017-11-19 08:48:15 UTC
The actual error message is:
"Creating the new post failed, with error: Could not authenticate you."
Comment 19 Mathias Homann 2017-11-19 08:59:17 UTC
...restarting my kde session dit it.
Comment 20 avlas 2017-11-20 20:19:26 UTC
I think the issue below actually relates to retweets (I didn't get the original tweet in choqok but a retweet of it). So I'm going to open a new bug specifically for retweets.

(In reply to avlas from comment #15)
> I compiled choqok with the fix and I still see something weird going on:
> 
> Look for example at this message:
> 
> https://twitter.com/i/web/status/931891495212404738
> 
> The text reads: Network Theory: It's used by scientists around the world
> studying the human genome, the efficiencies of green power grids, the
> economics of world trade and a hundred other applications.
> 
> That is 186 characters in total (and an image)
> 
> This message gets cut in choqok to: Network Theory: It's used by scientists
> around the world studying the human genome, the efficiencies of green power…
> https://t.co/KFowfJeOuN
> 
> Can this be fixed somehow?
Comment 21 Mathias Homann 2017-11-20 21:21:31 UTC
(In reply to Mathias Homann from comment #19)
> ...restarting my kde session dit it.

actually, I have the same error on and off.
Comment 22 avlas 2017-11-21 17:16:56 UTC
(In reply to Mathias Homann from comment #21)
> (In reply to Mathias Homann from comment #19)
> > ...restarting my kde session dit it.
> 
> actually, I have the same error on and off.

Suffering the same thing. I cannot send any tweet from choqok :/
Comment 23 avlas 2017-11-21 17:30:53 UTC
(Re-)validating the twitter account does not work in master version of choqok (nothing happens, that is the browser does not open any window/tab in which twitter informs about the oauth number to introduce back in choqok, neither choqok is waiting for anything).

All this works just fine downgrading to choqok version 1.6.
Comment 24 avlas 2017-11-21 17:33:08 UTC
(In reply to avlas from comment #23)
> (Re-)validating the twitter account does not work in master version of
> choqok (nothing happens, that is the browser does not open any window/tab in
> which twitter informs about the oauth number to introduce back in choqok,
> neither choqok is waiting for anything).
> 
> All this works just fine downgrading to choqok version 1.6.

I assumed, but not sure, that the issue came from last changes related to this issue. If you believe this is an entirely different thing, I will open a new bug...
Comment 25 avlas 2017-11-21 18:21:38 UTC
(In reply to avlas from comment #24)

> I assumed, but not sure, that the issue came from last changes related to
> this issue. If you believe this is an entirely different thing, I will open
> a new bug...

It seems there are two different issues here:

- the problem of not being able to revalidate an account (didn't try to validate an account from scratch) started in the transition from qoauth to qtnetworkauth (commit c43125e48387a8d3a914c97cd8d8765c5c07fa89)

- the problem of not being able to tweet even if the account is validated appeared in commits related to this bug, after commit 573e5a7411307644bb3f1882e0bb2a9aedd849f9
Comment 26 Antonio Rojas 2018-02-04 15:26:41 UTC
This patch doesn't fix the problem for me, it's still reproducible with git master. Moreover, this causes all retweets to have the unnecessary text "RT @account" preprended.
Comment 27 Christoph Feck 2018-02-22 22:18:15 UTC
Andrea, could you have a look at the recent comments?
Comment 28 Christoph Feck 2018-03-15 01:40:28 UTC
No response, changing status.
Comment 29 Andrea Scarpino 2018-03-19 08:40:25 UTC
(In reply to Antonio Rojas from comment #26)
> This patch doesn't fix the problem for me, it's still reproducible with git
> master. Moreover, this causes all retweets to have the unnecessary text "RT
> @account" preprended.

I'm sorry for the late reply.

I confirm your words. However, we don't prepend that text, so I suspect this is the original Twitter behaviour and I broke the Choqok code that was removing that text maybe? I'll investigate.
Comment 30 Frédéric Brière 2018-04-22 19:00:28 UTC
(In reply to Andrea Scarpino from comment #29)
> However, we don't prepend that text, so I suspect this
> is the original Twitter behaviour and I broke the Choqok code that was
> removing that text maybe?

Close.  Your patch overwrites the (already decoded) contents of the original Tweet with that of the retweet, which contains the "RT @user" prefix (and is apparently still limited to 140 characters).

I'll attach a simple fix.
Comment 31 Frédéric Brière 2018-04-22 19:01:50 UTC
Created attachment 112170 [details]
[PATCH] Don't overwrite original contents of retweets
Comment 32 Frédéric Brière 2018-04-22 19:15:20 UTC
Created attachment 112171 [details]
[PATCH] Don't overwrite original contents of retweets [v2]

Sorry, the previous patch was taken from a modified 1.6 branch, and while it applies cleanly to master, the offset line numbers are enough to throw git off.  Here's a revised version.
Comment 33 Andrea Scarpino 2018-04-24 11:21:58 UTC
(In reply to Frédéric Brière from comment #32)
> Sorry, the previous patch was taken from a modified 1.6 branch, and while it
> applies cleanly to master, the offset line numbers are enough to throw git
> off.  Here's a revised version.

Great, but please use phabricator instead.
Comment 34 Christoph Feck 2018-05-16 16:21:34 UTC
Frédéric, any success with https://phabricator.kde.org/differential/diff/create/ ?
Comment 35 Patrice Grundmann 2019-02-23 12:43:51 UTC
Hello,

When will come out Choqok version 1.7 with the bug fix?

Thanks for the job.
Comment 36 avlas 2019-02-23 14:28:22 UTC
(In reply to Patrice Grundmann from comment #35)
> Hello,
> 
> When will come out Choqok version 1.7 with the bug fix?
> 
> Thanks for the job.

I don't think there is any proper fix yet. Version 1.6 works better for me than master. As far as I can see, choqok is mostly unmaintained at this point.
Comment 37 avlas 2019-02-23 14:44:00 UTC
(In reply to avlas from comment #25)
> (In reply to avlas from comment #24)
> 
> > I assumed, but not sure, that the issue came from last changes related to
> > this issue. If you believe this is an entirely different thing, I will open
> > a new bug...
> 
> It seems there are two different issues here:
> 
> - the problem of not being able to revalidate an account (didn't try to
> validate an account from scratch) started in the transition from qoauth to
> qtnetworkauth (commit c43125e48387a8d3a914c97cd8d8765c5c07fa89)
> 
> - the problem of not being able to tweet even if the account is validated
> appeared in commits related to this bug, after commit
> 573e5a7411307644bb3f1882e0bb2a9aedd849f9

I tested master again, and to be fair, the above issues seem to have been indeed fixed.

My apologies for the previous comment. Is there anything retaining this issue to be set to Fixed Status?
Comment 38 avlas 2019-02-23 18:36:46 UTC
One issue I can see in master version, I recall it now I saw it before too, is that there are no links for cut messages, which makes choqok sort of useless as most of the displayed messages are cut (I assume at 140 characters) and there is no easy way to see the rest, not even via a link that you can open in the browser by a click.

Coming back to version 1.6 :(
Comment 39 Antonio Rojas 2019-02-23 19:32:58 UTC
(In reply to avlas from comment #38)
> One issue I can see in master version, I recall it now I saw it before too,
> is that there are no links for cut messages, which makes choqok sort of
> useless as most of the displayed messages are cut (I assume at 140
> characters) and there is no easy way to see the rest, not even via a link
> that you can open in the browser by a click.
> 
> Coming back to version 1.6 :(

I can only reproduce this with cited tweets. And in any case, you can always open it by clicking on the timestamp.

Even if this is no longer being developed, a release in its current state would be a great improvement.
Comment 40 avlas 2019-02-23 20:33:21 UTC
(In reply to Antonio Rojas from comment #39)
> I can only reproduce this with cited tweets. And in any case, you can always
> open it by clicking on the timestamp.

That's right
Comment 41 Eliseu Cendron Carvalho 2019-10-03 00:29:33 UTC
I just did some tests here and I can confirm Choqok 1.6 actually CAN post 280-char tweets - just pay attention to the character counter until it hits -140. It will not only post the tweet - when you check out your profile on Twitter website, it shows the entire text, not the shortened one as shown in Choqok's main window.
Based on that, I imagine it IS possible to actually upgrade the character counter to 280 somehow.
Comment 42 Patrice Grundmann 2020-01-11 00:54:30 UTC
(In reply to Eliseu Cendron Carvalho from comment #41)
> I just did some tests here and I can confirm Choqok 1.6 actually CAN post
> 280-char tweets - just pay attention to the character counter until it hits
> -140. It will not only post the tweet - when you check out your profile on
> Twitter website, it shows the entire text, not the shortened one as shown in
> Choqok's main window.
> Based on that, I imagine it IS possible to actually upgrade the character
> counter to 280 somehow.

Oh yes, it works when creating a tweet, but the feature most expected by everyone is to be able to read messages of more than 140 characters without clicking on the link to open the browser and read more.
Comment 43 Antonio Rojas 2020-02-05 12:06:20 UTC
Unfortunately the 1.7 release still has the RT issue, which is fixed by Frédéric patch in Comment 32 - can this please be merged?
Comment 44 Andrea Scarpino 2020-02-05 23:49:07 UTC
Git commit e5e60a1692fe7bed0e3537bef63747ef100d643f by Andrea Scarpino, on behalf of Frédéric Brière.
Committed on 05/02/2020 at 23:48.
Pushed by scarpino into branch 'master'.

twitter: Don't overwrite original contents of retweets

Retweets are apparently still subject to a 140-character limit, and may
be truncated if the original Tweet was an extended Tweet, or if the
additional "RT @user" prefix pushes them beyond the limit.

If `repeatedPostId` has been set by setRepeatedOfInfo(), then `content`
already contains the full text of the original Tweet, and we don't need
to do anything.

(This doesn't apply to TwitterSearch::readStatusesFromJsonMap(), which
does not call setRepeatedOfInfo() in the first place.)

M  +1    -1    microblogs/twitter/twittermicroblog.cpp

https://invent.kde.org/kde/choqok/commit/e5e60a1692fe7bed0e3537bef63747ef100d643f
Comment 45 Antonio Rojas 2020-02-06 07:24:50 UTC
Thank you!
Comment 46 Antonio Rojas 2020-02-18 21:12:43 UTC
*** Bug 417841 has been marked as a duplicate of this bug. ***