Bug 310512 - Plugin to expand URL Shortener urls
Summary: Plugin to expand URL Shortener urls
Status: RESOLVED FIXED
Alias: None
Product: telepathy
Classification: Frameworks and Libraries
Component: text-ui-message-filters (show other bugs)
Version: git-latest
Platform: unspecified Linux
: NOR wishlist
Target Milestone: Future
Assignee: Telepathy Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-22 16:24 UTC by Daniele E. Domenichelli
Modified: 2013-05-02 17:08 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 0.7.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniele E. Domenichelli 2012-11-22 16:24:39 UTC
URL Shortener can be used for evil, we should have a plugin to expand the url and show the real url instead.
Comment 1 Anant Kamath 2013-04-26 12:29:47 UTC
Could this service be considered : http://longurl.org/api ?
Comment 2 Daniele E. Domenichelli 2013-04-26 14:05:16 UTC
I think that's the way to go... we definitely can't support all existing shortening services.

The implementation should query for supported services on plugin creation, and then for every url check if it can be "unshortened"
Comment 3 David Edmundson 2013-04-26 14:08:43 UTC
an alternative approach is to go with something like Open Graph (http://ogp.me/)

It's a spec used in webpages to embed meta data (title + description + image). It's what Facebook use when you put a link in a status message. 

You can test the output you get from various websites here: http://www.google.com/webmasters/tools/richsnippets

This would make sexy previews for all pages.

Short URLs would redirect and therefore still work.
Comment 4 Anant Kamath 2013-04-26 20:05:46 UTC
AFAIK the number of sites that  include Open Graph meta data tags is limited (it's not a universal thing).

And to generate a preview we'd need to download the entire HTML source of the page at the URL and parse it in the filter itself to get those OGP tags if they're present. And if those tags aren't present then we can easily extract at most the page title. Extracting a meaningful 'description' or a human readable text excerpt and picking a thumbnail image might be more difficult .
 
How facebook and google plus apparently do it : http://www.raymondcamden.com/index.cfm/2011/7/26/How-are-Facebook-and-Google-creating-link-previews

A lot of different metadata 'protocols '(other than OGP) are in use in websites. 
So if not a we could probably look for a few certain popular tag types (eg. OGP, "<link rel=> tags, <meta description> ... etc) to generate a preview; if none of those tags are found, then a preview won't be generated (perhaps just the page title?) 

So, will it be acceptable for the filter to download the entire HTML source at the URL and create a HTML parser that will consider all combinations/types of tags that a website might use?
 
And short urls will still need to be lengthened before any of this takes place right?
Comment 5 Daniele E. Domenichelli 2013-04-26 20:27:41 UTC
I think (In reply to comment #3)
> an alternative approach is to go with something like Open Graph [...]
> This would make sexy previews for all pages. [...]
> Short URLs would redirect and therefore still work.

I think that's another plugin... web page previewer. I don't see a reason why we should have just the one you are suggesting though. I don't like web preview, so I usually disable them everywhere I can, but, at the same time I hate url shortener, and I would like to see the real address before clicking a link, and without connecting to another untrusted server...
On the other hand, web page preview is probably nice for a lot of people so it's worth to implement it as well...
Comment 6 Anant Kamath 2013-04-28 18:04:11 UTC
Can we arrive at a consensus? 

Like what Daniele said, how about this behaviour:
Two plugins: One for short url lengthening and the other for previews?
The first would just add the real url after a short url,and the other would display a preview
The preview plugin would have to incorporate the same url lengthening code too as the first plugin, to handle short urls.
If both plugins are enabled, the real long url (if it's a short url) is displayed, after which a preview is displayed.
Comment 7 David Edmundson 2013-04-28 18:05:42 UTC
Sounds good, was just throwing out an idea.
Are you planning on working on it? (URL expanding?)
Comment 8 Anant Kamath 2013-04-28 18:09:44 UTC
Sure. Will require JSONP, but the bugzilla plugin already uses JSONP.

The web preview might take some more time.
Is the code duplication (for URL expanding) between the url-expand plugin and the web preview plugin fine?
Comment 9 David Edmundson 2013-04-28 18:10:49 UTC
Yeah, ignore that web preview stuff for now.
Comment 10 Daniele E. Domenichelli 2013-04-29 07:37:45 UTC
@Anant Thanks for working on this :)

(In reply to comment #8)
> Is the code duplication (for URL expanding) between the url-expand plugin
> and the web preview plugin fine?

By setting the "weight" of the plugin, you can actually set which plugin runs first, so you can make the url-expanding run before the web-preview and therefore have the "expanded" url when the web-preview runs
I'm not sure what will happen if you disable the first, but I would ignore that for now. Just start with the url expanding ;)


Does longurl recursively expand urls? I mean, what if one uses one url shortener to short one url and then uses another one to shorten the shortened url? Does long url return the original one?
Comment 11 Anant Kamath 2013-04-29 11:26:17 UTC
>
> Does longurl recursively expand urls? I mean, what if one uses one url
> shortener to short one url and then uses another one to shorten the
> shortened
> url? Does long url return the original one?
>

It does. Except when the short url sevice itself blocks it. eg. when a
bit.ly link points to another shortened url, bit.ly redirects to an
intermediate warning page instead of the actual link.
Comment 12 Anant Kamath 2013-05-02 17:07:39 UTC
Solved. commit 8ccc35f38af1d910b8e04f5a820167d184724713Commit :