Summary: | items disappear, then get reloaded | ||
---|---|---|---|
Product: | [Applications] akregator | Reporter: | Oliver Grimm <logistikka> |
Component: | general | Assignee: | kdepim bugs <kdepim-bugs> |
Status: | RESOLVED WORKSFORME | ||
Severity: | normal | CC: | alex.merry, dominik.tritscher |
Priority: | NOR | Keywords: | triaged |
Version: | 1.1.1 | ||
Target Milestone: | --- | ||
Platform: | Debian testing | ||
OS: | Linux | ||
Latest Commit: | Version Fixed In: |
Description
Oliver Grimm
2005-09-01 23:58:30 UTC
Yeah, can confirm this with current SVN (branches/KDE/3.5/). Strange though. Seems that the URL is too long. You can reproduce that problem with any feed URL e.g. by adding to the URL &somestrangeparameter=very+long+text+very+long+text+very+long+text... Archiving doesn't work for URLs longer than 255 chars, as the url is used as filename for the archive, and 255 is the max. length at least on ext2/3. We should use the first 200 + someHash(completeURL) SVN commit 465610 by osterfeld: for archiving, truncate URLs longer than 255 chars and append a hash value. This makes archives work for feeds with very long URLs. (due to filesystem limitations for file name length) BUG: 111908 M +2 -2 Makefile.am M +2 -17 article.cpp M +0 -1 article.h M +13 -2 mk4storage/feedstoragemk4impl.cpp M +11 -0 utils.cpp M +8 -2 utils.h --- branches/KDE/3.5/kdepim/akregator/src/Makefile.am #465609:465610 @@ -72,7 +72,8 @@ storagefactorydummyimpl.cpp \ simplenodeselector.cpp \ feedstoragedummyimpl.cpp \ - dragobjects.cpp + dragobjects.cpp \ + utils.cpp libakregatorprivate_la_LDFLAGS = $(all_libraries) -avoid-version -no-undefined libakregatorprivate_la_LIBADD = $(top_builddir)/libkdepim/libkdepim.la \ @@ -141,7 +142,6 @@ tagfolderitem.cpp \ speechclient.cpp \ kspeechsink.skel \ - utils.cpp \ kspeech.stub libakregatorpart_la_LDFLAGS = $(KDE_RPATH) $(KDE_PLUGIN) $(LIB_KUTILS) -avoid-version -no-undefined $(all_libraries) --- branches/KDE/3.5/kdepim/akregator/src/article.cpp #465609:465610 @@ -28,6 +28,7 @@ #include "storage.h" #include "librss/librss.h" #include "shared.h" +#include "utils.h" #include <qdatetime.h> #include <qdom.h> @@ -88,7 +89,7 @@ { d->archive = archive; d->status = Private::New; - d->hash = calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() ); + d->hash = Utils::calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() ); d->guid = article.guid(); @@ -336,22 +337,6 @@ return d->archive->guidIsPermaLink(d->guid); } -/* taken from some website... -fo -* djb2 -* This algorithm was first reported by Dan Bernstein -* many years ago in comp.lang.c -*/ -uint Article::calcHash(const QString& str) -{ - if (str.isNull()) // handle null string as "", prevents crash - return calcHash(""); - const char* s = str.ascii(); - uint hash = 5381; - int c; - while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c - return hash; -} - bool Article::guidIsHash() const { return d->archive->guidIsHash(d->guid); --- branches/KDE/3.5/kdepim/akregator/src/article.h #465609:465610 @@ -128,7 +128,6 @@ private: void initialize(RSS::Article article, Backend::FeedStorage* archive); - static uint calcHash(const QString& str); static QString buildTitle(const QString& description); struct Private; --- branches/KDE/3.5/kdepim/akregator/src/mk4storage/feedstoragemk4impl.cpp #465609:465610 @@ -26,6 +26,7 @@ #include "storagemk4impl.h" #include "../article.h" +#include "../utils.h" #include "../librss/article.h" #include "../librss/document.h" #include <mk4.h> @@ -33,6 +34,7 @@ #include <qdom.h> #include <qfile.h> +#include <kdebug.h> #include <kglobal.h> #include <kstandarddirs.h> @@ -131,8 +133,17 @@ d->autoCommit = main->autoCommit(); d->url = url; d->mainStorage = main; - QString t = url; - QString t2 = url; + + QString url2 = url; + + if (url.length() > 255) + { + url2 = url.left(200) + QString::number(Akregator::Utils::calcHash(url), 16); + } + + kdDebug() << url2 << endl; + QString t = url2; + QString t2 = url2; QString filePath = main->archivePath() +"/"+ t.replace("/", "_").replace(":", "_"); d->oldArchivePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + t2.replace("/", "_").replace(":", "_") + ".xml"; d->convert = !QFile::exists(filePath + ".mk4") && QFile::exists(d->oldArchivePath); --- branches/KDE/3.5/kdepim/akregator/src/utils.cpp #465609:465610 @@ -33,4 +33,15 @@ return QString(str).replace(QRegExp("<[^>]*>"), ""); } +uint Utils::calcHash(const QString& str) +{ + if (str.isNull()) // handle null string as "", prevents crash + return calcHash(""); + const char* s = str.ascii(); + uint hash = 5381; + int c; + while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c + return hash; } + +} --- branches/KDE/3.5/kdepim/akregator/src/utils.h #465609:465610 @@ -26,6 +26,7 @@ #define AKREGATOR_UTILS_H class QString; +typedef unsigned int uint; namespace Akregator { @@ -35,8 +36,13 @@ /** removes HTML/XML tags (everything between < and >, that is) from a string. "<p><strong>foo</strong> bar</p>" becomes "foo bar" */ static QString stripTags(const QString& str); - // /** strips tags, resolves entities and replaces <br/> by new lines */ - //static QString htmlToPlainText(const QString& str); + /** taken from some website... -fo + * djb2 + * This algorithm was first reported by Dan Bernstein + * many years ago in comp.lang.c + */ + + static uint calcHash(const QString& str); }; } SVN commit 465611 by osterfeld: forward port of 111908 (fix archiving for feed URLs > 255 chars) CCBUG: 111908 M +2 -2 Makefile.am M +2 -17 article.cpp M +0 -1 article.h M +13 -2 mk4storage/feedstoragemk4impl.cpp M +11 -0 utils.cpp M +8 -2 utils.h --- trunk/KDE/kdepim/akregator/src/Makefile.am #465610:465611 @@ -72,7 +72,8 @@ storagefactorydummyimpl.cpp \ simplenodeselector.cpp \ feedstoragedummyimpl.cpp \ - dragobjects.cpp + dragobjects.cpp \ + utils.cpp libakregatorprivate_la_LDFLAGS = $(all_libraries) -avoid-version -no-undefined libakregatorprivate_la_LIBADD = $(top_builddir)/libkdepim/libkdepim.la \ @@ -141,7 +142,6 @@ tagfolderitem.cpp \ speechclient.cpp \ kspeechsink.skel \ - utils.cpp \ kspeech.stub libakregatorpart_la_LDFLAGS = $(KDE_RPATH) $(KDE_PLUGIN) $(LIB_KUTILS) -avoid-version -no-undefined $(all_libraries) --- trunk/KDE/kdepim/akregator/src/article.cpp #465610:465611 @@ -28,6 +28,7 @@ #include "storage.h" #include "librss/librss.h" #include "shared.h" +#include "utils.h" #include <qdatetime.h> #include <qdom.h> @@ -88,7 +89,7 @@ { d->archive = archive; d->status = Private::New; - d->hash = calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() ); + d->hash = Utils::calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() ); d->guid = article.guid(); @@ -336,22 +337,6 @@ return d->archive->guidIsPermaLink(d->guid); } -/* taken from some website... -fo -* djb2 -* This algorithm was first reported by Dan Bernstein -* many years ago in comp.lang.c -*/ -uint Article::calcHash(const QString& str) -{ - if (str.isNull()) // handle null string as "", prevents crash - return calcHash(""); - const char* s = str.ascii(); - uint hash = 5381; - int c; - while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c - return hash; -} - bool Article::guidIsHash() const { return d->archive->guidIsHash(d->guid); --- trunk/KDE/kdepim/akregator/src/article.h #465610:465611 @@ -129,7 +129,6 @@ private: void initialize(RSS::Article article, Backend::FeedStorage* archive); - static uint calcHash(const QString& str); static QString buildTitle(const QString& description); struct Private; --- trunk/KDE/kdepim/akregator/src/mk4storage/feedstoragemk4impl.cpp #465610:465611 @@ -26,6 +26,7 @@ #include "storagemk4impl.h" #include "../article.h" +#include "../utils.h" #include "../librss/article.h" #include "../librss/document.h" #include <mk4.h> @@ -34,6 +35,7 @@ #include <qfile.h> #include <QTextStream> +#include <kdebug.h> #include <kglobal.h> #include <kstandarddirs.h> @@ -132,8 +134,17 @@ d->autoCommit = main->autoCommit(); d->url = url; d->mainStorage = main; - QString t = url; - QString t2 = url; + + QString url2 = url; + + if (url.length() > 255) + { + url2 = url.left(200) + QString::number(Akregator::Utils::calcHash(url), 16); + } + + kdDebug() << url2 << endl; + QString t = url2; + QString t2 = url2; QString filePath = main->archivePath() +"/"+ t.replace("/", "_").replace(":", "_"); d->oldArchivePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + t2.replace("/", "_").replace(":", "_") + ".xml"; d->convert = !QFile::exists(filePath + ".mk4") && QFile::exists(d->oldArchivePath); --- trunk/KDE/kdepim/akregator/src/utils.cpp #465610:465611 @@ -33,4 +33,15 @@ return QString(str).replace(QRegExp("<[^>]*>"), ""); } +uint Utils::calcHash(const QString& str) +{ + if (str.isNull()) // handle null string as "", prevents crash + return calcHash(""); + const char* s = str.ascii(); + uint hash = 5381; + int c; + while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c + return hash; } + +} --- trunk/KDE/kdepim/akregator/src/utils.h #465610:465611 @@ -26,6 +26,7 @@ #define AKREGATOR_UTILS_H class QString; +typedef unsigned int uint; namespace Akregator { @@ -35,8 +36,13 @@ /** removes HTML/XML tags (everything between < and >, that is) from a string. "<p><strong>foo</strong> bar</p>" becomes "foo bar" */ static QString stripTags(const QString& str); - // /** strips tags, resolves entities and replaces <br/> by new lines */ - //static QString htmlToPlainText(const QString& str); + /** taken from some website... -fo + * djb2 + * This algorithm was first reported by Dan Bernstein + * many years ago in comp.lang.c + */ + + static uint calcHash(const QString& str); }; } SVN commit 465621 by osterfeld: backport of 111908 (archive broken for feed URLs longer than 255 chars) CCBUG: 111908 M +14 -5 archive.cpp M +3 -1 myarticle.h --- branches/KDE/3.4/kdepim/akregator/src/archive.cpp #465620:465621 @@ -27,6 +27,7 @@ #include "feedgroup.h" #include "feediconmanager.h" #include "treenode.h" +#include "myarticle.h" #include <kurl.h> #include <kdebug.h> @@ -112,13 +113,17 @@ f->setFavicon(QPixmap(KGlobal::dirs()->findResource("cache", iconFile+".png"))); // images are cache, articles is data.. good I think. - QString u=f->xmlUrl(); - - QString imageFileName=KGlobal::dirs()->saveLocation("cache", "akregator/Media/")+u.replace("/", "_").replace(":", "_")+".png"; + QString u = url.prettyURL(-1); + + if (u.length() > 255) + u = u.left(200) + QString::number(MyArticle::calcHash(u), 16); + + QString imageFileName=KGlobal::dirs()->saveLocation("cache", "akregator/Media/") + QString(u).replace("/", "_").replace(":", "_")+".png"; f->setImage(QPixmap(imageFileName, "PNG")); - QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + url.prettyURL(-1).replace("/", "_").replace(":", "_") + ".xml"; + QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + QString(u).replace("/", "_").replace(":", "_") + ".xml"; + //kdDebug() << "Will read feed from " << filePath << endl; //kdDebug() << "merged :"<<f->isMerged()<<endl; @@ -170,8 +175,12 @@ KURL url( f->xmlUrl() ); + + QString u = url.prettyURL(-1); + if (u.length() > 255) + u = u.left(200) + QString::number(MyArticle::calcHash(u), 16); - QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + url.prettyURL(-1).replace("/", "_").replace(":", "_") + ".xml"; + QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + u.replace("/", "_").replace(":", "_") + ".xml"; //kdDebug() << "Will save feed to " << filePath << endl; --- branches/KDE/3.4/kdepim/akregator/src/myarticle.h #465620:465621 @@ -96,8 +96,10 @@ bool operator>(const MyArticle &other) const; bool operator>=(const MyArticle &other) const; + static uint calcHash(const QString& str); + private: - static uint calcHash(const QString& str); + struct Private; Private *d; QString buildTitle(); The bug had been fixed for the IDW feed, but now it's back for another feed: - attach http://www.liberafolio.org/search_rss?SearchableText=&Title=&Description=&portal_type%3Alist=News+Item&portal_type%3Alist=Link&portal_type%3Alist=Document&Creator=&submit=Search&sort_on=created&sort_order=reverse&review_state=published&sort_limit:int=20 - fetch the feed - read some messages - fetch the feed again - the messages are marked "unread" again Hmm, I can't reproduce it for the LiberaFolio feed. The status is unchanged after fetching and all works fine. Does your Akregator archive the feed? Try 1) fetch feed. Mark all as read. 2) restart Akregator 3) select feed Are the items still in there or is it empty? I tried that. The items are gone. It's simply the same bug as it happened with the original bug I reported (the IDW feed), though it's fixed for the IDW feed now. Maybe there is a problem with using a non-standard charset when hashing? ... just guessing ... Possible, in this case ~/.kde/share/apps/akregator/Archive/ should contain multiple archives of the feed (look for *liberiafolio*mk4, ignore ___CATEGORIES and ___TAGS files) with different hashes at the end. Could you check? I checked it. There is no LiberaFolio file in the folder. I checked it for every feed and there is one for each feed attached, but not for LiberaFolio. still a bug in Akregator 1.2.3, KDE 3.5.4 Still a bug in Akregator 1.2.5, KDE 3.5.5. There is no archive file for LiberaFolio in the folder you mentioned. If I can provide any other information to get this one fixed, please let me know. I retried on a completely new system: Debian/etch, locale=de.utf8. The bug reappears. Another example of a feed with which the bug occurs is http://www.presseportal.de/rss/dienststelle_7.rss still a bug in KDE 3.5.8 (checked with the LiberaFolio Feed) still a bug in KDE 3.5.9 Still can reproduce with the feed url from initial report using Akregator 1.3.3 (KDE4.1.3). still a bug in KDE 3.5.10; got a new machine, the bug has reappeared. Distro is Debian/testing, LOCALE is de_DE.UTF-8 Today I found something else: The navigation list on the left hand side in akregator shows some unread messages (number in brackets) though there are no unread messages left for the feed http://www.kde.org/dot/kde-apps-content.rdf. This issue is only present with this particular feed. Once I have entered the item list for this feed and press the key for "jump to next unread message" nothing happens. Is this bug present in Akregator from KDE 4? Can't reproduce in KDE 4, and reporter is not responding. |