Bug 111908 - items disappear, then get reloaded
Summary: items disappear, then get reloaded
Status: RESOLVED WORKSFORME
Alias: None
Product: akregator
Classification: Applications
Component: general (show other bugs)
Version: 1.1.1
Platform: Debian testing Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords: triaged
Depends on:
Blocks:
 
Reported: 2005-09-01 23:58 UTC by Oliver Grimm
Modified: 2013-01-11 20:10 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Oliver Grimm 2005-09-01 23:58:30 UTC
Version:           1.1.1 (using KDE KDE 3.4.1)
Installed from:    Debian testing/unstable Packages
OS:                Linux

To reproduce do

- start Akregator
- attach the following RSS feed:

http://idw-online.de/pages/de/pressreleasesrss?langs=de_DE&province_ids=10&field_ids=4&field_ids=26&field_ids=5&field_ids=18&category_ids=10&category_ids=5&category_ids=7&category_ids=8&category_ids=1&category_ids=4&category_ids=6&category_ids=3&category_ids=2&category_ids=9

- fetch new items for this feed
- mark all items as "read"
- close Akregator
- quit Akregator by choosing "quit" from the tray icon menu
- restart Akregator
- have a look at the IDW feed you just set up
- all items are removed (!!) though they are only supposed to be marked as "read". If you compare this to other feeds, the problem doesn't occur with those. The "remove setting" is not set to "delete immediately".
- now fetch the IDW feed again
- you get the same old items again.

The problem only occurs with this particular feed.
Comment 1 Eckhart Wörner 2005-09-02 01:37:00 UTC
Yeah, can confirm this with current SVN (branches/KDE/3.5/). Strange though.
Comment 2 Eckhart Wörner 2005-09-02 02:00:54 UTC
Seems that the URL is too long. You can reproduce that problem with any feed URL e.g. by adding to the URL
&somestrangeparameter=very+long+text+very+long+text+very+long+text...
Comment 3 Frank Osterfeld 2005-09-30 10:41:31 UTC
Archiving doesn't work for URLs longer than 255 chars, as the url is used as filename for the archive, and 255 is the max. length at least on ext2/3.
We should use the first 200 + someHash(completeURL)
Comment 4 Frank Osterfeld 2005-09-30 11:48:53 UTC
SVN commit 465610 by osterfeld:

for archiving, truncate URLs longer than 255 chars and append a hash value.
This makes archives work for feeds with very long URLs. (due to filesystem limitations for file name length)

BUG: 111908


 M  +2 -2      Makefile.am  
 M  +2 -17     article.cpp  
 M  +0 -1      article.h  
 M  +13 -2     mk4storage/feedstoragemk4impl.cpp  
 M  +11 -0     utils.cpp  
 M  +8 -2      utils.h  


--- branches/KDE/3.5/kdepim/akregator/src/Makefile.am #465609:465610
@@ -72,7 +72,8 @@
                                 storagefactorydummyimpl.cpp \
                                 simplenodeselector.cpp \
                                 feedstoragedummyimpl.cpp \
-                                dragobjects.cpp
+                                dragobjects.cpp \
+                                utils.cpp
 				
 libakregatorprivate_la_LDFLAGS = $(all_libraries) -avoid-version -no-undefined 
 libakregatorprivate_la_LIBADD =	$(top_builddir)/libkdepim/libkdepim.la \
@@ -141,7 +142,6 @@
         tagfolderitem.cpp \
         speechclient.cpp \
 	kspeechsink.skel \
-        utils.cpp \
 	kspeech.stub
 
 libakregatorpart_la_LDFLAGS = $(KDE_RPATH) $(KDE_PLUGIN) $(LIB_KUTILS) -avoid-version -no-undefined $(all_libraries)
--- branches/KDE/3.5/kdepim/akregator/src/article.cpp #465609:465610
@@ -28,6 +28,7 @@
 #include "storage.h"
 #include "librss/librss.h"
 #include "shared.h"
+#include "utils.h"
 
 #include <qdatetime.h>
 #include <qdom.h>
@@ -88,7 +89,7 @@
 {
     d->archive = archive;
     d->status = Private::New;
-    d->hash = calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() );
+    d->hash = Utils::calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() );
 
     d->guid = article.guid();
     
@@ -336,22 +337,6 @@
     return d->archive->guidIsPermaLink(d->guid);
 }
 
-/* taken from some website... -fo
-* djb2
-* This algorithm was first reported by Dan Bernstein
-* many years ago in comp.lang.c
-*/
-uint Article::calcHash(const QString& str)
-{
-    if (str.isNull()) // handle null string as "", prevents crash
-        return calcHash("");
-    const char* s = str.ascii();
-    uint hash = 5381;
-    int c;
-    while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c
-    return hash;
-}
-
 bool Article::guidIsHash() const
 {
     return d->archive->guidIsHash(d->guid);
--- branches/KDE/3.5/kdepim/akregator/src/article.h #465609:465610
@@ -128,7 +128,6 @@
 
         private:
             void initialize(RSS::Article article, Backend::FeedStorage* archive);
-            static uint calcHash(const QString& str);
             static QString buildTitle(const QString& description);
             
             struct Private;
--- branches/KDE/3.5/kdepim/akregator/src/mk4storage/feedstoragemk4impl.cpp #465609:465610
@@ -26,6 +26,7 @@
 #include "storagemk4impl.h"
 
 #include "../article.h"
+#include "../utils.h"
 #include "../librss/article.h"
 #include "../librss/document.h"
 #include <mk4.h>
@@ -33,6 +34,7 @@
 #include <qdom.h>
 #include <qfile.h>
 
+#include <kdebug.h>
 #include <kglobal.h>
 #include <kstandarddirs.h>
 
@@ -131,8 +133,17 @@
     d->autoCommit = main->autoCommit();
     d->url = url;
     d->mainStorage = main;
-    QString t = url;
-    QString t2 = url;
+
+    QString url2 = url;
+
+    if (url.length() > 255)
+    {
+        url2 = url.left(200) + QString::number(Akregator::Utils::calcHash(url), 16);
+    }
+    
+    kdDebug() << url2 << endl;
+    QString t = url2;
+    QString t2 = url2;
     QString filePath = main->archivePath() +"/"+ t.replace("/", "_").replace(":", "_");
     d->oldArchivePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + t2.replace("/", "_").replace(":", "_") + ".xml";
     d->convert = !QFile::exists(filePath + ".mk4") && QFile::exists(d->oldArchivePath);
--- branches/KDE/3.5/kdepim/akregator/src/utils.cpp #465609:465610
@@ -33,4 +33,15 @@
     return QString(str).replace(QRegExp("<[^>]*>"), "");
 }
 
+uint Utils::calcHash(const QString& str)
+{
+    if (str.isNull()) // handle null string as "", prevents crash
+        return calcHash("");
+    const char* s = str.ascii();
+    uint hash = 5381;
+    int c;
+    while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c
+    return hash;
 }
+
+}
--- branches/KDE/3.5/kdepim/akregator/src/utils.h #465609:465610
@@ -26,6 +26,7 @@
 #define AKREGATOR_UTILS_H
 
 class QString;
+typedef unsigned int uint;
 
 namespace Akregator {
 
@@ -35,8 +36,13 @@
     /** removes HTML/XML tags (everything between < and >, that is) from a string.  "<p><strong>foo</strong> bar</p>" becomes "foo bar" */
     static QString stripTags(const QString& str);
 
-    // /** strips tags, resolves entities and replaces <br/> by new lines */
-    //static QString htmlToPlainText(const QString& str);
+    /** taken from some website... -fo
+    * djb2
+    * This algorithm was first reported by Dan Bernstein
+    * many years ago in comp.lang.c
+    */
+
+    static uint calcHash(const QString& str);
 };
 
 }
Comment 5 Frank Osterfeld 2005-09-30 11:51:38 UTC
SVN commit 465611 by osterfeld:

forward port of 111908 (fix archiving for feed URLs > 255 chars)
CCBUG: 111908


 M  +2 -2      Makefile.am  
 M  +2 -17     article.cpp  
 M  +0 -1      article.h  
 M  +13 -2     mk4storage/feedstoragemk4impl.cpp  
 M  +11 -0     utils.cpp  
 M  +8 -2      utils.h  


--- trunk/KDE/kdepim/akregator/src/Makefile.am #465610:465611
@@ -72,7 +72,8 @@
                                 storagefactorydummyimpl.cpp \
                                 simplenodeselector.cpp \
                                 feedstoragedummyimpl.cpp \
-                                dragobjects.cpp
+                                dragobjects.cpp \
+                                utils.cpp
 				
 libakregatorprivate_la_LDFLAGS = $(all_libraries) -avoid-version -no-undefined 
 libakregatorprivate_la_LIBADD =	$(top_builddir)/libkdepim/libkdepim.la \
@@ -141,7 +142,6 @@
         tagfolderitem.cpp \
         speechclient.cpp \
 	kspeechsink.skel \
-        utils.cpp \
 	kspeech.stub
 
 libakregatorpart_la_LDFLAGS = $(KDE_RPATH) $(KDE_PLUGIN) $(LIB_KUTILS) -avoid-version -no-undefined $(all_libraries)
--- trunk/KDE/kdepim/akregator/src/article.cpp #465610:465611
@@ -28,6 +28,7 @@
 #include "storage.h"
 #include "librss/librss.h"
 #include "shared.h"
+#include "utils.h"
 
 #include <qdatetime.h>
 #include <qdom.h>
@@ -88,7 +89,7 @@
 {
     d->archive = archive;
     d->status = Private::New;
-    d->hash = calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() );
+    d->hash = Utils::calcHash(article.title() + article.description() + article.link().url() + article.commentsLink().url() );
 
     d->guid = article.guid();
     
@@ -336,22 +337,6 @@
     return d->archive->guidIsPermaLink(d->guid);
 }
 
-/* taken from some website... -fo
-* djb2
-* This algorithm was first reported by Dan Bernstein
-* many years ago in comp.lang.c
-*/
-uint Article::calcHash(const QString& str)
-{
-    if (str.isNull()) // handle null string as "", prevents crash
-        return calcHash("");
-    const char* s = str.ascii();
-    uint hash = 5381;
-    int c;
-    while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c
-    return hash;
-}
-
 bool Article::guidIsHash() const
 {
     return d->archive->guidIsHash(d->guid);
--- trunk/KDE/kdepim/akregator/src/article.h #465610:465611
@@ -129,7 +129,6 @@
 
         private:
             void initialize(RSS::Article article, Backend::FeedStorage* archive);
-            static uint calcHash(const QString& str);
             static QString buildTitle(const QString& description);
             
             struct Private;
--- trunk/KDE/kdepim/akregator/src/mk4storage/feedstoragemk4impl.cpp #465610:465611
@@ -26,6 +26,7 @@
 #include "storagemk4impl.h"
 
 #include "../article.h"
+#include "../utils.h"
 #include "../librss/article.h"
 #include "../librss/document.h"
 #include <mk4.h>
@@ -34,6 +35,7 @@
 #include <qfile.h>
 #include <QTextStream>
 
+#include <kdebug.h>
 #include <kglobal.h>
 #include <kstandarddirs.h>
 
@@ -132,8 +134,17 @@
     d->autoCommit = main->autoCommit();
     d->url = url;
     d->mainStorage = main;
-    QString t = url;
-    QString t2 = url;
+
+    QString url2 = url;
+
+    if (url.length() > 255)
+    {
+        url2 = url.left(200) + QString::number(Akregator::Utils::calcHash(url), 16);
+    }
+    
+    kdDebug() << url2 << endl;
+    QString t = url2;
+    QString t2 = url2;
     QString filePath = main->archivePath() +"/"+ t.replace("/", "_").replace(":", "_");
     d->oldArchivePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + t2.replace("/", "_").replace(":", "_") + ".xml";
     d->convert = !QFile::exists(filePath + ".mk4") && QFile::exists(d->oldArchivePath);
--- trunk/KDE/kdepim/akregator/src/utils.cpp #465610:465611
@@ -33,4 +33,15 @@
     return QString(str).replace(QRegExp("<[^>]*>"), "");
 }
 
+uint Utils::calcHash(const QString& str)
+{
+    if (str.isNull()) // handle null string as "", prevents crash
+        return calcHash("");
+    const char* s = str.ascii();
+    uint hash = 5381;
+    int c;
+    while ( ( c = *s++ ) ) hash = ((hash << 5) + hash) + c; // hash*33 + c
+    return hash;
 }
+
+}
--- trunk/KDE/kdepim/akregator/src/utils.h #465610:465611
@@ -26,6 +26,7 @@
 #define AKREGATOR_UTILS_H
 
 class QString;
+typedef unsigned int uint;
 
 namespace Akregator {
 
@@ -35,8 +36,13 @@
     /** removes HTML/XML tags (everything between < and >, that is) from a string.  "<p><strong>foo</strong> bar</p>" becomes "foo bar" */
     static QString stripTags(const QString& str);
 
-    // /** strips tags, resolves entities and replaces <br/> by new lines */
-    //static QString htmlToPlainText(const QString& str);
+    /** taken from some website... -fo
+    * djb2
+    * This algorithm was first reported by Dan Bernstein
+    * many years ago in comp.lang.c
+    */
+
+    static uint calcHash(const QString& str);
 };
 
 }
Comment 6 Frank Osterfeld 2005-09-30 12:05:58 UTC
SVN commit 465621 by osterfeld:

backport of 111908 (archive broken for feed URLs longer than 255 chars)
CCBUG: 111908


 M  +14 -5     archive.cpp  
 M  +3 -1      myarticle.h  


--- branches/KDE/3.4/kdepim/akregator/src/archive.cpp #465620:465621
@@ -27,6 +27,7 @@
 #include "feedgroup.h"
 #include "feediconmanager.h"
 #include "treenode.h"
+#include "myarticle.h"
 
 #include <kurl.h>
 #include <kdebug.h>
@@ -112,13 +113,17 @@
         f->setFavicon(QPixmap(KGlobal::dirs()->findResource("cache", iconFile+".png")));
 
     // images are cache, articles is data.. good I think.
-    QString u=f->xmlUrl();
-    
-    QString imageFileName=KGlobal::dirs()->saveLocation("cache", "akregator/Media/")+u.replace("/", "_").replace(":", "_")+".png";
+    QString u = url.prettyURL(-1);
+
+    if (u.length() > 255)
+        u = u.left(200) + QString::number(MyArticle::calcHash(u), 16);
+
+    QString imageFileName=KGlobal::dirs()->saveLocation("cache", "akregator/Media/") + QString(u).replace("/", "_").replace(":", "_")+".png";
     f->setImage(QPixmap(imageFileName, "PNG")); 
     
-    QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + url.prettyURL(-1).replace("/", "_").replace(":", "_") + ".xml";
+    QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + QString(u).replace("/", "_").replace(":", "_") + ".xml";
 
+   
     //kdDebug() << "Will read feed from " << filePath << endl;
     //kdDebug() << "merged :"<<f->isMerged()<<endl;
 
@@ -170,8 +175,12 @@
         
 
     KURL url( f->xmlUrl() );
+    
+    QString u = url.prettyURL(-1);
+    if (u.length() > 255)
+        u = u.left(200) + QString::number(MyArticle::calcHash(u), 16);
 
-    QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + url.prettyURL(-1).replace("/", "_").replace(":", "_") + ".xml";
+    QString filePath = KGlobal::dirs()->saveLocation("data", "akregator/Archive/") + u.replace("/", "_").replace(":", "_") + ".xml";
 
     //kdDebug() << "Will save feed to " << filePath << endl;
 
--- branches/KDE/3.4/kdepim/akregator/src/myarticle.h #465620:465621
@@ -96,8 +96,10 @@
             bool operator>(const MyArticle &other) const;
             bool operator>=(const MyArticle &other) const;
             
+            static uint calcHash(const QString& str);
+
         private:
-            static uint calcHash(const QString& str);
+            
             struct Private;
             Private *d;
             QString buildTitle();
Comment 7 Oliver Grimm 2006-02-03 11:55:33 UTC
The bug had been fixed for the IDW feed, but now it's back for another feed:

- attach http://www.liberafolio.org/search_rss?SearchableText=&Title=&Description=&portal_type%3Alist=News+Item&portal_type%3Alist=Link&portal_type%3Alist=Document&Creator=&submit=Search&sort_on=created&sort_order=reverse&review_state=published&sort_limit:int=20
- fetch the feed
- read some messages
- fetch the feed again
- the messages are marked "unread" again
Comment 8 Frank Osterfeld 2006-02-04 15:48:46 UTC
Hmm, I can't reproduce it for the LiberaFolio feed. The status is unchanged after fetching and all works fine. Does your Akregator archive the feed?
Try

1) fetch feed. Mark all as read.
2) restart Akregator
3) select feed

Are the items still in there or is it empty?
Comment 9 Oliver Grimm 2006-02-10 15:56:08 UTC
I tried that. The items are gone. It's simply the same bug as it happened with the original bug I reported (the IDW feed), though it's fixed for the IDW feed now.

Maybe there is a problem with using a non-standard charset when hashing? ... just guessing ...
Comment 10 Frank Osterfeld 2006-02-10 16:24:25 UTC
Possible, in this case ~/.kde/share/apps/akregator/Archive/ should contain
multiple archives of the feed (look for *liberiafolio*mk4, ignore ___CATEGORIES and ___TAGS files) with different hashes at the end.
Could you check?
Comment 11 Oliver Grimm 2006-02-10 22:54:44 UTC
I checked it. There is no LiberaFolio file in the folder. I checked it for every  feed and there is one for each feed attached, but not for LiberaFolio.
Comment 12 Oliver Grimm 2006-08-21 00:28:16 UTC
still a bug in Akregator 1.2.3, KDE 3.5.4
Comment 13 Oliver Grimm 2006-11-10 09:57:58 UTC
Still a bug in Akregator 1.2.5, KDE 3.5.5.
There is no archive file for LiberaFolio in the folder you mentioned.
If I can provide any other information to get this one fixed, please let me know.
Comment 14 Oliver Grimm 2006-12-21 10:32:52 UTC
I retried on a completely new system: Debian/etch, locale=de.utf8. The bug reappears.
Comment 15 Oliver Grimm 2006-12-29 21:32:29 UTC
Another example of a feed with which the bug occurs is http://www.presseportal.de/rss/dienststelle_7.rss
Comment 16 Oliver Grimm 2008-03-30 16:34:21 UTC
still a bug in KDE 3.5.8 (checked with the LiberaFolio Feed)
Comment 17 Oliver Grimm 2008-04-19 14:46:34 UTC
still a bug in KDE 3.5.9
Comment 18 Dominik Tritscher 2008-11-09 12:37:57 UTC
Still can reproduce with the feed url from initial report using Akregator 1.3.3 (KDE4.1.3).
Comment 19 Oliver Grimm 2008-12-07 16:31:03 UTC
still a bug in KDE 3.5.10;
got a new machine, the bug has reappeared.
Distro is Debian/testing, LOCALE is de_DE.UTF-8
Comment 20 Oliver Grimm 2008-12-14 21:07:24 UTC
Today I found something else:
The navigation list on the left hand side in akregator shows some unread messages (number in brackets) though there are no unread messages left for the feed http://www.kde.org/dot/kde-apps-content.rdf. This issue is only present with this particular feed. Once I have entered the item list for this feed and press the key for "jump to next unread message" nothing happens.
Comment 21 Alex Merry 2011-10-03 17:00:42 UTC
Is this bug present in Akregator from KDE 4?
Comment 22 Alex Merry 2013-01-11 20:10:27 UTC
Can't reproduce in KDE 4, and reporter is not responding.