duplicate articles across multiple feeds

[slbug]

Describe the bug
since 3.10.4 some feeds which was ok previously started to produce duplicate articles on each refresh.

To Reproduce
click refresh feed

Screenshots

Please complete the following information:

• Version 3.10.8 :0294d207: (9196)
• OS version: 26.5.1 (25F80)

Additional information:
impacted feeds 4 out of 8.
sample impacted - https://feeds.feedburner.com/org/LOR (cached filename folder21.xml)

[barijaona]

Do you delete articles after reading them or are you keeping them until the automatic cleanup operates ?
Do you frequently flag articles ("starring" them), or mark them unread after reading ?

[slbug]

no delete, no starring, no marking unread.

also if i select article which is duplicate, selection will jump to same article which was arrived first.

[barijaona]

I need some clarification here:
1/ you mean that if the same article appears twice in the article list, and for instance you select the article which is located upper, the selection will jump spontaneously to the first fetched version of this article ?
2/ does this occur when there is a periodic "refresh all subscriptions" occuring, or even between these refreshes?
3/ what sort criteria and order is selected ?(cf. View / Sort By menu)

[slbug]

1. will jump to first in list same article (based on my sort)
2. always if there is duplicates
3. last updated desc

[barijaona]

could you add a screenshot of the article list with one of the series of impacted itmes shown ? Regarding the "View" menu, set as follows:

• Columns : have both "Last Update" and "Date Published" being checked
• Layout : Horizontal
• Filter By: All Articles

[slbug]

i selected last article (which is now marked as read) and selection jumped to first one on screenshot. it's all dups

[barijaona]

In Terminal, what is the output of the following command ?

sqlite3 ~/Library/Application\ Support/Vienna/messages.db "select message_id, createddate, date, title, revised_flag, read_flag from messages where folder_id=21 and message_id like '%https://www.linux.org.ru/news/opensource/18342955%';"

[slbug]

https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|1
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|0
https://www.linux.org.ru/news/opensource/18342955|1784623611.0|1784623611.0|NetworkManager 1.58|0|0

[barijaona]

Quite unexpected. With the same database running on 3.10.3, no problem ?

Could you run these two database checks ?

sqlite3 ~/Library/Application\ Support/Vienna/messages.db "PRAGMA integrity_check;" ".exit" 

(should return ok)
and

 sqlite3 ~/Library/Application\ Support/Vienna/messages.db "PRAGMA foreign_key_check;" ".exit" 

(should return empty text)

[slbug]

i've not tried to downgrade, but it was ok before .4

first command

*** in database main ***
Tree 8 page 14583 right child: Bad ptr map entry key=16128 expected=(5,14583) got=(5,13939)
Tree 8 page 14583 right child: Bad ptr map entry key=16279 expected=(5,14583) got=(5,13939)
Tree 8 page 17451 cell 1: Rowid 943101 out of order
Tree 8 page 14583 right child: Bad ptr map entry key=17423 expected=(5,14583) got=(5,17090)
Tree 8 page 17423 cell 0: Rowid 943051 out of order
Tree 8 page 14583 right child: Bad ptr map entry key=17390 expected=(5,14583) got=(5,17090)
Tree 8 page 17390 cell 1: Rowid 942986 out of order
Tree 8 page 14583 right child: Bad ptr map entry key=17361 expected=(5,14583) got=(5,24205)
Tree 8 page 14583 right child: 2nd reference to page 17361
Tree 8 page 14583 right child: Bad ptr map entry key=17334 expected=(5,14583) got=(5,17090)
Tree 8 page 17334 cell 1: Rowid 942813 out of order
Tree 8 page 14583 right child: 2nd reference to page 13836
Tree 8 page 14583 right child: 2nd reference to page 16805
Tree 8 page 14583 right child: 2nd reference to page 16948
Tree 8 page 14583 right child: 2nd reference to page 16776
Tree 8 page 14583 right child: 2nd reference to page 16744
Tree 8 page 14583 right child: 2nd reference to page 13625
Tree 8 page 14583 right child: 2nd reference to page 15127
Tree 8 page 14583 right child: 2nd reference to page 14701
Tree 8 page 14583 right child: 2nd reference to page 6814
Tree 8 page 14583 right child: 2nd reference to page 17004
Tree 8 page 14583 right child: 2nd reference to page 15560
Tree 8 page 14583 right child: 2nd reference to page 16981
Tree 8 page 14583 right child: 2nd reference to page 15953
Tree 8 page 14583 right child: 2nd reference to page 16206
Tree 8 page 14583 right child: 2nd reference to page 13055
Tree 8 page 14583 right child: 2nd reference to page 16711
Tree 8 page 14583 right child: 2nd reference to page 16202
Tree 8 page 14583 right child: 2nd reference to page 16105
Tree 8 page 14583 right child: 2nd reference to page 15481
Tree 8 page 14583 right child: 2nd reference to page 15251
Tree 8 page 14583 right child: 2nd reference to page 16687
Tree 8 page 14583 right child: 2nd reference to page 16652
Tree 8 page 14583 right child: 2nd reference to page 15265
Tree 8 page 14583 right child: 2nd reference to page 16929
Tree 8 page 14583 right child: 2nd reference to page 17112
Tree 8 page 14583 right child: 2nd reference to page 17084
Tree 8 page 14583 right child: 2nd reference to page 14527
Tree 8 page 14583 right child: 2nd reference to page 16119
Tree 8 page 14583 right child: 2nd reference to page 14998
Tree 8 page 14583 right child: 2nd reference to page 16044
Tree 8 page 14583 right child: 2nd reference to page 16251
Tree 8 page 14583 right child: 2nd reference to page 15373
Tree 8 page 14583 right child: 2nd reference to page 16228
Tree 8 page 14583 right child: 2nd reference to page 14983
Tree 8 page 14583 right child: 2nd reference to page 15342
Tree 8 page 14583 right child: 2nd reference to page 14616
Tree 8 page 14583 right child: 2nd reference to page 16020
Tree 8 page 14583 right child: 2nd reference to page 15892
Tree 8 page 14583 right child: 2nd reference to page 16566
Tree 8 page 14583 right child: 2nd reference to page 15825
Tree 8 page 14583 right child: 2nd reference to page 15314
Tree 8 page 14583 right child: 2nd reference to page 16917
Tree 8 page 14583 right child: 2nd reference to page 16882
Tree 8 page 14583 right child: 2nd reference to page 7019
Tree 8 page 14583 right child: 2nd reference to page 16274
Tree 8 page 14583 right child: 2nd reference to page 14871
Tree 8 page 14583 right child: 2nd reference to page 16635
Tree 8 page 14583 right child: 2nd reference to page 17297
Tree 8 page 14583 right child: 2nd reference to page 17266
Tree 8 page 14583 right child: 2nd reference to page 17233
Tree 8 page 14583 right child: 2nd reference to page 17205
Tree 8 page 14583 right child: 2nd reference to page 17174
Tree 8 page 14583 right child: 2nd reference to page 17146
Tree 8 page 14583 right child: 2nd reference to page 17026
Tree 8 page 14583 right child: 2nd reference to page 15415
Tree 8 page 14583 right child: 2nd reference to page 16545
Tree 8 page 14583 right child: 2nd reference to page 16500
Tree 8 page 14583 right child: 2nd reference to page 16100
Tree 8 page 14583 right child: 2nd reference to page 16426
Tree 8 page 14583 right child: 2nd reference to page 16849
Tree 8 page 14583 right child: 2nd reference to page 14683
Tree 8 page 14583 right child: 2nd reference to page 12361
Tree 8 page 14583 right child: 2nd reference to page 15723
Tree 8 page 14583 right child: 2nd reference to page 14977
Tree 8 page 14583 right child: 2nd reference to page 15273
Tree 8 page 14583 right child: 2nd reference to page 15693
Tree 8 page 14583 right child: 2nd reference to page 13567
Tree 8 page 14583 right child: 2nd reference to page 13958
Tree 8 page 14583 right child: 2nd reference to page 15187
Tree 8 page 14583 right child: 2nd reference to page 16803
Tree 8 page 14583 right child: 2nd reference to page 16798
Tree 8 page 14583 right child: 2nd reference to page 16767
Tree 8 page 14583 right child: 2nd reference to page 16735
Tree 8 page 14583 right child: 2nd reference to page 13572
Tree 8 page 14583 right child: 2nd reference to page 14958
Tree 8 page 14583 right child: 2nd reference to page 15627
Tree 8 page 14583 right child: 2nd reference to page 14698
Tree 8 page 14583 right child: 2nd reference to page 17094
Tree 8 page 14583 right child: 2nd reference to page 17063
Tree 8 page 14583 right child: 2nd reference to page 17038
Tree 8 page 14583 right child: 2nd reference to page 17010
Tree 8 page 14583 right child: 2nd reference to page 15004
Tree 8 page 14583 right child: 2nd reference to page 16978
Tree 8 page 14583 right child: 2nd reference to page 16945
Tree 8 page 14583 right child: 2nd reference to page 6700
Tree 8 page 14583 right child: 2nd reference to page 14388
Tree 8 page 14583 right child: 2nd reference to page 15435
Tree 8 page 14583 right child: 2nd reference to page 16197
Tree 8 page 14583 right child: 2nd reference to page 16709

2nd nothing

[barijaona]

We are confronted to a corrupted database, which apparently wasn't detected by the quick check we perform at startup. Difficult for me to guess the origin. My suggestion : double check your disk backups and run a thorough disk verification.

Regarding Vienna, first thing to do: save your subscriptions somewhere. Menu File-> Export Subscriptions.
The easiest workaround would be to put aside the current database, ~/Library/Application\ Support/Vienna/messages.db, relaunch Vienna which will recreate a fresh database and reimport this subscriptions list. You will probably lose some history though.

[slbug]

that's actually the biggest problem to loose 10 years of history :)

[barijaona]

But I see you have "move articles to trash" set to 1 month ? How can you keep 10 years of history this way ?

[slbug]

hahaha, but unread articles :)