Empty Movie/Music lists when refreshing a large music collection

[jmencak]

Bug report

Describe the bug

After refreshing Music/Songs with a large song library, all Movie and Music items disappear.

Expected Behavior

Movie and Song items not disappearing after refresh of a large (~40000 song) database.

Actual Behavior

Movie and Song items disappearing after refresh of a large (~40000 song) database.

Possible Fix

A quick search suggests that adding endTransaction() might solve the issue. I haven't looked through the code where to add/fix this yet. The search also suggests that making batches when importing large datasets should be considered. I.e., making endTransaction() and starting a new transaction after getting n entities.

To Reproduce

Steps to reproduce the behavior:

1. Create a large (cca 40000 song) music collection
2. Go to Music/Songs. There will be no songs, but Music/Artists, Music/Albums, Music/Genres will be there.
3. Refresh the Music/Songs list.
4. All Movies, Music/Artists, Music/Albums, Music/Genres will disappear and you'll get SQLiteConnectionPool: The connection pool for database '/data/user/0/org.xbmc.kore/databases/xbmc.sqlite' has been unable to grant a connection to thread 66 (LibrarySyncService) with flags 0x2 for 30.000002 seconds. in the debug log.

Debuglog

The debuglog can be found here:

The following adb logcat shows the possible issue:

11-11 22:43:46.161 27169 27276 W SQLiteConnectionPool: The connection pool for database '/data/user/0/org.xbmc.kore/databases/xbmc.sqlite' has been unable to grant a connection to thread 66 (LibrarySyncService) with flags 0x2 for 30.000002 seconds.
11-11 22:43:46.161 27169 27276 W SQLiteConnectionPool: Connections: 1 active, 0 idle, 0 available.
11-11 22:43:46.161 27169 27276 W SQLiteConnectionPool: 
11-11 22:43:46.161 27169 27276 W SQLiteConnectionPool: Requests in progress:
11-11 22:43:46.161 27169 27276 W SQLiteConnectionPool:   executeForCursorWindow started 31641ms ago - running, sql="SELECT songs._id, songs.title, songs.track, songs.duration, songs.file, songs.songid, songs.displayartist, albums.title, albums.genre, albums.year, albums.thumbnail FROM songs JOIN song_artists ON song_artists.host_id=songs.host_id AND song_artists.songid=songs.songid LEFT JOIN artists ON song_artists.host_id=artists.host_id AND song_artists.artistid=artists.artistid LEFT JOIN album_artists ON song_artists.host_id=album_artists.host_id AND songs.albumid=album_artists.albumid LEFT JOIN albums ON song_artists.host_id=albums.host_id AND album_artists.albumid=albums.albumid WHERE (songs.host_id=?) GROUP BY songs.songid ORDER BY  (CASE  WHEN songs.title LIKE 'The %' THEN SUBSTR(songs.title,5) || ', The'  ELSE songs.title END)  COLLATE NOCASE ASC", path=/data/user/0/org.xbmc.kore/databases/xbmc.sqlite
z`

redfin:/ # ls -al /data/user/0/org.xbmc.kore/databases/xbmc.sqlite                                                                                                                            
-rw-rw---- 1 u0_a235 u0_a235 4595712 2024-11-11 23:08 /data/user/0/org.xbmc.kore/databases/xbmc.sqlite

Your Environment

Used Operating system:

• Mobile device: Google Pixel 5
• Android version: 12
• Kodi version and platform: 20.1, Debian 12
• Kore version: 3.1.0-7-ga891d5f8