Attempt at Hires Lossless (Max) and Dolby Atmos support #100

[smoothquark]

Continued from michaelherger#100

[manuelmaniero]

Hi @smoothquark,I've installed your release of TIDAL Hi-Res plugin.
I've found that Daphile mediaserver cache is placed here: /var/daphile/mediaserver/cache/tmp
So, I've tried to play an Hi-Res track and effectively I have an xxxx.mpd file in the cache path.
Sound still doesn't came out...
I verified file types and seems to be ok:

Here's squeezelite logs:

TIDAL Plugin config:

I don't know If I'm doing it right, but I've tried to play the mpd from command line, this is the result.
In transcoding log nothing is written by palying an Hi-Res track.

I think it should be a problem with the "daphile" player that is should be like a squeezelite one, but now I have some doubts...

[smoothquark]

I am not sure why you are getting invalid data from the mpd. You can try the following to ascertain what is going on. It has to be fresh mpd though (less than hour) due to the token expiring.

the .mpd file size should be around 1591 bytes

This should give some meaningful data (assuming daphile has ffprobe installed):
ffprobe -i xxx.mpd

You could try playing it using ffplay (if it is installed).
ffplay -i xxx.mpd

On my desktop system with pipewire where I am running my LMS server, I had to change the owner of the file to myself for ffplay or mpv to play the file, but no issues with LMS playing to squeezelite on my system.

Also, this should show the mpd contents (XML file):
cat xxx.mpd

If the mpd file is OK, then I think it must be transcoding issue. You need to enable transcoding logs.

The transcoding logs will then be in the server.log

However, your squeezelite screenshot is showing that it is trying to play a flac file, which should be the case. Going through the above should narrow down whether the .mpd is somehow not being saved correctly, transcoding issue or something to do squeezelite.

[manuelmaniero]

@smoothquark I've tried the cat command and I've found that probably the problem is the MPD that seems to be incomplete (MPD generated 2 mins ago):


But I don't have any ideas for what causes this

[smoothquark]

That is very odd. Either the manifest is not decoded correctly or not saved correctly for some reason. I've just uploaded a new commit with debugging of the manifest. You'll need to enable TIDAL debugging (set to 'debug') and upload the server log to look at. It should probably be at /var/daphile/mediaserver/server.log.

This should just show the TIDAL related logs
cat <server.log> | grep Plugins::TIDAL

There will also be the response returned for the track and the encrypted manifest.

[manuelmaniero]

@smoothquark thanks for your support, I really appreciate it.
Here's my server.log file with Tidal in debug mode.
I've inserted to the queue the entire album "The Wall" by Pink Floyd so there are many tracks appearing in server.log.
I've extracted it immediately after a reboot, it should be clear enough
server.log

[smoothquark]

Thanks. I shall have a look at it later this evening. I am currently doing more cleaning up of the code.

[smoothquark]

Looks like the manifest is correctly decoded. So, there must be a problem with how the file is saved. I shall go through the file saving code again and see if I can find anything that may clarify the cause.

[smoothquark]

Those tracks are all playing fine on my system. Anyway, I have made some changes to the repository. I am redoing the main branch and trying to make the changes more logical. Whilst I continue working on the main branch, I have created a 'testing' branch with the minimum changes required for HiRes and Atmos support and also some minor changes to the mpd file save code. I doubt it will make a difference but perhaps worth a try. I assume none of your hires tracks are playing? Silly question - I presume there is enough space in the cache folder and the permissions are correct? Are there any other files in the cache/tmp folder? The only other thing I can think of is whether there is a Perl version issue. I don't like the idea of cluttering the cache/tmp folder with mpd files but there does not seem to be a choice. There is also an option to use the system /tmp folder, which we could try if necessary.
Thank you once again for testing this out.

[manuelmaniero]

Something has changed.
I've done a brutal "chmod 777" on cache/tmp and I've installed the 'testing' branch plugin.
Now, the XML seems to be complete:
SKEKrQxtNI.mpd.zip
If I try ffprobe command I get this result:

Maybe should be a problem with ffmpeg? It's strange that I haven't any transcoding logs in my system If I play an HiRes track.

For your question: yes, the problem is for all HiRes tracks. I have enough space on cache/tmp. No, there aren't other files in /var/daphile/mediaserver/cache/tmp, only the MPD that I try to play.

[smoothquark]

Yes, the MPD file is now correct and even plays on my system with both mpv and ffplay, presumably because the token is fresh enough. I am not sure whether the changes I made on the testing branch have made any difference. The only real difference was to force close the write rather than rely on an auto close.
You ffmpeg version is newer than mine and I would expect it to support DASH assuming it was compiled in. You can check as follows:
ffmpeg -formats | grep dash

You should get:
DE dash DASH Muxer
DE webm_dash_manifest WebM DASH Manifest

If you don't, then MPD DASH support is not compiled in. If you do, there must be some odd permission issue. When I run ffprobe, I get a libpostproc entry - but I think that is only for video processing.

[smoothquark]

My other thought is whether it has anything to do with the LMS version? I am not sure which version does Daphile run. Mine is version 9.0.1 on Debian Bookworm. Unfortunately I do not have currently have a spare PC to try Daphile on, but I have been testing out mpd via upmpdcli on another Debian Bookworm system too, so I could try installing Daphile on it and see what happens.

[manuelmaniero]

LMS version in Daphile is 8.5.3
With ffmpeg -formats | grep dash I get the following:

Maybe the problem is that "dash DASH Muxer" is only E and not DE?

[smoothquark]

Ah yes, I think that might be the case, unfortunately. I don't know if there is any way or plans to have MPD support incorporated within Daphile. A quick search shows this up (https://forums.lyrion.org/forum/user-forums/logitech-media-server/1697301-daphile-lms-deezer) regarding some issues with LMS-Deezer but an update to Daphile resolved it.

LMS does not have built in support for DASH, but I think the BBC Sounds and Youtube plugins have implemented their own versions. If I am not mistaken, MP4 support started of as a plugin too before become incorporated into LMS. Its implementation is rather flaky, almost certainly due to different version and codecs within MP4. I have been considering trying to get MPD support done for TIDAL but it does look like a lot of work.

[manuelmaniero]

I've changed the entire system to try to resolve.
So, I've installed Ubuntu Server + Snakeoil OS + LMS 9.0.2
I have recompiled ffmpeg so now I'm in this situation:

But the problem now is different, seems that Tidal Plugin can't generate the command line to play my file:
server.log (2).zip
Should be a player problem or what else? CDQ tracks are ok via Tidal in LMS...

Edit: I've made a test with a local WAV track 24b/192kHz with no issues and my DAC recognise it correctly

[smoothquark]

Thank you for that. Some observations:

1. The FullTextSearch plugin is creating a lot of noise in the server.log. Can you please either turn off the plugin or its debugging.

2. There is a Warning: /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/TIDAL: Permission denied at /usr/share/squeezeboxserver/CPAN/File/Next.pm line 217.

The TIDAL plugin may not be able to write the .mpd file. Are there any .mpd files in the cache/tmp folder? I think it should be at /var/lib/squeezeboxserver/cache/tmp.

ls -l /var/lib/squeezeboxserver/cache:
drwxr-xr-x 2 squeezeboxserver nogroup 4096 Mar 15 17:37 tmp

3. There are some database errors too. Not sure why this is the case. Are you trying to play from My Music -> Releases? If so, please try playing from Apps->TIDAL. The testing branch does not have any import code changes, although when I tested it, my albums were successfully imported and played correctly via Releases. Your Lyrion version is 9.0.2, whilst mine is still at 9.0.1. I shall try upgrading mine to see if I can replicate the database errors.

4. It does not look like you have enabled logging for Transcoding?

[smoothquark]

I've added further logging in the testing branch to show where the mpd file is being saved to to help with debugging.

[manuelmaniero]

Solved by reinstalling Ubuntu but in "Desktop" version that has native support for D/E "Muxer" + LMS 9.0.2 + Squeezelite as a player.
Now HiRes tracks play perfectly.
I think in previous installation I could made some errors in configuring ffmpeg cause when I installed it, originally supports only encoding in DASH so I done a "./configure" command with some parameters (I asked ChatGPT...).

[smoothquark]

That is good news indeed. Glad you got it working. I've now added more functionality in the main branch (documented in README-CHANGES.md). I am trying to get the plugin to work better than the TIDAL apps (probably not too difficult as the apps are awful). I have also been looking at playing mpd files without resorting to transcoding via ffmpeg, but not sure whether it is worth the effort though. It probably would be, if only to avoid cluttering up the cache/tmp folder.

[manuelmaniero]

@smoothquark update: Daphile is now officially on LMS 9.
So, I've done another try.
I used the beta with SSH access. Installed "standard" TIDAL plugin and replaced the content of the dir with your branch. Registered account, inserted Client ID and Secret and...music flows like a charm. CDQ, 24bit/44.1, 24bit/96kHz and so on.

Thanks for all your work.

[smoothquark]

Thank you. It is working well for me too, much better than the Tidal apps! I do need to make a few tweaks though - for example the bitrate shown is not correct. I do get the odd occasion where some tracks stop playing, but it is not reproducible.

[manuelmaniero]

@smoothquark bad news: the plugin doesn’t work anymore for me(see comments on the original michaelherger repo)

[smoothquark]

Hmm.. I just tried playing an album from Tidal on my system and it is working fine. Looking at the comments on michaelherger#104 it looks like the OAuth Client ID, and secret have been updated. Interestingly, my bash script using the same Client ID and secret has stopped working, so I presume it will soon stop working on my system too. I am quite busy over the next few days, but should hopefully have some time next week to have a look. This was always going to be a hack to get HiRes and Atmos working. I'll try committing Michael's changes to my fork, but I suspect Atmos will stop working.

[smoothquark]

I just updated TidalAPI on my system and now my tidal bash scripts using the API work, but had to reauthenticate. I am using an androidTV client id/secret and it appears that some of them (including mine) still work. When I get more time, I shall try restarting Lyrion and see if my plugin stops working. Even if it continues to work, it would probably stop at some point.

By the way, do you play Atmos on your system? If you don't, then I think my fork could still work. Perhaps try uninstalling my fork, install Michael's (which will then give you the updated client ID/secret), reauthenticate and then try again with my fork? I had a quick look at his commits - it looks like the main changes have been with the id/secret and authentication code and I think once I update my fork, everything should be working again (hopefully...).

[manuelmaniero]

No, I don't use Atmos. My setup consists of a simple PC streamer for a Hi-Fi (2-channel) system.
I'll give it a try - I'll let you know if reinstalling Michael's fork and then reinstalling yours by replacing the files works.

[manuelmaniero]

Good news - I have tried to reinstall Michael's plugin, then refresh the account authentication and restored your fork. It works perfectly!

[smoothquark]

Great. Thank you for the update. I think Michael's plugin has changed the default client id/secret which I think also requires a different methodology for authentication. I shall have a more detailed look when I get the chance, hopefully later this week, as I think my fork will fail soon when the id/secret that I am using inevitably gets unrecognised.

[robo2707]

It just stopped working, we’re waiting for a fix :)

[smoothquark]

I've updated my fork. Works without needing to install Michael's version. However, Atmos is now broken. As expected, Tidal authentication has changed. So, although my Atmos albums are being picked up (using my own client ID/secret) they are not being played as Atmos but as either HiRes stereo or CD quality.
I shall have to spend a bit more time looking to see if I can get Atmos working again, or perhaps going back to the previous version. TBH, I am not sure if I can be bothered with getting Atmos working - most of it is pretty rubbish and gimmicky anyway.
Also, I am getting the odd dropout - not sure if it is my code or my network - they are not repeatable though.

[robo2707]

Yes, I reauthenticated.

[smoothquark]

I just tried a clean install of my fork with the stock client ID/secret and HIRES (FLAC) selected & Country: GB

1. With all 4 options (TIDAL DASH, Prefer HIRES LOSSLESS, Dash stream support and Tidal Dolby Atmos) unchecked, "save settings" and without restarting LMS: only CD versions are listed and play
2. With all 4 options (TIDAL DASH, Prefer HIRES LOSSLESS, Dash stream support and Tidal Dolby Atmos) unchecked, "save settings" and restarting LMS: only CD versions are listed and play
3. With TIDAL DASH checked, Prefer HIRES LOSSLESS, Dash stream support and Tidal Dolby Atmos) unchecked "save settings" and without restarting LMS: TIDAL Max versions are listed and play
4. As No. 3 with all 4 checked and not restarting LMS, Atmos albums are listed but do not play as Atmos. This is expected as specific client id/secrets are required to play Atmos but not necessarily to list them.
5. As No. 4 but with my client ID/secret that was playing Atmos prior to me updating my fork with Michael's code. I shall check this out when I get a bit more time. This is what I expected with the changes in authentication.

[smoothquark]

Another suggestion is to check your cache folder - /var/lib/squeezeboxserver/cache/tmp on my Debian system. This folder gets cleared when LMS restarts but I also have a script which runs once a week to delete the .mpd files in the cache folder.

[robo2707]

I did a fresh installation of Daphile. Tidal works in Hi-Res after your modification. Thanks. I must have had something broken in the system; it was easier to start over than to dig into it.

[smoothquark]

Great that you got it working.

[smoothquark]

As expected, Michael's code forces the client id/secret which he has provided. I have made some changes to the code for custom id/secret to be used to enable Dolby Atmos. So far, has been working fine on my system but I think there may be some issues when the tokens refresh. I shall keep an eye out and make changes as necessary.