Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ability to adjust a mixer's volume (and decouple custom audio stores from global volumes) #6326

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

leocb
Copy link

@leocb leocb commented Jul 5, 2024

Pair of ppy/osu#28733
This change decouples the volume settings from the track/sample store and instead link it with the specific mixer, allowing other mixes to control their own volume.

osu.Framework/Audio/AudioManager.cs Outdated Show resolved Hide resolved
osu.Framework/Audio/Sample/SampleChannel.cs Outdated Show resolved Hide resolved
osu.Framework/Audio/Sample/SampleChannelBass.cs Outdated Show resolved Hide resolved
osu.Framework/Audio/Track/TrackBass.cs Outdated Show resolved Hide resolved
osu.Framework/Audio/Track/TrackBass.cs Outdated Show resolved Hide resolved
@pull-request-size pull-request-size bot added size/S and removed size/M labels Jul 5, 2024
@peppy
Copy link
Member

peppy commented Jul 5, 2024

I've pushed a change with proposed changes as it's easier than trying to drop in a review.

@peppy
Copy link
Member

peppy commented Jul 5, 2024

Note now this is a slightly breaking changeGet*Store calls no longer come with the volume applied. In practice I can't envisage how this could cause breakage, but I'd definitely want a second @ppy/team-client opinion.

@peppy peppy changed the title Add mixer Volume control of all channel it owns Add the ability to adjust a mixer's volume (and decouple custom audio stores from global volumes) Jul 5, 2024
@pull-request-size pull-request-size bot added size/L and removed size/S labels Jul 7, 2024
@leocb
Copy link
Author

leocb commented Jul 7, 2024

I fixed a bug where AudioMixer was being added to the ActiveMixers collection before it was initialized (Audio Thread race condition)
I also improved the Ctrl+F9 Audio mixer visualizer responsiveness and added a peak bar on the meter.
These two are separate commits, please let me know if you'd like for me to open another separate PR for either of them

@peppy
Copy link
Member

peppy commented Jul 8, 2024

I fixed a bug where AudioMixer was being added to the ActiveMixers collection before it was initialized (Audio Thread race condition)

Could you describe how / where this bug was showing up?

@leocb
Copy link
Author

leocb commented Jul 8, 2024

When creating a new mixer, in the debug visualizer, sometimes the mixer output level bar would not show any activity (sound would still play fine).
I believe the visualiser was the only one subscribed to the ActiveMixer's collection changed event

@peppy peppy requested a review from smoogipoo July 10, 2024 05:13
@leocb
Copy link
Author

leocb commented Jul 15, 2024

Hi! I'd like to keep the ball rolling on this. Is there something missing I can help with? Or is it just a matter of waiting a bit more for people to review it?

Copy link
Contributor

@smoogipoo smoogipoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've previously voiced my concerns IRL that I don't have faith in this way of going about things by loosely tying components externally rather than via the well established recursive hierarchy.

But my warning will not be heeded, so I guess we'll have to see how this goes.

Comment on lines 32 to 33
if (channel is IAdjustableAudioComponent adj)
adj.BindAdjustments(this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the Remove() one enqueued to the channel, but this one isn't?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

peppy made this change, by my local testing, it doesn't matter if the adjustment bind is executed inside or outside of the audio thread... I can at least make it consistent across both methods, is it preferable that both are enqueue'd or not?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing with AudioCollectionManager<T>, binding adjustments there is enqueued to the audio thread, so might as well do the same here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, both are now enqueue'd

@bdach
Copy link
Collaborator

bdach commented Jul 16, 2024

that I don't have faith in this way of going about things by loosely tying components externally rather than via the well established recursive hierarchy

By this, you mean the hierarchy of drawables, or something else?

Because if you are referring to the drawable hierarchy I can conjure up several cases wherein tying audio interactions to drawables had undesirable results due to idiosyncrasies of the drawable hierarchy (masking, presence...) causing audio components tied to drawables to behave in an unexpected manner. I generally have lost most of belief that tying audio and drawables is correct.

@smoogipoo
Copy link
Contributor

I mean the hierarchy of audio components, which is admittedly a bit different to the drawable one.

@smoogipoo
Copy link
Contributor

smoogipoo commented Jul 16, 2024

Now that I think about it even more, with the context that this PR doesn't touch the drawable hierarchy at all (DrawableSample et. al), how does a DrawableSample get its volume now? Can that even be described correctly anymore?

Anyone want to take a shot at guessing where its Volume sources are coming from? Especially in the case of a DrawableMixer (which is an AudioContainer so it can have DrawableSample added to it)?

@bdach
Copy link
Collaborator

bdach commented Jul 16, 2024

To be truthful I have personally not been wanting to look at this PR as I kinda don't like the game-side feature that this is attempting to facilitate, I think it's not necessary / incurs too much complexity. If I'm alone on this / there's a consensus that we want that feature in some form, then I might try reading through this, but otherwise I kinda personally don't really care, am fine ignoring this, and would like to spend my time elsewhere.

@leocb
Copy link
Author

leocb commented Jul 16, 2024

Thank you all for your comments. I believe I understand well enough of how the audio stack works, so here's my personal response to your concerns (please correct me if I'm wrong):

how does a DrawableSample get its volume now?

@smoogipoo By default, the volume of everything is 1. Previous to this PR, an adjustment was added to every track/sample, to respond to the music/effects settings volume sliders
After this PR, since everything already goes to a mixer, only one adjustment is needed per volume setting, one at the mixer level.
An adjustment can still be applied over the individual sample on a per-need basis, but AFAIK this is not done anywhere
From an audio mixing perspective, I'd say this mixer volume behavior is expected and makes more sense than binding the settings volume sliders to every individual track/sample

Anyone want to take a shot at guessing where its Volume sources are coming from? Especially in the case of a DrawableMixer (which is an AudioContainer so it can have DrawableSample added to it)?

In the case of DrawableMixer, a new mixer is created to control the volume of the samples played inside of it.
Then, the DrawableSample checks if any parent has a mixer present, if it has, the sample will bind to that mixer, with a fallback to the default sample mixer - this already exists in the code.
The volume adjustment of the container mixer is always present (defaults to 1), but not initially bound to anything, in the Osu game PR related to this one, I bind it to a new audio setting slider called "Gameplay" (name still pending)


I think it's not necessary / incurs too much complexity.

@bdach I believe this PR makes the whole system more flexible, meaning new mixers can be used whenever necessary in the game repository and be effective.

If I'm alone on this / there's a consensus that we want that feature in some form, then I might try reading through this

This is desired by some that are annoyed by the hitsounds but still want to hear the new UI sfx, myself included. This discussion and 1, 2 forum posts pushed me over the edge to open this PR
Also, the effective part of this PR that changes the sample/mixer volume behavior is small, most changes of this PR are improvements to the audio mixer debug screen, which I can move to another PR if you want me to.

@peppy
Copy link
Member

peppy commented Jul 17, 2024

Hit sound volume adjust is in my eyes one of the top 20 feature requests from the game over the years. The most common request is "I want to hear UI sounds but not hitsounds" which is something I personally disagree with, but the voice for this is present so I do want to provide it as an option.

@frenzibyte
Copy link
Member

frenzibyte commented Jul 21, 2024

how does a DrawableSample get its volume now?

By default, the volume of everything is 1. Previous to this PR, an adjustment was added to every track/sample, to respond to the music/effects settings volume sliders After this PR, since everything already goes to a mixer, only one adjustment is needed per volume setting, one at the mixer level. An adjustment can still be applied over the individual sample on a per-need basis, but AFAIK this is not done anywhere From an audio mixing perspective, I'd say this mixer volume behavior is expected and makes more sense than binding the settings volume sliders to every individual track/sample

Anyone want to take a shot at guessing where its Volume sources are coming from? Especially in the case of a DrawableMixer (which is an AudioContainer so it can have DrawableSample added to it)?

In the case of DrawableMixer, a new mixer is created to control the volume of the samples played inside of it. Then, the DrawableSample checks if any parent has a mixer present, if it has, the sample will bind to that mixer, with a fallback to the default sample mixer - this already exists in the code. The volume adjustment of the container mixer is always present (defaults to 1), but not initially bound to anything, in the Osu game PR related to this one, I bind it to a new audio setting slider called "Gameplay" (name still pending)

At face value, I'm seeing that the audio adjustments have transferred from the stores to the mixers, and that the mixers only accept Tracks and SampleChannels to bind its adjustments to it. Therefore, a behavioural change has technically arised wherein Samples/DrawableSamples no longer apply the mixer-level adjustments to its Aggregate components. The channels inside have adjustments applied, but Samples don't.

I'm not sure if there are issues beyond that the Aggregate components don't read correctly, but it's still something.

I personally feel that AudioMixer should inherit from AudioCollectionManagers and follow the usual recursive pattern (as I presume is what smoogi meant), accepting all kinds of AudioComponents and applying mixer properties to IAudioChannels specifically. However, I'm not sure how easy it is to make that happen, and whether it's gonna make the code feel any better than how it works right now.

@smoogipoo
Copy link
Contributor

smoogipoo commented Jul 22, 2024

image

Does this help explaining why I'm confused? I don't even know if this is the full view of things.

What I mean is not "how does it get its volume" - I can answer that myself because I wrote half this thing, my concern is that it's difficult to follow and now (I believe) there are multiple branches because this isn't using the already tried and tested hierarchical relationship of components. It's injecting the mixer's volume directly into components willy-nilly.

What happens when we support nested mixers? I think it'll look something like this:

image

Unless I'm missing something.

@leocb
Copy link
Author

leocb commented Jul 22, 2024

@smoogipoo I believe thinking about it in a more traditional sound mixing routing way should help. I will use "" to help me here.
This is what I was thinking about when I opened this PR:
image
What we call mixers is now designed to work as the traditional "sub-group", where we can adjust the volume, eq, balance, etc, of some set "channels" before the sound routing output goes to the master.
Where before adjusting the setting volumes would "reduce the volume fader" of each individual "channel", now the settings will just lower the "sub-group volume fader".
In the case of nested mixers, the output routing of a "sub-group" will be simply go to another "sub-group", while not common, it's perfectly reasonable to exist.
Instead of branches, try to think about it as funnels.

@smoogipoo
Copy link
Contributor

The theory is fine, but that's not how o!f works to my understanding. Am I to understand that the diagram I've put above is not how o!f currently (not in the most perfect scenario) interacts as a result of this PR? I haven't tested it so it's possible I'm totally wrong.

In my scenario, it means if you have a parent DrawableAudioMixer, and set its volume to 0.5, the SampleChannel playing will have volume = 0.25. I will test and get back to you on that.

I don't disagree with the perfect scenario of a traditional DAW, but for that to work in o!f, things would need to be redesigned. The current structure is more like a DAW where everything's recursive.

@leocb
Copy link
Author

leocb commented Jul 22, 2024

In your diagram, Is the Sample Volume on the left, the volume in the settings? If so, then that connection with the sample no longer exists in this PR (removed lines 186 and 194 on Audio/AudioManager.cs)
meaning the sample volume setting is not applied twice to the sample (50% in the settings would not play the sample at 0.25)

@smoogipoo
Copy link
Contributor

smoogipoo commented Jul 22, 2024

There's actually two issues here...

  1. DrawableAudioMixer doesn't bind its adjustments to the underlying mixer.
diff --git a/osu.Framework/Graphics/Audio/DrawableAudioMixer.cs b/osu.Framework/Graphics/Audio/DrawableAudioMixer.cs
index 4e3c696ea..d558f73bb 100644
--- a/osu.Framework/Graphics/Audio/DrawableAudioMixer.cs
+++ b/osu.Framework/Graphics/Audio/DrawableAudioMixer.cs
@@ -20,6 +20,7 @@ public partial class DrawableAudioMixer : AudioContainer, IAudioMixer
         private void load(AudioManager audio)
         {
             mixer = audio.CreateAudioMixer(Name);
+            mixer.BindAdjustments(this);
         }

         public void Add(IAudioChannel channel)
  1. With the above fix, this test demonstrates my issue (in osu!framework):
using NUnit.Framework;
using osu.Framework.Allocation;
using osu.Framework.Audio.Sample;
using osu.Framework.Graphics.Audio;
using osu.Framework.Tests.Visual;

namespace osu.Framework.Tests
{
    public partial class TestSceneTesting : FrameworkTestScene
    {
        private DrawableSample sample = null!;

        [BackgroundDependencyLoader]
        private void load(ISampleStore samples)
        {
            Add(new DrawableAudioMixer
            {
                Volume = { Value = 0.5f },
                Child = sample = new DrawableSample(samples.Get("long.mp3"))
            });
        }

        [Test]
        public void RunTest()
        {
            SampleChannel channel = null!;
            AddStep("play channel", () =>
            {
                channel = sample.GetChannel();
                channel.Looping = true;
                channel.Play();
            });

            AddAssert("sample volume is 0.5", () => sample.AggregateVolume.Value, () => Is.EqualTo(0.5));
            AddAssert("channel volume is 0.5", () => channel.AggregateVolume.Value, () => Is.EqualTo(0.5));
        }
    }
}

That second assertion should pass, but the volume is actually 0.25 as I originally stated.

@smoogipoo
Copy link
Contributor

smoogipoo commented Jul 22, 2024

In your diagram, Is the Sample Volume on the left, the volume in the settings?

No, that's the Volume property on a Sample class (SampleBass, SampleVirtual, etc).

@peppy peppy self-requested a review July 22, 2024 04:17
@peppy
Copy link
Member

peppy commented Jul 22, 2024

Working through this with @smoogipoo, here's a partial diff of a direction that might work better, but it's incomplete and by no means guaranteed to be correct / usable. Posting for further consideration

diff --git a/osu.Framework.Tests/Audio/BassTestComponents.cs b/osu.Framework.Tests/Audio/BassTestComponents.cs
index 416b0ccbf..ea0387e07 100644
--- a/osu.Framework.Tests/Audio/BassTestComponents.cs
+++ b/osu.Framework.Tests/Audio/BassTestComponents.cs
@@ -34,8 +34,10 @@ public BassTestComponents(bool init = true)
 
             Mixer = CreateMixer();
             Resources = new DllResourceStore(typeof(TrackBassTest).Assembly);
-            TrackStore = new TrackStore(Resources, Mixer);
-            SampleStore = new SampleStore(Resources, Mixer);
+            TrackStore = new TrackStore(Resources);
+            SampleStore = new SampleStore(Resources);
+            Mixer.AddItem(TrackStore);
+            Mixer.AddItem(SampleStore);
 
             Add(TrackStore, SampleStore);
         }
diff --git a/osu.Framework/Audio/AudioManager.cs b/osu.Framework/Audio/AudioManager.cs
index 89e27b862..d87764f82 100644
--- a/osu.Framework/Audio/AudioManager.cs
+++ b/osu.Framework/Audio/AudioManager.cs
@@ -184,15 +184,15 @@ public AudioManager(AudioThread audioThread, ResourceStore<byte[]> trackStore, R
 
             globalTrackStore = new Lazy<TrackStore>(() =>
             {
-                var store = new TrackStore(trackStore, TrackMixer);
-                AddItem(store);
+                var store = new TrackStore(trackStore);
+                TrackMixer.AddItem(store);
                 return store;
             });
 
             globalSampleStore = new Lazy<SampleStore>(() =>
             {
-                var store = new SampleStore(sampleStore, SampleMixer);
-                AddItem(store);
+                var store = new SampleStore(sampleStore);
+                SampleMixer.AddItem(store);
                 return store;
             });
 
@@ -293,8 +293,8 @@ public ITrackStore GetTrackStore(IResourceStore<byte[]> store = null, AudioMixer
         {
             if (store == null) return globalTrackStore.Value;
 
-            TrackStore tm = new TrackStore(store, mixer ?? TrackMixer);
-            globalTrackStore.Value.AddItem(tm);
+            TrackStore tm = new TrackStore(store);
+            (mixer ?? TrackMixer).AddItem(tm);
             return tm;
         }
 
@@ -313,8 +313,8 @@ public ISampleStore GetSampleStore(IResourceStore<byte[]> store = null, AudioMix
         {
             if (store == null) return globalSampleStore.Value;
 
-            SampleStore sm = new SampleStore(store, mixer ?? SampleMixer);
-            globalSampleStore.Value.AddItem(sm);
+            SampleStore sm = new SampleStore(store);
+            (mixer ?? SampleMixer).AddItem(sm);
             return sm;
         }
 
diff --git a/osu.Framework/Audio/Mixing/AudioMixer.cs b/osu.Framework/Audio/Mixing/AudioMixer.cs
index da8a3c765..8ae388453 100644
--- a/osu.Framework/Audio/Mixing/AudioMixer.cs
+++ b/osu.Framework/Audio/Mixing/AudioMixer.cs
@@ -9,7 +9,7 @@ namespace osu.Framework.Audio.Mixing
     /// <summary>
     /// Mixes together multiple <see cref="IAudioChannel"/>s into one output.
     /// </summary>
-    public abstract class AudioMixer : AdjustableAudioComponent, IAudioMixer
+    public abstract class AudioMixer : AudioCollectionManager<AudioComponent>, IAudioMixer
     {
         public readonly string Identifier;
 
@@ -27,9 +27,11 @@ protected AudioMixer(AudioMixer? fallbackMixer, string identifier)
             Identifier = identifier;
         }
 
-        public void Add(IAudioChannel channel)
+        protected override void ItemAdded(AudioComponent item)
         {
-            channel.EnqueueAction(() =>
+            base.ItemAdded(item);
+
+            if (item is IAudioChannel channel)
             {
                 if (channel.Mixer == this)
                     return;
@@ -43,10 +45,15 @@ public void Add(IAudioChannel channel)
                 AddInternal(channel);
 
                 channel.Mixer = this;
-            });
+            }
         }
 
-        public void Remove(IAudioChannel channel) => Remove(channel, true);
+        protected override void ItemRemoved(AudioComponent item)
+        {
+            base.ItemRemoved(item);
+            if (item is IAudioChannel channel)
+                Remove(channel, true);
+        }
 
         public abstract void AddEffect(IEffectParameter effect, int priority = 0);
 
@@ -77,8 +84,8 @@ protected void Remove(IAudioChannel channel, bool returnToDefault)
                     adj.UnbindAdjustments(this);
 
                 // Add the channel back to the default mixer so audio can always be played.
-                if (returnToDefault)
-                    fallbackMixer.AsNonNull().Add(channel);
+                if (returnToDefault && channel is AudioComponent comp)
+                    fallbackMixer.AsNonNull().AddItem(comp);
             });
         }
 
diff --git a/osu.Framework/Audio/Sample/SampleBass.cs b/osu.Framework/Audio/Sample/SampleBass.cs
index 5052a2cd5..9487b6270 100644
--- a/osu.Framework/Audio/Sample/SampleBass.cs
+++ b/osu.Framework/Audio/Sample/SampleBass.cs
@@ -1,8 +1,6 @@
 // Copyright (c) ppy Pty Ltd <[email protected]>. Licensed under the MIT Licence.
 // See the LICENCE file in the repository root for full licence text.
 
-using osu.Framework.Audio.Mixing.Bass;
-
 namespace osu.Framework.Audio.Sample
 {
     internal sealed class SampleBass : Sample
@@ -12,13 +10,11 @@ internal sealed class SampleBass : Sample
         public override bool IsLoaded => factory.IsLoaded;
 
         private readonly SampleBassFactory factory;
-        private readonly BassAudioMixer mixer;
 
-        internal SampleBass(SampleBassFactory factory, BassAudioMixer mixer)
+        internal SampleBass(SampleBassFactory factory)
             : base(factory.Name)
         {
             this.factory = factory;
-            this.mixer = mixer;
 
             PlaybackConcurrency.BindTo(factory.PlaybackConcurrency);
         }
@@ -26,7 +22,7 @@ internal SampleBass(SampleBassFactory factory, BassAudioMixer mixer)
         protected override SampleChannel CreateChannel()
         {
             var channel = new SampleChannelBass(this);
-            mixer.Add(channel);
+            AddItem(channel);
             return channel;
         }
     }
diff --git a/osu.Framework/Audio/Sample/SampleBassFactory.cs b/osu.Framework/Audio/Sample/SampleBassFactory.cs
index 3b863a68b..695975c49 100644
--- a/osu.Framework/Audio/Sample/SampleBassFactory.cs
+++ b/osu.Framework/Audio/Sample/SampleBassFactory.cs
@@ -5,7 +5,6 @@
 using System.Runtime.InteropServices;
 using ManagedBass;
 using osu.Framework.Allocation;
-using osu.Framework.Audio.Mixing.Bass;
 using osu.Framework.Bindables;
 using osu.Framework.Platform;
 
@@ -32,15 +31,12 @@ internal class SampleBassFactory : AudioCollectionManager<AdjustableAudioCompone
         /// </summary>
         internal readonly Bindable<int> PlaybackConcurrency = new Bindable<int>(Sample.DEFAULT_CONCURRENCY);
 
-        private readonly BassAudioMixer mixer;
-
         private NativeMemoryTracker.NativeMemoryLease? memoryLease;
         private byte[]? data;
 
-        public SampleBassFactory(byte[] data, string name, BassAudioMixer mixer)
+        public SampleBassFactory(byte[] data, string name)
         {
             this.data = data;
-            this.mixer = mixer;
 
             Name = name;
 
@@ -97,7 +93,7 @@ private void loadSample()
             memoryLease = NativeMemoryTracker.AddMemory(this, dataLength);
         }
 
-        public Sample CreateSample() => new SampleBass(this, mixer) { OnPlay = onPlay };
+        public Sample CreateSample() => new SampleBass(this) { OnPlay = onPlay };
 
         private void onPlay(Sample sample)
         {
diff --git a/osu.Framework/Audio/Sample/SampleStore.cs b/osu.Framework/Audio/Sample/SampleStore.cs
index a47a9f15a..298fd03bc 100644
--- a/osu.Framework/Audio/Sample/SampleStore.cs
+++ b/osu.Framework/Audio/Sample/SampleStore.cs
@@ -9,8 +9,6 @@
 using System.Threading;
 using System.Threading.Tasks;
 using JetBrains.Annotations;
-using osu.Framework.Audio.Mixing;
-using osu.Framework.Audio.Mixing.Bass;
 using osu.Framework.IO.Stores;
 using osu.Framework.Statistics;
 
@@ -19,16 +17,14 @@ namespace osu.Framework.Audio.Sample
     internal class SampleStore : AudioCollectionManager<AdjustableAudioComponent>, ISampleStore
     {
         private readonly ResourceStore<byte[]> store;
-        private readonly AudioMixer mixer;
 
         private readonly Dictionary<string, SampleBassFactory> factories = new Dictionary<string, SampleBassFactory>();
 
         public int PlaybackConcurrency { get; set; } = Sample.DEFAULT_CONCURRENCY;
 
-        internal SampleStore([NotNull] IResourceStore<byte[]> store, [NotNull] AudioMixer mixer)
+        internal SampleStore([NotNull] IResourceStore<byte[]> store)
         {
             this.store = new ResourceStore<byte[]>(store);
-            this.mixer = mixer;
 
             AddExtension(@"wav");
             AddExtension(@"mp3");
@@ -49,7 +45,7 @@ public Sample Get(string name)
                     this.LogIfNonBackgroundThread(name);
 
                     byte[] data = store.Get(name);
-                    factory = factories[name] = data == null ? null : new SampleBassFactory(data, name, (BassAudioMixer)mixer) { PlaybackConcurrency = { Value = PlaybackConcurrency } };
+                    factory = factories[name] = data == null ? null : new SampleBassFactory(data, name) { PlaybackConcurrency = { Value = PlaybackConcurrency } };
 
                     if (factory != null)
                         AddItem(factory);
diff --git a/osu.Framework/Audio/Track/TrackStore.cs b/osu.Framework/Audio/Track/TrackStore.cs
index 495e3f071..6666c6e9d 100644
--- a/osu.Framework/Audio/Track/TrackStore.cs
+++ b/osu.Framework/Audio/Track/TrackStore.cs
@@ -9,7 +9,6 @@
 using System.Threading;
 using System.Threading.Tasks;
 using JetBrains.Annotations;
-using osu.Framework.Audio.Mixing;
 using osu.Framework.IO.Stores;
 
 namespace osu.Framework.Audio.Track
@@ -17,12 +16,10 @@ namespace osu.Framework.Audio.Track
     internal class TrackStore : AudioCollectionManager<AdjustableAudioComponent>, ITrackStore
     {
         private readonly IResourceStore<byte[]> store;
-        private readonly AudioMixer mixer;
 
-        internal TrackStore([NotNull] IResourceStore<byte[]> store, [NotNull] AudioMixer mixer)
+        internal TrackStore([NotNull] IResourceStore<byte[]> store)
         {
             this.store = store;
-            this.mixer = mixer;
 
             (store as ResourceStore<byte[]>)?.AddExtension(@"mp3");
         }
@@ -49,7 +46,6 @@ public Track Get(string name)
 
             TrackBass trackBass = new TrackBass(dataStream, name);
 
-            mixer.Add(trackBass);
             AddItem(trackBass);
 
             return trackBass;

@smoogipoo
Copy link
Contributor

smoogipoo commented Jul 22, 2024

I think I'd be alright with this PR if we removed DrawableAudioMixer completely, with the future goal of removing the Drawable audio components entirely. I don't think the two hierarchies can get involved with one another, and I think the above diff is going in a direction we need to go, but I don't have a full clear view of how things should look.

We were discussing just using BDL for audio, however manual that is...

[Resolved]
private IAudioMixer mixer { get; set; } = null!;

[Resolved]
private ISampleStore samples { get; set; } = null!;

[Resolved]
private AudioManager audio { get; set; } = null!;

private void doSomething()
{
    // Play a sample...
    var sample = samples.Get(...);
    mixer.Add(sample);
    sample.GetChannel().Play();

   // Create a mixer...
   var ourMixer = audio.CreateMixer();
   mixer.Add(ourMixer); // Nested mixers, or you can play samples on ourMixer directly...
   // We can cache the mixer and use BDL to handle overrides.

   // How do we handle the track vs sample mixer with this structure? We'll need to figure that one out!
}

Maybe.... Or perhaps something like x.Play(mixer);, again, not 100% sure on the structure here but the tl;dr is to make it more manual.

The other way is to remove the audio hierarchy and force using the drawable components, which was the path I first thought of. My reasoning being is historically we've had issues in osu!stable where samples were kept active in the background because some component somewhere didn't stop a sample, and having them be automatically managed might resolve some of those issues (though it's also been brought up that drawable disposal is async sooooooo...).

@leocb
Copy link
Author

leocb commented Aug 7, 2024

Hello everyone! Messing with the Audio stack is beyond what I can do and I believe it's better left to you, more experienced devs.

That said, I really want to have the hitsound mute/volume feature merged.

I want to kindly ask the reviewers if any of the following options would be acceptable right now:

  1. I could revert the osu!game PR back to being a simple mute toggle (No core changes needed), we could always add a slider to it later when this audio mixer stuff is more mature.
  2. Accept this PR as it is now and make these changes you're talking about in a later PR, since, as of this moment, hitsound volume is only instance where an additional mixer is used anyway. From my individual playtesting, this PR works and nothing is broken. Sure it may not be ideal, but it does indeed work.

As always, thank you!

@peppy
Copy link
Member

peppy commented Aug 8, 2024

That said, I really want to have the hitsound mute/volume feature merged.

Please be patient and wait for an outcome. While you may want this, it's just a drop in the bucket of things we're aiming to get done. I don't believe we need to take shortcuts or make temporary hacks to expedite this feature as people have lived without it for decades.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants