diff --git a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita index dcd6e8897f6..50656b22c88 100644 --- a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita @@ -49,7 +49,7 @@
  • 视频直播：扬声器

    • 调用该 API 可以改变上述默认音频路由。成功改变音频路由后，SDK 会触发 回调。

    - +

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。
    diff --git a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita index 574cc87f529..5acc058e351 100644 --- a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita @@ -24,7 +24,7 @@ Future<void> setEnableSpeakerphone(bool speakerOn);

    详情 -

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    +

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    该方法只设置用户在当前频道内使用的音频路由，不会影响 SDK 默认的音频路由。如果用户离开当前频道并加入新的频道，则用户还是会使用 SDK 默认的音频路由。

      diff --git a/dita/RTC-NG/RTC_NG_API_Unity.ditamap b/dita/RTC-NG/RTC_NG_API_Unity.ditamap index 2d5a04765d7..87ee27dea8b 100644 --- a/dita/RTC-NG/RTC_NG_API_Unity.ditamap +++ b/dita/RTC-NG/RTC_NG_API_Unity.ditamap @@ -686,6 +686,7 @@ + diff --git a/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap b/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap index d86579f0f65..15ca8bce264 100644 --- a/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap +++ b/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap @@ -4,7 +4,7 @@ Keys RTC NG Windows Links - + diff --git a/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap b/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap index 363e74df5c9..1989a6ab4dd 100644 --- a/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap +++ b/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap @@ -4,6 +4,7 @@ Keys RTC NG Unity Links + diff --git a/dita/RTC-NG/release/release_notes.dita b/dita/RTC-NG/release/release_notes.dita index 584e5c9439c..e520a76decd 100644 --- a/dita/RTC-NG/release/release_notes.dita +++ b/dita/RTC-NG/release/release_notes.dita @@ -99,8 +99,10 @@
    • 摄像头采集改进

      优化了摄像头采集的视频处理机制，减少噪点、提高亮度、改善色彩，使采集到的图形更加清晰、明亮和真实。

      • -
    • 获取音频设备类型 -

      该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

      +
    • 获取音频设备类型（macOS） +

      该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

      +

      该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

      +

      该版本新增 、、、 方法，分别用于获取音频播放、采集设备的设备信息及类型。

    • 虚拟背景算法优化

      为了提高纯色背景下开启虚拟背景时人像分割的精确度和稳定性，该版本优化了绿幕分割算法： @@ -181,10 +183,11 @@

    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • 中新增 deviceTypeName (macOS)
  • 新增 deviceTypeName
  • 新增 e2eDelay
  • 新增
    • diff --git a/en-US/dita/RTC-NG/release/release_notes.dita b/en-US/dita/RTC-NG/release/release_notes.dita index eabf34e749a..451f0cbd8ab 100644 --- a/en-US/dita/RTC-NG/release/release_notes.dita +++ b/en-US/dita/RTC-NG/release/release_notes.dita @@ -108,8 +108,10 @@
  • Camera capture improvements

    Improvements have been made to the video processing mechanism of camera capture, reducing noise, enhancing brightness, and improving color, making the captured images clearer, brighter, and more realistic.

    • -
  • Audio device type detection -

    This version adds the deviceTypeName member to , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +
  • Audio device type detection (macOS) +

    This version adds the deviceTypeName member to , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +

    This version adds the deviceTypeName in , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +

    This version adds the , , , and method to obtain the information and type of audio playback and recording deivces.

  • Virtual Background Algorithm Optimization

    To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: @@ -195,10 +197,11 @@

    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • adds deviceTypeName (macOS)
  • adds deviceTypeName
  • adds e2eDelay
  • adds
    • diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md index bfaf73413fc..0eb5fc8865e 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md @@ -1,3 +1,108 @@ +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`registerFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`onFaceInfo`](/api-ref/rtc/electron/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Portrait center stage (macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +3. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/electron/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`enableEncryption`](/api-ref/rtc/electron/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `EncryptionErrorDatastreamDecryptionFailure` and `EncryptionErrorDatastreamEncryptionFailure` enumerations. + +4. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`setDualStreamMode`](/api-ref/rtc/electron/API/toc_dual_stream#api_irtcengine_setdualstreammode2), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +5. **Other features** + + - New method [`enableEncryptionEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/electron/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`getCallIdEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the `deviceTypeName` in [`AudioDeviceInfo`](/api-ref/rtc/electron/API/class_audiodeviceinfo), used to identify the type of audio devices, such as built-in, USB, HDMI, etc. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EarMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (macOS, Windows) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AudioEffectPreset`](/api-ref/rtc/electron/API/enum_audioeffectpreset), a new enumeration `RoomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/electron/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [`enableLoopbackRecording`](/api-ref/rtc/electron/API/toc_audio_capture#api_irtcengine_enableloopbackrecording) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- In audio-video interactions, if a user inserted headphones into the device and manually switched the system audio output to speakers, and later the app called [`setPlaybackDevice`](/api-ref/rtc/electron/API/toc_audio_device#api_iaudiodevicemanager_setplaybackdevice) to specify the headphone as the audio playback device, audio output did not switch back to speakers as expected after the headphones were removed. (macOS) + +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- On devices using Intel graphics cards, occasionally there was a performance regression when publishing a small video stream. (Windows) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. + +#### API Changes + +**Added** + +- [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (macOS) +- [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (macOS) +- [`registerFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`unregisterFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/electron/API/class_ifaceinfoobserver) +- [`onFaceInfo`](/api-ref/rtc/electron/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MediaSourceType`](/api-ref/rtc/electron/API/enum_mediasourcetype) adds `SpeechDrivenVideoSource` +- [`VideoSourceType`](/api-ref/rtc/electron/API/enum_videosourcetype) adds `VideoSourceSpeechDriven` +- [`EncryptionConfig`](/api-ref/rtc/electron/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`EncryptionErrorType`](/api-ref/rtc/electron/API/enum_encryptionerrortype) adds the following enumerations: + - `EncryptionErrorDatastreamDecryptionFailure` + - `EncryptionErrorDatastreamEncryptionFailure` +- [`AudioDeviceInfo`](/api-ref/rtc/electron/API/class_audiodeviceinfo) adds `deviceTypeName` (macOS) +- [`RemoteAudioStats`](/api-ref/rtc/electron/API/class_remoteaudiostats) adds `e2eDelay` +- [`ErrorCodeType`](/api-ref/rtc/electron/API/enum_errorcodetype) adds `ErrDatastreamDecryptionFailed` +- [`AudioEffectPreset`](/api-ref/rtc/electron/API/enum_audioeffectpreset) adds `RoomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [`getCallIdEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_getcallidex) +- [`enableEncryptionEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_enableencryptionex) +- [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/electron/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`EarMonitoringFilterType`](/api-ref/rtc/electron/API/enum_earmonitoringfiltertype) adds a new enumeration `EarMonitoringFilterBuiltInAudioFilters`(1 << 15) + ## v4.3.0 v4.3.0 was released on xx xx, 2024. diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md index b3139796757..c2593b99d6c 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md @@ -9,6 +9,148 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v6.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [registerFaceInfoObserver](API/api_imediaengine_registerfaceinfoobserver.html) method and [onFaceInfo](API/callback_ifaceinfoobserver_onfaceinfo.html) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Portrait center stage (iOS, macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [setCameraStabilizationMode](API/api_irtcengine_setcamerastabilizationmode.html), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [queryCameraFocalLengthCapability](API/api_irtcengine_querycamerafocallengthcapability.html) to check the device's focal length capabilities, and then call [setCameraCapturerConfiguration](API/api_irtcengine_setcameracapturerconfiguration.html) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `videoSourceCameraThird` (11) and `videoSourceCameraFourth` (12) in [VideoSourceType](API/enum_videosourcetype.html) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [startCameraCapture](API/api_irtcengine_startcameracapture.html). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [ChannelMediaOptions](API/class_channelmediaoptions.html) add support for Android. Set these parameters to `true` when joining a channel with [joinChannel](API/api_irtcengine_joinchannel2.html) to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [startCameraCapture](API/api_irtcengine_startcameracapture.html) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [EncryptionConfig](API/class_encryptionconfig.html) for enabling data stream encryption. You can set this when you activate encryption with [enableEncryption](API/api_irtcengine_enableencryption.html). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `encryptionErrorDatastreamDecryptionFailure` and `encryptionErrorDatastreamEncryptionFailure`enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [setDualStreamMode](API/api_irtcengine_setdualstreammode2.html), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [enableEncryptionEx](API/api_irtcengineex_enableencryptionex.html) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [setAudioMixingPlaybackSpeed](API/api_irtcengine_setaudiomixingplaybackspeed.html) is introduced for setting the playback speed of audio files. + - New method [getCallIdEx](API/api_irtcengineex_getcallidex.html) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the `deviceTypeName` in [AudioDeviceInfo](API/class_audiodeviceinfo.html), used to identify the type of audio devices, such as built-in, USB, HDMI, etc. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `earMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (iOS, macOS, Windows) + - Enhanced performance and stability of the local compositing feature, reducing its CPU usage. (Android) + - Improved stability in processing video by the raw video frame observer. (iOS) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [AudioEffectPreset](API/enum_audioeffectpreset.html), a new enumeration `roomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [RemoteAudioStats](API/class_remoteaudiostats.html), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [enableLoopbackRecording](API/api_irtcengine_enableloopbackrecording.html) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- In scenarios using camera capture for local video, when the app was moved to the background and [disableVideo](API/api_irtcengine_disablevideo.html) or stopPreview was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. + +#### API Changes + +**Added** + +- [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html) (iOS, macOS) +- [isCameraCenterStageSupported](API/api_irtcengine_iscameracenterstagesupported.html) (iOS, macOS) +- [setCameraStabilizationMode](API/api_irtcengine_setcamerastabilizationmode.html) (iOS) +- [CameraStabilizationMode](API/enum_camerastabilizationmode.html) (iOS) +- [registerFaceInfoObserver](API/api_imediaengine_registerfaceinfoobserver.html) +- [unregisterFaceInfoObserver](API/api_imediaengine_unregisterfaceinfoobserver.html) +- [FaceInfoObserver](API/class_ifaceinfoobserver.html) +- [onFaceInfo](API/callback_ifaceinfoobserver_onfaceinfo.html) +- [MediaSourceType](API/enum_mediasourcetype.html) adds `speechDrivenVideoSource` +- [VideoSourceType](API/enum_videosourcetype.html) adds `videoSourceSpeechDriven` +- [EncryptionConfig](API/class_encryptionconfig.html) adds `datastreamEncryptionEnabled` +- [EncryptionErrorType](API/enum_encryptionerrortype.html) adds the following enumerations: + - `encryptionErrorDatastreamDecryptionFailure` + - `encryptionErrorDatastreamEncryptionFailure` +- [AudioDeviceInfo](API/class_audiodeviceinfo.html) adds `deviceTypeName` (macOS) +- [RemoteAudioStats](API/class_remoteaudiostats.html) adds `e2eDelay` +- [ErrorCodeType](API/enum_errorcodetype.html) adds `errDatastreamDecryptionFailed` +- [AudioEffectPreset](API/enum_audioeffectpreset.html) adds `roomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [getCallIdEx](API/api_irtcengineex_getcallidex.html) +- [enableEncryptionEx](API/api_irtcengineex_enableencryptionex.html) +- [setAudioMixingPlaybackSpeed](API/api_irtcengine_setaudiomixingplaybackspeed.html) +- [queryCameraFocalLengthCapability](API/api_irtcengine_querycamerafocallengthcapability.html) (Android, iOS) +- [FocalLengthInfo](API/class_focallengthinfo.html) (Android, iOS) +- [CameraFocalLengthType](API/enum_camerafocallengthtype.html) (Android, iOS) +- [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraFocalLengthType` (Android, iOS) +- [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraId` (Android) +- [EarMonitoringFilterType](API/enum_earmonitoringfiltertype.html) adds a new enumeration `earMonitoringFilterBuiltInAudioFilters`(1 << 15) + + ## v6.3.0 v6.3.0 was released on xx xx, 2024. diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md index 74a8334fa0d..ffc94d40d15 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md @@ -8,6 +8,135 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`registerFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`onFaceInfo`](/api-ref/rtc/rn/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Portrait center stage (iOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [`setCameraStabilizationMode`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [`queryCameraFocalLengthCapability`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) to check the device's focal length capabilities, and then call [`setCameraCapturerConfiguration`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_setcameracapturerconfiguration) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `VideoSourceCameraThird` (11) and `VideoSourceCameraFourth` (12) in [`VideoSourceType`](/api-ref/rtc/rn/API/enum_videosourcetype) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [`startCameraCapture`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_startcameracapture). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [`ChannelMediaOptions`](/api-ref/rtc/rn/API/class_channelmediaoptions) add support for Android. Set these parameters to `true` when joining a channel with [`joinChannel`](/api-ref/rtc/rn/API/toc_channel#api_irtcengine_joinchannel2) to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [`startCameraCapture`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_startcameracapture) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/rn/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`enableEncryption`](/api-ref/rtc/rn/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `EncryptionErrorDatastreamDecryptionFailure` and `EncryptionErrorDatastreamEncryptionFailure` enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`setDualStreamMode`](/api-ref/rtc/rn/API/toc_dual_stream#api_irtcengine_setdualstreammode2), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [`enableEncryptionEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/rn/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`getCallIdEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +2. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EarMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +3. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (iOS) + - Enhanced performance and stability of the local compositing feature, reducing its CPU usage. (Android) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AudioEffectPreset`](/api-ref/rtc/rn/API/enum_audioeffectpreset), a new enumeration `RoomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/rn/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- In scenarios using camera capture for local video, when the app was moved to the background and [`disableVideo`](/api-ref/rtc/rn/API/toc_video_basic#api_irtcengine_disablevideo) or was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. + +#### API Changes + +**Added** + +- [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (iOS) +- [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (iOS) +- [`setCameraStabilizationMode`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode) (iOS) +- [`CameraStabilizationMode`](/api-ref/rtc/rn/API/enum_camerastabilizationmode) (iOS) +- [`registerFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`unregisterFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/rn/API/class_ifaceinfoobserver) +- [`onFaceInfo`](/api-ref/rtc/rn/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MediaSourceType`](/api-ref/rtc/rn/API/enum_mediasourcetype) adds `SpeechDrivenVideoSource` +- [`VideoSourceType`](/api-ref/rtc/rn/API/enum_videosourcetype) adds `VideoSourceSpeechDriven` +- [`EncryptionConfig`](/api-ref/rtc/rn/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`EncryptionErrorType`](/api-ref/rtc/rn/API/enum_encryptionerrortype) adds the following enumerations: + - `EncryptionErrorDatastreamDecryptionFailure` + - `EncryptionErrorDatastreamEncryptionFailure` +- [`RemoteAudioStats`](/api-ref/rtc/rn/API/class_remoteaudiostats) adds `e2eDelay` +- [`ErrorCodeType`](/api-ref/rtc/rn/API/enum_errorcodetype) adds `ErrDatastreamDecryptionFailed` +- [`AudioEffectPreset`](/api-ref/rtc/rn/API/enum_audioeffectpreset) adds `RoomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [`getCallIdEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_getcallidex) +- [`enableEncryptionEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_enableencryptionex) +- [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/rn/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`queryCameraFocalLengthCapability`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) (Android, iOS) +- [`FocalLengthInfo`](/api-ref/rtc/rn/API/class_focallengthinfo) (Android, iOS) +- [`CameraFocalLengthType`](/api-ref/rtc/rn/API/enum_camerafocallengthtype) (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration) adds a new member `cameraFocalLengthType` (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration) adds a new member `cameraId` (Android) +- [`EarMonitoringFilterType`](/api-ref/rtc/rn/API/enum_earmonitoringfiltertype) adds a new enumeration `EarMonitoringFilterBuiltInAudioFilters`(1 << 15) + ## v4.3.0-build.1 v4.3.0-build.1 was released on xx xx, 2024. diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md index 5635d2bd88f..caca678badb 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md @@ -8,6 +8,150 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`RegisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`OnFaceInfo`](/api-ref/rtc/unity/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Portrait center stage (iOS, macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the portrait center stage feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [`SetCameraStabilizationMode`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [`QueryCameraFocalLengthCapability`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) to check the device's focal length capabilities, and then call [`SetCameraCapturerConfiguration`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_setcameracapturerconfiguration) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `VIDEO_SOURCE_CAMERA_THIRD` (11) and `VIDEO_SOURCE_CAMERA_FOURTH` (12) in [`VIDEO_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_videosourcetype) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [`StartCameraCapture`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_startcameracapture). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [`ChannelMediaOptions`](/api-ref/rtc/unity/API/class_channelmediaoptions) add support for Android. Set these parameters to `true` when joining a channel with [`JoinChannel`](/api-ref/rtc/unity/API/toc_channel#api_irtcengine_joinchannel2)[2/2] to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [`StartCameraCapture`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_startcameracapture) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/unity/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`EnableEncryption`](/api-ref/rtc/unity/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `ENCRYPTION_ERROR_DATASTREAM_DECRYPTION_FAILURE` and `ENCRYPTION_ERROR_DATASTREAM_ENCRYPTION_FAILURE` enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`SetDualStreamMode`](/api-ref/rtc/unity/API/toc_dual_stream#api_irtcengine_setdualstreammode2)[2/2], the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [`EnableEncryptionEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`SetAudioMixingPlaybackSpeed`](/api-ref/rtc/unity/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`GetCallIdEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the [GetPlaybackDefaultDevice [2/2]](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2), [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2], [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2], and [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] method to obtain the information and type of audio playback and recording devices. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EAR_MONITORING_FILTER_REUSE_POST_PROCESSING_FILTER` (1 << 15) in `EAR_MONITORING_FILTER_TYPE`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (Windows) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AUDIO_EFFECT_PRESET`](/api-ref/rtc/unity/API/enum_audioeffectpreset), a new enumeration `ROOM_ACOUSTICS_CHORUS` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/unity/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [`EnableLoopbackRecording`](/api-ref/rtc/unity/API/toc_audio_capture#api_irtcengine_enableloopbackrecording) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- An occasional echo was observed when playing the audio stream of a specified user before mixing. (macOS, Windows) +- During interactions, when a local user set the system default playback device to speakers using , there was no sound from the remote end. (Windows) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- In scenarios using camera capture for local video, when the app was moved to the background and [`DisableVideo`](/api-ref/rtc/unity/API/toc_video_basic#api_irtcengine_disablevideo) or [`StopPreview`](/api-ref/rtc/unity/API/toc_video_basic#api_irtcengine_stoppreview)[1/2] was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. + +#### API Changes + +**Added** + +- [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (iOS, macOS) +- [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (iOS, macOS) +- [`SetCameraStabilizationMode`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode) (iOS) +- [`CAMERA_STABILIZATION_MODE`](/api-ref/rtc/unity/API/enum_camerastabilizationmode) (iOS) +- [`RegisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`UnregisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/unity/API/class_ifaceinfoobserver) +- [`OnFaceInfo`](/api-ref/rtc/unity/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MEDIA_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_mediasourcetype) adds `SPEECH_DRIVEN_VIDEO_SOURCE` +- [`VIDEO_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_videosourcetype) adds `VIDEO_SOURCE_SPEECH_DRIVEN` +- [`EncryptionConfig`](/api-ref/rtc/unity/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`ENCRYPTION_ERROR_TYPE`](/api-ref/rtc/unity/API/enum_encryptionerrortype) adds the following enumerations: + - `ENCRYPTION_ERROR_DATASTREAM_DECRYPTION_FAILURE` + - `ENCRYPTION_ERROR_DATASTREAM_ENCRYPTION_FAILURE` +- [`GetPlaybackDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2)[2/2] (macOS) +- [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2] (macOS) +- [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2] (macOS) +- [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] (macOS) +- [`RemoteAudioStats`](/api-ref/rtc/unity/API/class_remoteaudiostats) adds `e2eDelay` +- [`ERROR_CODE_TYPE`](/api-ref/rtc/unity/API/enum_errorcodetype) adds `ERR_DATASTREAM_DECRYPTION_FAILED` +- [`AUDIO_EFFECT_PRESET`](/api-ref/rtc/unity/API/enum_audioeffectpreset) adds `ROOM_ACOUSTICS_CHORUS`, enhancing the spatial presence of vocals in chorus scenarios. +- [`GetCallIdEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_getcallidex) +- [`EnableEncryptionEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_enableencryptionex) +- [`SetAudioMixingPlaybackSpeed`](/api-ref/rtc/unity/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`QueryCameraFocalLengthCapability`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) (Android, iOS) +- [`FocalLengthInfo`](/api-ref/rtc/unity/API/class_focallengthinfo) (Android, iOS) +- [`CAMERA_FOCAL_LENGTH_TYPE`](/api-ref/rtc/unity/API/enum_camerafocallengthtype) (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration) adds a new member `cameraFocalLengthType` (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration) adds a new member `cameraId` (Android) +- [`EAR_MONITORING_FILTER_TYPE`](/api-ref/rtc/unity/API/enum_earmonitoringfiltertype) adds a new enumeration `EAR_MONITORING_FILTER_BUILT_IN_AUDIO_FILTERS`(1 <<15) + ## v4.3.0 v4.3.0 was released on xx xx, 2024. @@ -66,7 +210,7 @@ This release has optimized the implementation of some functions, involving renam 6. **Log encryption behavior changes** - For security and performance reasons, as of this release, the SDK encrypts logs and no longer supports printing plaintext logs via the console. + For security and performance reasons, as of this release, the SDK encrypts logs and no longer supports printing plaintext logs via the console. Refer to the following solutions for different needs: diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md index d0909afb8a0..e6d68155637 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md @@ -106,6 +106,7 @@ This version fixed the following issues: - Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. - On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. - In scenarios using camera capture for local video, when the app was moved to the background and [`disableVideo`](/api-ref/rtc/android/API/toc_video_basic#api_irtcengine_disablevideo) or [`stopPreview`](/api-ref/rtc/android/API/toc_video_basic#api_irtcengine_stoppreview)[1/2] was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md index f9a627ee18e..687e8aeb824 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md @@ -28,11 +28,11 @@ This version is released on 2024 Month x, Day x. **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. For more details, see [](). -3. **Center stage camera** +3. **Portrait center stage** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization** @@ -98,6 +98,7 @@ This version is released on 2024 Month x, Day x. This version fixed the following issues: - Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md index 9d8cea28726..54bdc9cb744 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md @@ -16,11 +16,11 @@ This version is released on 2024 Month x, Day x. The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). -2. **Center stage camera** +2. **Portrait center stage** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 3. **Data stream encryption** @@ -83,6 +83,7 @@ This version fixed the following issues: - Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. - When a user plugged and unplugged a Bluetooth or wired headset once, the audio state change callback [`stateChanged`](/api-ref/rtc/macos/API/toc_common_device#api_irtcengine_statechanged) was triggered multiple times. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md index c468637716a..8cd9351b86d 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md @@ -68,6 +68,7 @@ This version fixed the following issues: - During interactions, when a local user set the system default playback device to speakers using [`setDevice`](/api-ref/rtc/windows/API/toc_audio_device#api_iaudiodevicecollection_setdevice), there was no sound from the remote end. - When sharing an Excel document window, remote users occasionally saw a green screen. - On devices using Intel graphics cards, occasionally there was a performance regression when publishing a small video stream. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes