From a4fa61862a88f78532d70e54db886b2bcea00a9c Mon Sep 17 00:00:00 2001 From: Cilla-luodan <85477033+Cilla-luodan@users.noreply.github.com> Date: Wed, 24 Apr 2024 10:45:07 +0800 Subject: [PATCH 01/17] wording and bugfix --- .../RTC 4.x/release-notes/en-US/native/release_android_ng.md | 1 + .../RTC 4.x/release-notes/en-US/native/release_ios_ng.md | 5 +++-- .../RTC 4.x/release-notes/en-US/native/release_mac_ng.md | 5 +++-- .../RTC 4.x/release-notes/en-US/native/release_windows_ng.md | 1 + 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md index d0909afb8a0..e6d68155637 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_android_ng.md @@ -106,6 +106,7 @@ This version fixed the following issues: - Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. - On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. - In scenarios using camera capture for local video, when the app was moved to the background and [`disableVideo`](/api-ref/rtc/android/API/toc_video_basic#api_irtcengine_disablevideo) or [`stopPreview`](/api-ref/rtc/android/API/toc_video_basic#api_irtcengine_stoppreview)[1/2] was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md index f9a627ee18e..7c99d2ed46e 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md @@ -28,9 +28,9 @@ This version is released on 2024 Month x, Day x. **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. For more details, see [](). -3. **Center stage camera** +3. **Portrait center stage** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage). @@ -98,6 +98,7 @@ This version is released on 2024 Month x, Day x. This version fixed the following issues: - Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md index 9d8cea28726..001c97019e8 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md @@ -16,9 +16,9 @@ This version is released on 2024 Month x, Day x. The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). -2. **Center stage camera** +2. **Portrait center stage** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage). @@ -83,6 +83,7 @@ This version fixed the following issues: - Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. - When a user plugged and unplugged a Bluetooth or wired headset once, the audio state change callback [`stateChanged`](/api-ref/rtc/macos/API/toc_common_device#api_irtcengine_statechanged) was triggered multiple times. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md index c468637716a..8cd9351b86d 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_windows_ng.md @@ -68,6 +68,7 @@ This version fixed the following issues: - During interactions, when a local user set the system default playback device to speakers using [`setDevice`](/api-ref/rtc/windows/API/toc_audio_device#api_iaudiodevicecollection_setdevice), there was no sound from the remote end. - When sharing an Excel document window, remote users occasionally saw a green screen. - On devices using Intel graphics cards, occasionally there was a performance regression when publishing a small video stream. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes From f8e3e0a1443d7438f239123eeb2678fcec84b7c2 Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 14:41:03 +0800 Subject: [PATCH 02/17] update release --- dita/RTC-NG/RTC_NG_API_Unity.ditamap | 1 + dita/RTC-NG/release/release_notes.dita | 15 +++++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/dita/RTC-NG/RTC_NG_API_Unity.ditamap b/dita/RTC-NG/RTC_NG_API_Unity.ditamap index 2d5a04765d7..87ee27dea8b 100644 --- a/dita/RTC-NG/RTC_NG_API_Unity.ditamap +++ b/dita/RTC-NG/RTC_NG_API_Unity.ditamap @@ -686,6 +686,7 @@ + diff --git a/dita/RTC-NG/release/release_notes.dita b/dita/RTC-NG/release/release_notes.dita index 584e5c9439c..43cd86143c2 100644 --- a/dita/RTC-NG/release/release_notes.dita +++ b/dita/RTC-NG/release/release_notes.dita @@ -99,8 +99,10 @@
  • 摄像头采集改进

    优化了摄像头采集的视频处理机制，减少噪点、提高亮度、改善色彩，使采集到的图形更加清晰、明亮和真实。

    • -
  • 获取音频设备类型 -

    该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

    +
  • 获取音频设备类型（macOS） +

    该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

    +

    该版本在 中新增 deviceTypeName 成员，用于获取音频设备的类型，如 built-in、USB、HDMI 等。

    +

    该版本新增 、、、 方法，分别用于获取音频播放、采集设备的设备信息及类型。

  • 虚拟背景算法优化

    为了提高纯色背景下开启虚拟背景时人像分割的精确度和稳定性，该版本优化了绿幕分割算法： @@ -181,10 +183,11 @@

    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • 中新增 deviceTypeName (macOS)
  • 新增 deviceTypeName
  • 新增 e2eDelay
  • 新增
    • From cf7d3117e639b114eacf5c19c6975d04c65edd12 Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 14:48:59 +0800 Subject: [PATCH 03/17] Update release_notes.dita --- en-US/dita/RTC-NG/release/release_notes.dita | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/en-US/dita/RTC-NG/release/release_notes.dita b/en-US/dita/RTC-NG/release/release_notes.dita index eabf34e749a..4d40d10e7f4 100644 --- a/en-US/dita/RTC-NG/release/release_notes.dita +++ b/en-US/dita/RTC-NG/release/release_notes.dita @@ -108,8 +108,10 @@
  • Camera capture improvements

    Improvements have been made to the video processing mechanism of camera capture, reducing noise, enhancing brightness, and improving color, making the captured images clearer, brighter, and more realistic.

    • -
  • Audio device type detection -

    This version adds the deviceTypeName member to , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +
  • Audio device type detection (macOS) +

    This version adds the deviceTypeName member to , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +

    This version adds the deviceTypeName in , used to identify the type of audio devices, such as built-in, USB, HDMI, etc.

    +

    This version adds the , , , and method to obtain the information and type of audio playback and recording deivces.

  • Virtual Background Algorithm Optimization

    To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: @@ -195,10 +197,11 @@

    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • -
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • (macOS)
    • +
  • adds deviceTypeName (macOS)
  • adds deviceTypeName
  • adds e2eDelay
  • adds
    • From fc17c0028e156d3a810362e70c63fa0403cc3f44 Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 14:51:12 +0800 Subject: [PATCH 04/17] Update release_notes.dita --- en-US/dita/RTC-NG/release/release_notes.dita | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en-US/dita/RTC-NG/release/release_notes.dita b/en-US/dita/RTC-NG/release/release_notes.dita index 4d40d10e7f4..451f0cbd8ab 100644 --- a/en-US/dita/RTC-NG/release/release_notes.dita +++ b/en-US/dita/RTC-NG/release/release_notes.dita @@ -201,7 +201,7 @@
  • (macOS)
  • (macOS)
  • (macOS)
    • -
  • adds deviceTypeName (macOS)
    • +
  • adds deviceTypeName (macOS)
  • adds deviceTypeName
  • adds e2eDelay
  • adds
    • From d42a914b0ae270983e71079c85b6930161e27c62 Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 14:54:43 +0800 Subject: [PATCH 05/17] Update release_notes.dita --- dita/RTC-NG/release/release_notes.dita | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dita/RTC-NG/release/release_notes.dita b/dita/RTC-NG/release/release_notes.dita index 43cd86143c2..e520a76decd 100644 --- a/dita/RTC-NG/release/release_notes.dita +++ b/dita/RTC-NG/release/release_notes.dita @@ -187,7 +187,7 @@
  • (macOS)
  • (macOS)
  • (macOS)
    • -
  • 中新增 deviceTypeName (macOS)
    • +
  • 中新增 deviceTypeName (macOS)
  • 新增 deviceTypeName
  • 新增 e2eDelay
  • 新增
    • From c77bf747716419c2edfde9a09a4e690a67d22d44 Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 15:53:07 +0800 Subject: [PATCH 06/17] unity en --- .../en-US/framework/release_unity_ng.md | 139 +++++++++++++++++- 1 file changed, 138 insertions(+), 1 deletion(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md index 5635d2bd88f..d5dc4fb83eb 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md @@ -8,6 +8,143 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`RegisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`OnFaceInfo`](/api-ref/rtc/unity/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Portrait center stage (iOS, macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the portrait center stage feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [`SetCameraStabilizationMode`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [`QueryCameraFocalLengthCapability`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) to check the device's focal length capabilities, and then call [`SetCameraCapturerConfiguration`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_setcameracapturerconfiguration) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `VIDEO_SOURCE_CAMERA_THIRD` (11) and `VIDEO_SOURCE_CAMERA_FOURTH` (12) in [`VIDEO_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_videosourcetype) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [`StartCameraCapture`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_startcameracapture). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [`ChannelMediaOptions`](/api-ref/rtc/unity/API/class_channelmediaoptions) add support for Android. Set these parameters to `true` when joining a channel with [`JoinChannel`](/api-ref/rtc/unity/API/toc_channel#api_irtcengine_joinchannel2)[2/2] to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [`StartCameraCapture`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_startcameracapture) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/unity/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`EnableEncryption`](/api-ref/rtc/unity/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `ENCRYPTION_ERROR_DATASTREAM_DECRYPTION_FAILURE` and `ENCRYPTION_ERROR_DATASTREAM_ENCRYPTION_FAILURE` enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`SetDualStreamMode`](/api-ref/rtc/unity/API/toc_dual_stream#api_irtcengine_setdualstreammode2)[2/2], the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [`EnableEncryptionEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`SetAudioMixingPlaybackSpeed`](/api-ref/rtc/unity/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`GetCallIdEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the [GetPlaybackDefaultDevice [2/2]](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2), [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2], [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2], and [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] method to obtain the information and type of audio playback and recording deivces. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EAR_MONITORING_FILTER_REUSE_POST_PROCESSING_FILTER` (1 << 15) in `EAR_MONITORING_FILTER_TYPE`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (Windows) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AUDIO_EFFECT_PRESET`](/api-ref/rtc/unity/API/enum_audioeffectpreset), a new enumeration `ROOM_ACOUSTICS_CHORUS` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/unity/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- The SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. + +#### API Changes + +**Added** + +- [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (iOS, macOS) +- [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (iOS, macOS) +- [`SetCameraStabilizationMode`](/api-ref/rtc/unity/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode) (iOS) +- [`CAMERA_STABILIZATION_MODE`](/api-ref/rtc/unity/API/enum_camerastabilizationmode) (iOS) +- [`RegisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`UnregisterFaceInfoObserver`](/api-ref/rtc/unity/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/unity/API/class_ifaceinfoobserver) +- [`OnFaceInfo`](/api-ref/rtc/unity/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MEDIA_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_mediasourcetype) adds `SPEECH_DRIVEN_VIDEO_SOURCE` +- [`VIDEO_SOURCE_TYPE`](/api-ref/rtc/unity/API/enum_videosourcetype) adds `VIDEO_SOURCE_SPEECH_DRIVEN` +- [`EncryptionConfig`](/api-ref/rtc/unity/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`ENCRYPTION_ERROR_TYPE`](/api-ref/rtc/unity/API/enum_encryptionerrortype) adds the following enumerations: + - `ENCRYPTION_ERROR_DATASTREAM_DECRYPTION_FAILURE` + - `ENCRYPTION_ERROR_DATASTREAM_ENCRYPTION_FAILURE` +- [`GetPlaybackDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2)[2/2] (macOS) +- [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2] (macOS) +- [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2] (macOS) +- [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] (macOS) +- [`RemoteAudioStats`](/api-ref/rtc/unity/API/class_remoteaudiostats) adds `e2eDelay` +- [`ERROR_CODE_TYPE`](/api-ref/rtc/unity/API/enum_errorcodetype) adds `ERR_DATASTREAM_DECRYPTION_FAILED` +- [`AUDIO_EFFECT_PRESET`](/api-ref/rtc/unity/API/enum_audioeffectpreset) adds `ROOM_ACOUSTICS_CHORUS`, enhancing the spatial presence of vocals in chorus scenarios. +- [`GetCallIdEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_getcallidex) +- [`EnableEncryptionEx`](/api-ref/rtc/unity/API/toc_network#api_irtcengineex_enableencryptionex) +- [`SetAudioMixingPlaybackSpeed`](/api-ref/rtc/unity/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`QueryCameraFocalLengthCapability`](/api-ref/rtc/unity/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) (Android, iOS) +- [`FocalLengthInfo`](/api-ref/rtc/unity/API/class_focallengthinfo) (Android, iOS) +- [`CAMERA_FOCAL_LENGTH_TYPE`](/api-ref/rtc/unity/API/enum_camerafocallengthtype) (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration) adds a new member `cameraFocalLengthType` (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/unity/API/class_cameracapturerconfiguration) adds a new member `cameraId` (Android) +- [`EAR_MONITORING_FILTER_TYPE`](/api-ref/rtc/unity/API/enum_earmonitoringfiltertype) adds a new enumeration `EAR_MONITORING_FILTER_BUILT_IN_AUDIO_FILTERS`(1 <<15) + ## v4.3.0 v4.3.0 was released on xx xx, 2024. @@ -66,7 +203,7 @@ This release has optimized the implementation of some functions, involving renam 6. **Log encryption behavior changes** - For security and performance reasons, as of this release, the SDK encrypts logs and no longer supports printing plaintext logs via the console. + For security and performance reasons, as of this release, the SDK encrypts logs and no longer supports printing plaintext logs via the console. Refer to the following solutions for different needs: From 4d7b09e3ee79770f29b97b6f3370b85c10897f04 Mon Sep 17 00:00:00 2001 From: jinyu Date: Wed, 24 Apr 2024 16:08:44 +0800 Subject: [PATCH 07/17] Update release_flutter_ng.md --- .../en-US/framework/release_flutter_ng.md | 142 ++++++++++++++++++ 1 file changed, 142 insertions(+) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md index b3139796757..d71df77af21 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md @@ -9,6 +9,148 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v6.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [registerFaceInfoObserver](API/api_imediaengine_registerfaceinfoobserver.html) method and [onFaceInfo](API/callback_ifaceinfoobserver_onfaceinfo.html) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Center stage camera (iOS, macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [isCameraCenterStageSupported](API/api_irtcengine_iscameracenterstagesupported.html). A list of supported devices can be found in the API documentation at [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [setCameraStabilizationMode](API/api_irtcengine_setcamerastabilizationmode.html), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [queryCameraFocalLengthCapability](API/api_irtcengine_querycamerafocallengthcapability.html) to check the device's focal length capabilities, and then call [setCameraCapturerConfiguration](API/api_irtcengine_setcameracapturerconfiguration.html) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `videoSourceCameraThird` (11) and `videoSourceCameraFourth` (12) in [VideoSourceType](API/enum_videosourcetype.html) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [startCameraCapture](API/api_irtcengine_startcameracapture.html). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [ChannelMediaOptions](API/class_channelmediaoptions.html) add support for Android. Set these parameters to `true` when joining a channel with [joinChannel](API/api_irtcengine_joinchannel2.html) to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [startCameraCapture](API/api_irtcengine_startcameracapture.html) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [EncryptionConfig](API/class_encryptionconfig.html) for enabling data stream encryption. You can set this when you activate encryption with [enableEncryption](API/api_irtcengine_enableencryption.html). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `encryptionErrorDatastreamDecryptionFailure` and `encryptionErrorDatastreamEncryptionFailure`enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [setDualStreamMode](API/api_irtcengine_setdualstreammode2.html), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [enableEncryptionEx](API/api_irtcengineex_enableencryptionex.html) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [setAudioMixingPlaybackSpeed](API/api_irtcengine_setaudiomixingplaybackspeed.html) is introduced for setting the playback speed of audio files. + - New method [getCallIdEx](API/api_irtcengineex_getcallidex.html) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the `deviceTypeName` in [AudioDeviceInfo](API/class_audiodeviceinfo.html), used to identify the type of audio devices, such as built-in, USB, HDMI, etc. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `earMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (iOS, macOS, Windows) + - Enhanced performance and stability of the local compositing feature, reducing its CPU usage. (Android) + - Improved stability in processing video by the raw video frame observer. (iOS) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [AudioEffectPreset](API/enum_audioeffectpreset.html), a new enumeration `roomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [RemoteAudioStats](API/class_remoteaudiostats.html), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [enableLoopbackRecording](API/api_irtcengine_enableloopbackrecording.html) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- During interactions, when a local user set the system default playback device to speakers using , there was no sound from the remote end. (Windows) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- In scenarios using camera capture for local video, when the app was moved to the background and [disableVideo](API/api_irtcengine_disablevideo.html) or stopPreview was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) + +#### API Changes + +**Added** + +- [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html) (iOS, macOS) +- [isCameraCenterStageSupported](API/api_irtcengine_iscameracenterstagesupported.html) (iOS, macOS) +- [setCameraStabilizationMode](API/api_irtcengine_setcamerastabilizationmode.html) (iOS) +- [CameraStabilizationMode](API/enum_camerastabilizationmode.html) (iOS) +- [registerFaceInfoObserver](API/api_imediaengine_registerfaceinfoobserver.html) +- [unregisterFaceInfoObserver](API/api_imediaengine_unregisterfaceinfoobserver.html) +- [FaceInfoObserver](API/class_ifaceinfoobserver.html) +- [onFaceInfo](API/callback_ifaceinfoobserver_onfaceinfo.html) +- [MediaSourceType](API/enum_mediasourcetype.html) adds `speechDrivenVideoSource` +- [VideoSourceType](API/enum_videosourcetype.html) adds `videoSourceSpeechDriven` +- [EncryptionConfig](API/class_encryptionconfig.html) adds `datastreamEncryptionEnabled` +- [EncryptionErrorType](API/enum_encryptionerrortype.html) adds the following enumerations: + - `encryptionErrorDatastreamDecryptionFailure` + - `encryptionErrorDatastreamEncryptionFailure` +- [AudioDeviceInfo](API/class_audiodeviceinfo.html) adds `deviceTypeName` (macOS) +- [RemoteAudioStats](API/class_remoteaudiostats.html) adds `e2eDelay` +- [ErrorCodeType](API/enum_errorcodetype.html) adds `errDatastreamDecryptionFailed` +- [AudioEffectPreset](API/enum_audioeffectpreset.html) adds `roomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [getCallIdEx](API/api_irtcengineex_getcallidex.html) +- [enableEncryptionEx](API/api_irtcengineex_enableencryptionex.html) +- [setAudioMixingPlaybackSpeed](API/api_irtcengine_setaudiomixingplaybackspeed.html) +- [queryCameraFocalLengthCapability](API/api_irtcengine_querycamerafocallengthcapability.html) (Android, iOS) +- [FocalLengthInfo](API/class_focallengthinfo.html) (Android, iOS) +- [CameraFocalLengthType](API/enum_camerafocallengthtype.html) (Android, iOS) +- [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraFocalLengthType` (Android, iOS) +- [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraId` (Android) +- [EarMonitoringFilterType](API/enum_earmonitoringfiltertype.html) adds a new enumeration `earMonitoringFilterBuiltInAudioFilters`(1 <<15) + + ## v6.3.0 v6.3.0 was released on xx xx, 2024. From 57a0a5bbe66ad56d2b6708eaf9563ce51738ed4c Mon Sep 17 00:00:00 2001 From: Nero-Hu Date: Wed, 24 Apr 2024 16:09:32 +0800 Subject: [PATCH 08/17] update props for guide-audio-route --- .../API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita | 2 +- dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita index dcd6e8897f6..5972b8f107c 100644 --- a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita @@ -50,7 +50,7 @@

    调用该 API 可以改变上述默认音频路由。成功改变音频路由后，SDK 会触发 回调。

    -

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。

    +

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。
    参数 diff --git a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita index 574cc87f529..3861c50eaf4 100644 --- a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita @@ -24,7 +24,7 @@ Future<void> setEnableSpeakerphone(bool speakerOn);

    详情 -

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    +

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    该方法只设置用户在当前频道内使用的音频路由，不会影响 SDK 默认的音频路由。如果用户离开当前频道并加入新的频道，则用户还是会使用 SDK 默认的音频路由。

      From 05aaeebddd3f8e2c691a50ebb06281d2d766ce1e Mon Sep 17 00:00:00 2001 From: Nero-Hu Date: Wed, 24 Apr 2024 16:14:57 +0800 Subject: [PATCH 09/17] fix props --- .../API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita | 2 +- dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita index 5972b8f107c..1be13570e8f 100644 --- a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita @@ -50,7 +50,7 @@

    调用该 API 可以改变上述默认音频路由。成功改变音频路由后，SDK 会触发 回调。

    -

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。

    +

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。
    参数 diff --git a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita index 3861c50eaf4..18b33fd793e 100644 --- a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita @@ -24,7 +24,7 @@ Future<void> setEnableSpeakerphone(bool speakerOn);

    详情 -

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。 +

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    该方法只设置用户在当前频道内使用的音频路由，不会影响 SDK 默认的音频路由。如果用户离开当前频道并加入新的频道，则用户还是会使用 SDK 默认的音频路由。

      From cca93128829c2a0ce51ebcb7cb7e672f9fb1af0b Mon Sep 17 00:00:00 2001 From: jinyu Date: Wed, 24 Apr 2024 16:15:30 +0800 Subject: [PATCH 10/17] Update release_flutter_ng.md --- .../release-notes/en-US/framework/release_flutter_ng.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md index d71df77af21..b202c002d7d 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md @@ -29,11 +29,11 @@ This version is released on 2024 Month x, Day x. **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. -3. **Center stage camera (iOS, macOS)** +3. **Portrait center stage** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [isCameraCenterStageSupported](API/api_irtcengine_iscameracenterstagesupported.html). A list of supported devices can be found in the API documentation at [enableCameraCenterStage](API/api_irtcengine_enablecameracenterstage.html). + Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization (iOS)** @@ -117,6 +117,7 @@ This version fixed the following issues: - On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) - When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) - In scenarios using camera capture for local video, when the app was moved to the background and [disableVideo](API/api_irtcengine_disablevideo.html) or stopPreview was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes From 37bb774aa80f585f1dce7d766a36270df990ab65 Mon Sep 17 00:00:00 2001 From: Nero-Hu Date: Wed, 24 Apr 2024 16:27:21 +0800 Subject: [PATCH 11/17] RN/Electron: update en release --- .../en-US/framework/release_electron_ng.md | 104 ++++++++++++++ .../en-US/framework/release_rn_ng.md | 128 ++++++++++++++++++ 2 files changed, 232 insertions(+) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md index bfaf73413fc..92f4862c435 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md @@ -1,3 +1,107 @@ +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`registerFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`onFaceInfo`](/api-ref/rtc/electron/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Center stage camera (macOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +3. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/electron/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`enableEncryption`](/api-ref/rtc/electron/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `EncryptionErrorDatastreamDecryptionFailure` and `EncryptionErrorDatastreamEncryptionFailure` enumerations. + +4. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`setDualStreamMode`](/api-ref/rtc/electron/API/toc_dual_stream#api_irtcengine_setdualstreammode2), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +5. **Other features** + + - New method [`enableEncryptionEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/electron/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`getCallIdEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Optimization for game scenario screen sharing (Windows)** + + This version specifically optimizes screen sharing for game scenarios, enhancing performance, stability, and clarity in ultra-high definition (4K, 60 fps) game scenarios, resulting in a clearer, smoother, and more stable gaming experience for players. + +2. **Audio device type detection (macOS)** + + This version adds the `deviceTypeName` in [`AudioDeviceInfo`](/api-ref/rtc/electron/API/class_audiodeviceinfo), used to identify the type of audio devices, such as built-in, USB, HDMI, etc. + +3. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +4. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EarMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +5. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (macOS, Windows) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AudioEffectPreset`](/api-ref/rtc/electron/API/enum_audioeffectpreset), a new enumeration `RoomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/electron/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [`enableLoopbackRecording`](/api-ref/rtc/electron/API/toc_audio_capture#api_irtcengine_enableloopbackrecording) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- In audio-video interactions, if a user inserted headphones into the device and manually switched the system audio output to speakers, and later the app called [`setPlaybackDevice`](/api-ref/rtc/electron/API/toc_audio_device#api_iaudiodevicemanager_setplaybackdevice) to specify the headphone as the audio playback device, audio output did not switch back to speakers as expected after the headphones were removed. (macOS) + +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- On devices using Intel graphics cards, occasionally there was a performance regression when publishing a small video stream. (Windows) + +#### API Changes + +**Added** + +- [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (macOS) +- [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (macOS) +- [`registerFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`unregisterFaceInfoObserver`](/api-ref/rtc/electron/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/electron/API/class_ifaceinfoobserver) +- [`onFaceInfo`](/api-ref/rtc/electron/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MediaSourceType`](/api-ref/rtc/electron/API/enum_mediasourcetype) adds `SpeechDrivenVideoSource` +- [`VideoSourceType`](/api-ref/rtc/electron/API/enum_videosourcetype) adds `VideoSourceSpeechDriven` +- [`EncryptionConfig`](/api-ref/rtc/electron/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`EncryptionErrorType`](/api-ref/rtc/electron/API/enum_encryptionerrortype) adds the following enumerations: + - `EncryptionErrorDatastreamDecryptionFailure` + - `EncryptionErrorDatastreamEncryptionFailure` +- [`AudioDeviceInfo`](/api-ref/rtc/electron/API/class_audiodeviceinfo) adds `deviceTypeName` (macOS) +- [`RemoteAudioStats`](/api-ref/rtc/electron/API/class_remoteaudiostats) adds `e2eDelay` +- [`ErrorCodeType`](/api-ref/rtc/electron/API/enum_errorcodetype) adds `ErrDatastreamDecryptionFailed` +- [`AudioEffectPreset`](/api-ref/rtc/electron/API/enum_audioeffectpreset) adds `RoomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [`getCallIdEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_getcallidex) +- [`enableEncryptionEx`](/api-ref/rtc/electron/API/toc_network#api_irtcengineex_enableencryptionex) +- [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/electron/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`EarMonitoringFilterType`](/api-ref/rtc/electron/API/enum_earmonitoringfiltertype) adds a new enumeration `EarMonitoringFilterBuiltInAudioFilters`(1 << 15) + ## v4.3.0 v4.3.0 was released on xx xx, 2024. diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md index 74a8334fa0d..39fb81c308c 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md @@ -8,6 +8,134 @@ On Android 14 devices (such as OnePlus 11), screen sharing may not be available AirPods Pro does not support A2DP protocol in communication audio mode, which may lead to connection failure in that mode. +## v4.3.1 + +This version is released on 2024 Month x, Day x. + +#### New Features + +1. **Speech Driven Avatar** + + The SDK introduces a speech driven extension that converts speech information into corresponding facial expressions to animate avatar. You can access the facial information through the newly added [`registerFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) method and [`onFaceInfo`](/api-ref/rtc/rn/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) callback. This facial information conforms to the ARKit standard for Blend Shapes (BS), which you can further process using third-party 3D rendering engines. + + The speech driven extension is a trimmable dynamic library, and details about the increase in app size are available at [reduce-app-size](). + + **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). + +2. **Privacy manifest file (iOS)** + + To meet Apple's safety compliance requirements for app publication, the SDK now includes a privacy manifest file, `PrivacyInfo.xcprivacy`, detailing the SDK's API calls that access or use user data, along with a description of the types of data collected. + + **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. + +3. **Center stage camera (iOS)** + + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + + Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + +4. **Camera stabilization (iOS)** + + To improve video stability in mobile filming, low-light environments, and hand-held shooting scenarios, this version introduces a camera stabilization feature. You can activate this feature and select an appropriate stabilization mode by calling [`setCameraStabilizationMode`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode), achieving more stable and clearer video footage. + +5. **Wide and ultra-wide cameras (Android, iOS)** + + To allow users to capture a broader field of view and more complete scene content, this release introduces support for wide and ultra-wide cameras. You can first call [`queryCameraFocalLengthCapability`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) to check the device's focal length capabilities, and then call [`setCameraCapturerConfiguration`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_setcameracapturerconfiguration) and set `cameraFocalLengthType` to the supported focal length types, including wide and ultra-wide. + +6. **Multi-camera capture (Android)** + + This release introduces additional functionalities for Android camera capture: + + 1. Support for capturing and publishing video streams from the third and fourth cameras: + + - `VideoSourceCameraThird` (11) and `VideoSourceCameraFourth` (12) in [`VideoSourceType`](/api-ref/rtc/rn/API/enum_videosourcetype) add support for Android, specifically for the third and fourth camera sources. This change allows you to specify up to four camera streams when initiating camera capture by calling [`startCameraCapture`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_startcameracapture). + - `publishThirdCameraTrack` and `publishFourthCameraTrack` in [`ChannelMediaOptions`](/api-ref/rtc/rn/API/class_channelmediaoptions) add support for Android. Set these parameters to `true` when joining a channel with [`joinChannel`](/api-ref/rtc/rn/API/toc_channel#api_irtcengine_joinchannel2) to publish video streams captured from the third and fourth cameras. + + 2. Support for specifying cameras by camera ID: + + A new parameter `cameraId` is added to [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration). For devices with multiple cameras, where `cameraDirection` cannot identify or access all available cameras, you can obtain the camera ID through Android's native system APIs and specify the desired camera by calling [`startCameraCapture`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_startcameracapture) with the specific `cameraId`. + +7. **Data stream encryption** + + This version adds `datastreamEncryptionEnabled` to [`EncryptionConfig`](/api-ref/rtc/rn/API/class_encryptionconfig) for enabling data stream encryption. You can set this when you activate encryption with [`enableEncryption`](/api-ref/rtc/rn/API/toc_network#api_irtcengine_enableencryption). If there are issues causing failures in data stream encryption or decryption, these can be identified by the newly added `EncryptionErrorDatastreamDecryptionFailure` and `EncryptionErrorDatastreamEncryptionFailure` enumerations. + +8. **Adaptive configuration for low-quality video streams** + + This version introduces adaptive configuration for low-quality video streams. When you activate dual-stream mode and set up low-quality video streams on the sending side using [`setDualStreamMode`](/api-ref/rtc/rn/API/toc_dual_stream#api_irtcengine_setdualstreammode2), the SDK defaults to the following behaviors: + + - The default encoding resolution for low-quality video streams is set to 50% of the original video encoding resolution. + - The bitrate for the small streams is automatically matched based on the video resolution and frame rate, eliminating the need for manual specification. + +9. **Other features** + + - New method [`enableEncryptionEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_enableencryptionex) is added for enabling media stream or data stream encryption in multi-channel scenarios. + - New method [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/rn/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) is introduced for setting the playback speed of audio files. + - New method [`getCallIdEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_getcallidex) is introduced for retrieving call IDs in multi-channel scenarios. + +#### Improvements + +1. **Virtual Background Algorithm Optimization** + + To enhance the accuracy and stability of human segmentation when activating virtual backgrounds against solid colors, this version optimizes the green screen segmentation algorithm: + + - Supports recognition of any solid color background, no longer limited to green screens. + - Improves accuracy in recognizing background colors and reduces the background exposure during human segmentation. + - After segmentation, the edges of the human figure (especially around the fingers) are more stable, significantly reducing flickering at the edges. + +2. **CPU consumption reduction of in-ear monitoring** + + This release adds an enumerator `EarMonitoringFilterReusePostProcessingFilter` (1 << 15) in `EarMonitoringFilterType`. For complex audio processing scenarios, you can specify this option to reuse the audio filter post sender-side processing in in-ear monitoring, thereby reducing CPU consumption. Note that this option may increase the latency of in-ear monitoring, which is suitable for latency-tolerant scenarios requiring low CPU consumption. + +3. **Other improvements** + + This version also includes the following improvements: + + - Optimization of video encoding and decoding strategies in non-screen sharing scenarios to save system performance overhead. (iOS) + - Enhanced performance and stability of the local compositing feature, reducing its CPU usage. (Android) + - Enhanced media player capabilities to handle WebM format videos, including support for rendering alpha channels. + - In [`AudioEffectPreset`](/api-ref/rtc/rn/API/enum_audioeffectpreset), a new enumeration `RoomAcousticsChorus` (chorus effect) is added, enhancing the spatial presence of vocals in chorus scenarios. + - In [`RemoteAudioStats`](/api-ref/rtc/rn/API/class_remoteaudiostats), a new `e2eDelay` field is added to report the delay from when the audio is captured on the sending end to when the audio is played on the receiving end. + +#### Issues fixed + +This version fixed the following issues: + +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- In scenarios using camera capture for local video, when the app was moved to the background and [`disableVideo`](/api-ref/rtc/rn/API/toc_video_basic#api_irtcengine_disablevideo) or was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) + +#### API Changes + +**Added** + +- [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) (iOS) +- [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported) (iOS) +- [`setCameraStabilizationMode`](/api-ref/rtc/rn/API/toc_camera_capture#api_irtcengine_setcamerastabilizationmode) (iOS) +- [`CameraStabilizationMode`](/api-ref/rtc/rn/API/enum_camerastabilizationmode) (iOS) +- [`registerFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_registerfaceinfoobserver) +- [`unregisterFaceInfoObserver`](/api-ref/rtc/rn/API/toc_speech_driven#api_imediaengine_unregisterfaceinfoobserver) +- [`IFaceInfoObserver`](/api-ref/rtc/rn/API/class_ifaceinfoobserver) +- [`onFaceInfo`](/api-ref/rtc/rn/API/toc_speech_driven#callback_ifaceinfoobserver_onfaceinfo) +- [`MediaSourceType`](/api-ref/rtc/rn/API/enum_mediasourcetype) adds `SpeechDrivenVideoSource` +- [`VideoSourceType`](/api-ref/rtc/rn/API/enum_videosourcetype) adds `VideoSourceSpeechDriven` +- [`EncryptionConfig`](/api-ref/rtc/rn/API/class_encryptionconfig) adds `datastreamEncryptionEnabled` +- [`EncryptionErrorType`](/api-ref/rtc/rn/API/enum_encryptionerrortype) adds the following enumerations: + - `EncryptionErrorDatastreamDecryptionFailure` + - `EncryptionErrorDatastreamEncryptionFailure` +- [`RemoteAudioStats`](/api-ref/rtc/rn/API/class_remoteaudiostats) adds `e2eDelay` +- [`ErrorCodeType`](/api-ref/rtc/rn/API/enum_errorcodetype) adds `ErrDatastreamDecryptionFailed` +- [`AudioEffectPreset`](/api-ref/rtc/rn/API/enum_audioeffectpreset) adds `RoomAcousticsChorus`, enhancing the spatial presence of vocals in chorus scenarios. +- [`getCallIdEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_getcallidex) +- [`enableEncryptionEx`](/api-ref/rtc/rn/API/toc_network#api_irtcengineex_enableencryptionex) +- [`setAudioMixingPlaybackSpeed`](/api-ref/rtc/rn/API/toc_audio_mixing#api_irtcengine_setaudiomixingplaybackspeed) +- [`queryCameraFocalLengthCapability`](/api-ref/rtc/rn/API/toc_video_device#api_irtcengine_querycamerafocallengthcapability) (Android, iOS) +- [`FocalLengthInfo`](/api-ref/rtc/rn/API/class_focallengthinfo) (Android, iOS) +- [`CameraFocalLengthType`](/api-ref/rtc/rn/API/enum_camerafocallengthtype) (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration) adds a new member `cameraFocalLengthType` (Android, iOS) +- [`CameraCapturerConfiguration`](/api-ref/rtc/rn/API/class_cameracapturerconfiguration) adds a new member `cameraId` (Android) +- [`EarMonitoringFilterType`](/api-ref/rtc/rn/API/enum_earmonitoringfiltertype) adds a new enumeration `EarMonitoringFilterBuiltInAudioFilters`(1 << 15) + ## v4.3.0-build.1 v4.3.0-build.1 was released on xx xx, 2024. From 407a166069fb3ec2fb82682232aa85ce0db33073 Mon Sep 17 00:00:00 2001 From: Nero-Hu Date: Wed, 24 Apr 2024 16:42:31 +0800 Subject: [PATCH 12/17] update portrait center stage --- .../release-notes/en-US/framework/release_electron_ng.md | 7 ++++--- .../RTC 4.x/release-notes/en-US/framework/release_rn_ng.md | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md index 92f4862c435..0eb5fc8865e 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_electron_ng.md @@ -12,11 +12,11 @@ This version is released on 2024 Month x, Day x. **Attention:** The speech driven avatar feature is currently in beta testing. To use it, please contact [technical support](mailto:support@agora.io). -2. **Center stage camera (macOS)** +2. **Portrait center stage (macOS)** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/electron/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 3. **Data stream encryption** @@ -76,6 +76,7 @@ This version fixed the following issues: - When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) - On devices using Intel graphics cards, occasionally there was a performance regression when publishing a small video stream. (Windows) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md index 39fb81c308c..ffc94d40d15 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_rn_ng.md @@ -28,11 +28,11 @@ This version is released on 2024 Month x, Day x. **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. -3. **Center stage camera (iOS)** +3. **Portrait center stage (iOS)** - To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the center stage camera feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. + To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/rn/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization (iOS)** @@ -104,6 +104,7 @@ This version fixed the following issues: - Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) - On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) - In scenarios using camera capture for local video, when the app was moved to the background and [`disableVideo`](/api-ref/rtc/rn/API/toc_video_basic#api_irtcengine_disablevideo) or was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) +- When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes From c33f38911ef38a612825d305267e874d41e408a3 Mon Sep 17 00:00:00 2001 From: jinyu Date: Wed, 24 Apr 2024 17:24:36 +0800 Subject: [PATCH 13/17] Update release_flutter_ng.md --- .../RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md index b202c002d7d..4d37f4b3e2e 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md @@ -29,7 +29,7 @@ This version is released on 2024 Month x, Day x. **Note:** If you need to publish an app with SDK versions prior to v4.3.1 to the Apple App Store, you must manually add the `PrivacyInfo.xcprivacy` file to your Xcode project. -3. **Portrait center stage** +3. **Portrait center stage (iOS, macOS)** To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. From 81faa8b190e51816f8ee9f7d9610bbe4d550dc81 Mon Sep 17 00:00:00 2001 From: Nero-Hu Date: Wed, 24 Apr 2024 17:24:41 +0800 Subject: [PATCH 14/17] fix props for audio route --- .../api_irtcengine_setdefaultaudioroutetospeakerphone.dita | 4 ++-- dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita | 2 +- dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap | 2 +- dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap | 1 + 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita index 1be13570e8f..50656b22c88 100644 --- a/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setdefaultaudioroutetospeakerphone.dita @@ -49,8 +49,8 @@
    • 视频直播：扬声器

    调用该 API 可以改变上述默认音频路由。成功改变音频路由后，SDK 会触发 回调。

    - -

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。

    + +

    当手机插入外接设备，如蓝牙设备或耳机时，系统的音频路由会发生改变。详细的路由变化规律请参考 音频路由。
    参数 diff --git a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita index 18b33fd793e..5acc058e351 100644 --- a/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita +++ b/dita/RTC-NG/API/api_irtcengine_setenablespeakerphone.dita @@ -24,7 +24,7 @@ Future<void> setEnableSpeakerphone(bool speakerOn);

    详情 -

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。 +

    如果 SDK 默认的音频路由（见音频路由）或 的设置无法满足你的需求，你可以调用 切换当前的音频路由。成功改变音频路由后，SDK 会触发 回调。

    该方法只设置用户在当前频道内使用的音频路由，不会影响 SDK 默认的音频路由。如果用户离开当前频道并加入新的频道，则用户还是会使用 SDK 默认的音频路由。

      diff --git a/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap b/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap index d86579f0f65..15ca8bce264 100644 --- a/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap +++ b/dita/RTC-NG/config/keys-rtc-ng-links-rn.ditamap @@ -4,7 +4,7 @@ Keys RTC NG Windows Links - + diff --git a/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap b/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap index 363e74df5c9..1989a6ab4dd 100644 --- a/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap +++ b/dita/RTC-NG/config/keys-rtc-ng-links-unity.ditamap @@ -4,6 +4,7 @@ Keys RTC NG Unity Links + From ff91be376bf534582cfc03a919af8b47d5887586 Mon Sep 17 00:00:00 2001 From: jinyu Date: Wed, 24 Apr 2024 17:45:34 +0800 Subject: [PATCH 15/17] Update release_flutter_ng.md --- .../release-notes/en-US/framework/release_flutter_ng.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md index 4d37f4b3e2e..c2593b99d6c 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_flutter_ng.md @@ -33,7 +33,7 @@ This version is released on 2024 Month x, Day x. To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/flutter/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization (iOS)** @@ -113,7 +113,6 @@ This version fixed the following issues: - Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. - In screen sharing scenarios, when the app enabled sound card capture with [enableLoopbackRecording](API/api_irtcengine_enableloopbackrecording.html) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) - Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) -- During interactions, when a local user set the system default playback device to speakers using , there was no sound from the remote end. (Windows) - On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) - When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) - In scenarios using camera capture for local video, when the app was moved to the background and [disableVideo](API/api_irtcengine_disablevideo.html) or stopPreview was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) @@ -149,7 +148,7 @@ This version fixed the following issues: - [CameraFocalLengthType](API/enum_camerafocallengthtype.html) (Android, iOS) - [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraFocalLengthType` (Android, iOS) - [CameraCapturerConfiguration](API/class_cameracapturerconfiguration.html) adds a new member `cameraId` (Android) -- [EarMonitoringFilterType](API/enum_earmonitoringfiltertype.html) adds a new enumeration `earMonitoringFilterBuiltInAudioFilters`(1 <<15) +- [EarMonitoringFilterType](API/enum_earmonitoringfiltertype.html) adds a new enumeration `earMonitoringFilterBuiltInAudioFilters`(1 << 15) ## v6.3.0 From 31facd002d3498ac4985f0fc63c140d5007c8a4c Mon Sep 17 00:00:00 2001 From: Cilla-luodan <85477033+Cilla-luodan@users.noreply.github.com> Date: Wed, 24 Apr 2024 17:59:02 +0800 Subject: [PATCH 16/17] 1 --- .../RTC 4.x/release-notes/en-US/framework/release_unity_ng.md | 2 +- markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md | 2 +- markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md index d5dc4fb83eb..807f4b7fd75 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md @@ -32,7 +32,7 @@ This version is released on 2024 Month x, Day x. To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate the portrait center stage feature. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`IsCameraCenterStageSupported`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`EnableCameraCenterStage`](/api-ref/rtc/unity/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization (iOS)** diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md index 7c99d2ed46e..687e8aeb824 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_ios_ng.md @@ -32,7 +32,7 @@ This version is released on 2024 Month x, Day x. To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/ios/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 4. **Camera stabilization** diff --git a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md index 001c97019e8..54bdc9cb744 100644 --- a/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/native/release_mac_ng.md @@ -20,7 +20,7 @@ This version is released on 2024 Month x, Day x. To enhance the presentation effect in online meetings, shows, and online education scenarios, this version introduces the [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage) method to activate portrait center stage. This ensures that presenters, regardless of movement, always remain centered in the video frame, achieving better presentation effects. - Before enabling Center Stage, it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage). + Before enabling portrait center stage it is recommended to verify whether your current device supports this feature by calling [`isCameraCenterStageSupported`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_iscameracenterstagesupported). A list of supported devices can be found in the API documentation at [`enableCameraCenterStage`](/api-ref/rtc/macos/API/toc_center_stage#api_irtcengine_enablecameracenterstage). 3. **Data stream encryption** From 0e14ef691fce7d49cf907c5bc3cb144e24ba8d1a Mon Sep 17 00:00:00 2001 From: Suri539 Date: Wed, 24 Apr 2024 18:10:29 +0800 Subject: [PATCH 17/17] Update release_unity_ng.md --- .../release-notes/en-US/framework/release_unity_ng.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md index d5dc4fb83eb..d2180268bfd 100644 --- a/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md +++ b/markdown/RTC 4.x/release-notes/en-US/framework/release_unity_ng.md @@ -80,7 +80,7 @@ This version is released on 2024 Month x, Day x. 2. **Audio device type detection (macOS)** - This version adds the [GetPlaybackDefaultDevice [2/2]](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2), [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2], [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2], and [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] method to obtain the information and type of audio playback and recording deivces. + This version adds the [GetPlaybackDefaultDevice [2/2]](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdefaultdevice2), [`GetRecordingDefaultDevice`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdefaultdevice2)[2/2], [`GetPlaybackDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getplaybackdeviceinfo2)[2/2], and [`GetRecordingDeviceInfo`](/api-ref/rtc/unity/API/toc_audio_device#api_iaudiodevicemanager_getrecordingdeviceinfo2)[2/2] method to obtain the information and type of audio playback and recording devices. 3. **Virtual Background Algorithm Optimization** @@ -107,7 +107,14 @@ This version is released on 2024 Month x, Day x. This version fixed the following issues: -- The SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- Fixed an issue where SEI data output did not synchronize with video rendering when playing media streams containing SEI data using the media player. +- In screen sharing scenarios, when the app enabled sound card capture with [`EnableLoopbackRecording`](/api-ref/rtc/unity/API/toc_audio_capture#api_irtcengine_enableloopbackrecording) to capture audio from the shared screen, the transmission of sound card captured audio failed after a local user manually disabled the local audio capture device, causing remote users to not hear the shared screen's audio. (Windows) +- Broadcasters using certain models of devices under speaker mode experienced occasional local audio capture failures when switching the app process to the background and then back to the foreground, causing remote users to not hear the broadcaster's audio. (Android) +- An occasional echo was observed when playing the audio stream of a specified user before mixing. (macOS, Windows) +- During interactions, when a local user set the system default playback device to speakers using , there was no sound from the remote end. (Windows) +- On devices with Android 8.0, enabling screen sharing occasionally caused the app to crash. (Android) +- When sharing an Excel document window, remote users occasionally saw a green screen. (Windows) +- In scenarios using camera capture for local video, when the app was moved to the background and [`DisableVideo`](/api-ref/rtc/unity/API/toc_video_basic#api_irtcengine_disablevideo) or [`StopPreview`](/api-ref/rtc/unity/API/toc_video_basic#api_irtcengine_stoppreview)[1/2] was called to stop video capture, camera capture was unexpectedly activated when the app was brought back to the foreground. (Android) - When the network conditions of the sender deteriorated (for example, in poor network environments), the receiver occasionally experienced a decrease in video smoothness and an increase in lag. #### API Changes