fs_inode:Change the type of i_crefs to atomic_int #13443

crafcat7 · 2024-09-14T03:39:52Z

Summary

Benefit from PR#13044, we can change the the inode->i_crefs to atomic. That we can avoid problems:
Deadlocks caused by the server requesting reads from the client and the client requesting writes from the server when operating a file system with multiple cores (e.g. RPMSGFS).

The main changes are as follows:
1.Modified the i_crefs from int16_t to atomic_int
2.Modified the i_crefs add, delete, read, and initialize interfaces to atomic operations

The main change of this PR is to change the i_crefs count of inode to atomic type and use atomic operations to maintain i_crefs.

Impact

Impact on user: NO, This is a change within VFS and should be transparent to external developers.
Impact on build: YES, This may affect compilation because the members of the inode structure have changed. Different compilers/platforms have different support for atomic, so some warnings are skipped.
Impact on compatibility: This implementation aims to be backward compatible. No existing functionality is expected to be broken.

Testing

Build Host(s): Linux x86
Target(s): sim/nsh

Basic operations on the file system can work normally

anchao · 2024-09-14T03:57:49Z

Please check that all i_crefs are replaced correctly, eg:
https://github.com/apache/nuttx/blob/master/drivers/serial/pty.c#L335

arch/arm/src/cxd56xx/cxd56_sph.c

anchao · 2024-09-14T06:26:41Z

Ditto, please make sure to go through the code carefully
https://github.com/apache/nuttx/blob/master/fs/vfs/fs_dir.c#L83

crafcat7 · 2024-09-14T09:17:02Z

Please check that all i_crefs are replaced correctly, eg: https://github.com/apache/nuttx/blob/master/drivers/serial/pty.c#L335

Done

Ditto, please make sure to go through the code carefully https://github.com/apache/nuttx/blob/master/fs/vfs/fs_dir.c#L83

Done

include/nuttx/atomic.h

crafcat7 · 2024-09-18T02:57:32Z

I found that some platforms have the following warning when using different compilers

====================================================================================
Configuration/Tool: samd20-xplained/nsh,CONFIG_ARM_TOOLCHAIN_CLANG
2024-09-14 17:29:29
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI
  Enabling CONFIG_ARM_TOOLCHAIN_CLANG
  Building NuttX...
Error: inode/fs_inodeaddref.c:50:7: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0  bytes) [-Werror,-Watomic-alignment]
   50 |       atomic_fetch_add(&inode->i_crefs, 1);

In the current strategy, if the compiler itself does not support Atomic, Nuttx Base Atomic (low-performance version) will be used. So can we ignore it?

Summary: 1.Modified the i_crefs from int16_t to atomic_int 2.Modified the i_crefs add, delete, read, and initialize interfaces to atomic operations The purpose of this change is to avoid deadlock in cross-core scenarios, where A Core blocks B Core’s request for a write operation to A Core when A Core requests a read operation to B Core. Signed-off-by: chenrun1 <[email protected]>

Error: vfs/fs_epoll.c:126:3: error: macro 'ATOMIC_VAR_INIT' has been marked as deprecated [-Werror,-Wdeprecated-pragma] ATOMIC_VAR_INIT(1), /* i_crefs */ ^ /Applications/Xcode_15.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/stdatomic.h:54:41: note: macro marked 'deprecated' here ^ 1 error generated. make[1]: *** [fs_epoll.o] Error 1 Error: socket/socket.c:78:3: error: macro 'ATOMIC_VAR_INIT' has been marked as deprecated [-Werror,-Wdeprecated-pragma] ATOMIC_VAR_INIT(1), /* i_crefs */ Signed-off-by: chenrun1 <[email protected]>

…upport it Summary: Configuration/Tool: zp214xpa/nsh,CONFIG_ARM_TOOLCHAIN_GNU_EABI 2024-09-14 06:11:00 ------------------------------------------------------------------------------------ Cleaning... Configuring... Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Building NuttX... arm-none-eabi-ld: /github/workspace/sources/nuttx/staging/libfs.a(fs_inoderemove.o): in function `inode_remove': fs_inoderemove.c:(.text.inode_remove+0x94): undefined reference to `__sync_synchronize' arm-none-eabi-ld: fs_inoderemove.c:(.text.inode_remove+0x9c): undefined reference to `__sync_synchronize' make[1]: *** [Makefile:212: nuttx] Error 1 make: *** [tools/Unix.mk:548: nuttx] Error 2 make: Target 'all' not remade because of errors. Signed-off-by: chenrun1 <[email protected]>

arch/arm/src/tlsr82/Toolchain.defs

When the toolchain does not support atomic, it will use the version implemented by NuttX (low performance version). This scenario is consistent with the original design, so we can ignore it. see bug here: https://bugs.llvm.org/show_bug.cgi?id=43603 Error: inode/fs_inodeaddref.c:50:7: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment] 50 | atomic_fetch_add(&inode->i_crefs, 1); | ^ /tools/clang-arm-none-eabi/lib/clang/17/include/stdatomic.h:152:43: note: expanded from macro 'atomic_fetch_add' 152 | #define atomic_fetch_add(object, operand) __c11_atomic_fetch_add(object, operand, __ATOMIC_SEQ_CST) | ^ 1 error generated. make[1]: *** [Makefile:83: fs_inodeaddref.o] Error 1 Error: inode/fs_inodefind.c:74:7: error: large atomic operation may incur significant performance penalty; the access size (4 bytes) exceeds the max lock-free size (0 bytes) [-Werror,-Watomic-alignment] 74 | atomic_fetch_add(&node->i_crefs, 1); Signed-off-by: chenrun1 <[email protected]>

xiaoxiang781216 · 2024-09-21T15:51:33Z

include/nuttx/atomic.h

@@ -90,6 +90,12 @@ extern "C++"
 #  include <nuttx/lib/stdatomic.h>
 #endif

+#if !(__clang__)


xiaoxiang781216 · 2024-09-21T15:51:53Z

fs/vfs/fs_timerfd.c

@@ -123,7 +123,7 @@ static struct inode g_timerfd_inode =
  NULL,                   /* i_parent */
  NULL,                   /* i_peer */
  NULL,                   /* i_child */
-  ATOMIC_VAR_INIT(1),     /* i_crefs */
+  ATOMIC_INIT(1),         /* i_crefs */


remove ATOMIC_VAR_INIT/ATOMIC_INIT

xiaoxiang781216 · 2024-09-21T15:54:05Z

include/nuttx/fs/fs.h

@@ -410,7 +411,7 @@ struct inode
  FAR struct inode *i_parent;   /* Link to parent level inode */
  FAR struct inode *i_peer;     /* Link to same level inode */
  FAR struct inode *i_child;    /* Link to lower level inode */
-  int16_t           i_crefs;    /* References to inode */
+  atomic_int        i_crefs;    /* References to inode */


atomic_short

xiaoxiang781216 · 2024-09-21T15:59:18Z

fs/shm/shmfs.c

@@ -185,21 +185,15 @@ static int shmfs_release(FAR struct inode *inode)
   * The inode is released after this call, hence checking if i_crefs <= 1.
   */

-  int ret = inode_lock();
-  if (ret >= 0)
+  inode_lock();


remove node_lock

xiaoxiang781216 · 2024-09-21T15:59:50Z

fs/shm/shmfs.c

-  int ret = inode_lock();
-
-  if (ret >= 0)
+  inode_lock();


remove inode_lock

xiaoxiang781216 · 2024-09-21T16:00:08Z

fs/vfs/fs_pseudofile.c

@@ -353,26 +353,23 @@ static int pseudofile_munmap(FAR struct task_group_s *group,
   * The inode is released after this call, hence checking if i_crefs <= 1.
   */

-  int ret = inode_lock();
-  if (ret >= 0)
+  inode_lock();


crafcat7 force-pushed the inode_atomic branch from 2eb25f6 to 2f854a4 Compare September 14, 2024 03:44

crafcat7 force-pushed the inode_atomic branch from 2f854a4 to 24170b6 Compare September 14, 2024 04:11

anchao reviewed Sep 14, 2024

View reviewed changes

arch/arm/src/cxd56xx/cxd56_sph.c Outdated Show resolved Hide resolved

crafcat7 force-pushed the inode_atomic branch from 24170b6 to ca181c6 Compare September 14, 2024 05:55

crafcat7 force-pushed the inode_atomic branch 2 times, most recently from acadd16 to e5535a7 Compare September 14, 2024 09:12

crafcat7 force-pushed the inode_atomic branch 4 times, most recently from 75fbe24 to b291fb2 Compare September 14, 2024 16:33

xiaoxiang781216 reviewed Sep 16, 2024

View reviewed changes

include/nuttx/atomic.h Outdated Show resolved Hide resolved

crafcat7 force-pushed the inode_atomic branch 3 times, most recently from c720470 to 4911fee Compare September 18, 2024 02:53

crafcat7 force-pushed the inode_atomic branch from 4911fee to 236d671 Compare September 18, 2024 03:00

crafcat7 force-pushed the inode_atomic branch from 236d671 to 3b6c346 Compare September 18, 2024 03:04

crafcat7 added 2 commits September 18, 2024 11:20

crafcat7 force-pushed the inode_atomic branch 7 times, most recently from e90852e to 6b05d09 Compare September 18, 2024 17:23

xiaoxiang781216 reviewed Sep 18, 2024

View reviewed changes

arch/arm/src/tlsr82/Toolchain.defs Outdated Show resolved Hide resolved

crafcat7 force-pushed the inode_atomic branch 2 times, most recently from 607130d to cf85911 Compare September 20, 2024 02:36

github-actions bot added the Size: M The size of the change in this PR is medium label Sep 20, 2024

crafcat7 force-pushed the inode_atomic branch 2 times, most recently from d501da2 to 22ae464 Compare September 20, 2024 09:13

crafcat7 force-pushed the inode_atomic branch from 22ae464 to bfc7828 Compare September 21, 2024 13:00

github-actions bot added Arch: arm Issues related to ARM (32-bit) architecture Area: File System File System issues Area: OS Components OS Components issues Area: Crypto labels Sep 21, 2024

xiaoxiang781216 reviewed Sep 21, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fs_inode:Change the type of i_crefs to atomic_int #13443

fs_inode:Change the type of i_crefs to atomic_int #13443

crafcat7 commented Sep 14, 2024 •

edited

Loading

anchao commented Sep 14, 2024

anchao commented Sep 14, 2024

crafcat7 commented Sep 14, 2024

crafcat7 commented Sep 18, 2024

xiaoxiang781216 Sep 21, 2024

xiaoxiang781216 Sep 21, 2024

xiaoxiang781216 Sep 21, 2024

xiaoxiang781216 Sep 21, 2024

xiaoxiang781216 Sep 21, 2024

xiaoxiang781216 Sep 21, 2024

fs_inode:Change the type of i_crefs to atomic_int #13443

Are you sure you want to change the base?

fs_inode:Change the type of i_crefs to atomic_int #13443

Conversation

crafcat7 commented Sep 14, 2024 • edited Loading

Summary

Impact

Testing

anchao commented Sep 14, 2024

anchao commented Sep 14, 2024

crafcat7 commented Sep 14, 2024

crafcat7 commented Sep 18, 2024

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

xiaoxiang781216 Sep 21, 2024

Choose a reason for hiding this comment

crafcat7 commented Sep 14, 2024 •

edited

Loading