Getting reference to realm object that may or may not exist

[davidkessler-ch]

How frequently does the bug occur?

Always

Description

I have a RealmSwift object User, I want to simply load the object into memory by it's primary key. If no object with that primary key exists, I want to create one and get it back as well. I have a concurrent write that modifies the same object (same primary key).

I have written the following function:

func fetchOrCreateUser(forPrimaryKey: String) async throws -> User {
       let realm = try await Realm()
        let user = try await realm.asyncWrite {
            let user = realm.object(ofType: User.self, forPrimaryKey: forPrimaryKey)
            if let user = user {
                return user
            } else {
                let newObject = User(userId: forPrimaryKey)// -> Problem!
        // for some reason this still overwrites users,
        // so these users must get written between this and the line before..
        // Isn't the realm supposed to be locked inside this write transaction?
                realm.add(newObject)
                return newObject
            }
        }
        return user.freeze()
    }

Independently User objects get populated with data by using:

func write(object: User) async throws {        
        let realm = try await Realm()
        try await realm.asyncWrite {
            realm.create(type(of: object), value: object, update: .modified)
        }
    }

However, in our application (where there is concurrency) users get independently populated. The code above results in User objects that are populated with data (using the write(...) above) being overwritten, only leaving their "userId" property set. This seems like the realm doesn't get properly locked during the "asyncWrite" transaction.
In my understanding, it does not matter in what order I call the two functions, since the documentation on asyncWrite states:

"... With the writeAsync() API, waiting to obtain the write lock and committing a transaction occur in the background ..."

So the write lock should still happen (everything else would also be odd, since race conditions would easily happen...)
And further the docs state:

" [...], while the write block itself is executed, this does block new transactions on the calling thread. ..."

So clearly, both of the asyncWrites in my case should run after each other, in whatever order. Both of these should be running on the same thread, namely the main thread...

---

This is an example of data being lost by this issue:

let senderUserId = "usr1"
Task {
    var senderUserTest = try await fetchOrCreateUser(forPrimaryKey: senderUserId)
}
Task {
    let newUser = User(userId: senderUserId, userName: "blaaabo")
    try await write(object: newUser)
}

try await Task.sleep(seconds: 3)

// if one loads the user objects now, the name "blaaabo" is not in the User object with id "usr1"

// note: this works fine without the two Task {}, but simply in succession...

---

If I use write instead of asyncWrite I get the error

Thread 1: "The Realm is already in a write transaction"

Stacktrace & log output

The crash when using `write` gives;

*** Terminating app due to uncaught exception 'RLMException', reason: 'The Realm is already in a write transaction'

*** First throw call stack:
(
	0   CoreFoundation                      0x00000001804658a8 __exceptionPreprocess + 172
	1   libobjc.A.dylib                     0x000000018005c09c objc_exception_throw + 56
	2   LSN-ios-2                           0x0000000104120774 _Z18RLMSetErrorOrThrowP7NSErrorPU15__autoreleasingS0_ + 336
	3   LSN-ios-2                           0x000000010408b71c _Z26RLMRealmTranslateExceptionPU15__autoreleasingP7NSError + 296
	4   LSN-ios-2                           0x00000001040902cc -[RLMRealm beginWriteTransactionWithError:] + 140
	5   LSN-ios-2                           0x0000000104090234 -[RLMRealm beginWriteTransaction] + 52
	6   LSN-ios-2                           0x00000001042084dc $s10RealmSwift0A0V10beginWriteyyF + 64
	7   LSN-ios-2                           0x00000001042082f0 $s10RealmSwift0A0V5write16withoutNotifying_xSaySo20RLMNotificationTokenCG_xyKXEtKlF + 212
	8   LSN-ios-2                           0x0000000102d34c98 $s9LSN_ios_218PersistanceServiceC5write6objectyAA9LSNObject_p_tYaKFTY2_ + 220
	9   LSN-ios-2                           0x0000000102d88751 $s9LSN_ios_218DataRoutingServiceC26didReceiveObjectFromServer5event6objectyAA13IncomingEventO_Se_ptYaKFTQ9_ + 1
	10  LSN-ios-2                           0x0000000102cc86e9 $s9LSN_ios_213ServerServiceC18handleReceivedData4data5eventySayypG_AA13IncomingEventOtYaKFTQ1_ + 1
	11  LSN-ios-2                           0x0000000102cc7541 $s9LSN_ios_213ServerServiceC14handleResponse4data3ack5eventySayypG_8SocketIO0J10AckEmitterCAA13IncomingEventOtYaFTQ1_ + 1
	12  LSN-ios-2                           0x0000000102cc72d5 $s9LSN_ios_213ServerServiceC26addObjectReceivingHandlersyyFySayypG_8SocketIO0I10AckEmitterCtcfU1_ytSgyYaYbcfU_TQ1_ + 1
	13  LSN-ios-2                           0x0000000102ccbf39 $s9LSN_ios_213ServerServiceC26addObjectReceivingHandlersyyFySayypG_8SocketIO0I10AckEmitterCtcfU1_ytSgyYaYbcfU_TATQ0_ + 1
	14  LSN-ios-2                           0x0000000102c97b35 $sxIeghHr_xs5Error_pIegHrzo_s8SendableRzs5NeverORs_r0_lTRTQ0_ + 1
	15  LSN-ios-2                           0x0000000102c97c91 $sxIeghHr_xs5Error_pIegHrzo_s8SendableRzs5NeverORs_r0_lTRTATQ0_ + 1
	16  libswift_Concurrency.dylib          0x00000001b6a539c5 _ZL23completeTaskWithClosurePN5swift12AsyncContextEPNS_10SwiftErrorE + 1
)
libc++abi: terminating due to uncaught exception of type NSException

### Can you reproduce the bug?

Always

### Reproduction Steps

Implement above functions, call them using `Task { ... }` independently. (In our case they are called by independent websocket events)
```swift
let senderUserId = "usr1"
Task {
    var senderUserTest = try await fetchOrCreateUser(forPrimaryKey: senderUserId)
}
Task {
    let newUser = User(userId: senderUserId, userName: "blaaabo")
    try await write(object: newUser)
}

Version

10.44

What Atlas Services are you using?

Local Database only

Are you using encryption?

No

Platform OS and version(s)

iOS 16, 17 and potentially other

Build environment

Xcode version: Version 15.0 (15A240d)
Dependency manager and version: SPM

[davidkessler-ch]

The following implementation using write works, but I would like to do this using asyncWrite.

func fetchOrCreateUser(forPrimaryKey: String) async throws -> User? {
        let user = try await MainActor.run {
            let realm = try Realm()
            let user = try realm.write {
                let user = realm.object(ofType: User.self, forPrimaryKey: forPrimaryKey)
                if let user = user {
                    return user
                } else {
                    let newObject = User(value: ["userId": forPrimaryKey])
                    realm.add(newObject, update: .modified)
                    return realm.object(ofType: User.self, forPrimaryKey: forPrimaryKey)!
                }
            }
            return user
        }
        return user.freeze()
}

and

func write(object: User) async throws {
        try await MainActor.run {
            let realm = try Realm()
            try realm.write {
                realm.create(User.self, value: object, update: .modified) 
            }
        }
    }

and everywhere else accessing realm only using

try await MainActor.run {
            let realm = try Realm()
            // do something
}

[Jaycyn]

There may be some confusion due to how Realm named these functions: The question states

In my understanding, it does not matter in what order I call the two functions, since the documentation on asyncWrite states:

"... With the writeAsync() API, waiting to obtain the write lock and committing a transaction occur in the background ..."

It should be noted that asyncWrite and writeAsync are two different functions.

The writeAsync() API allows for performing async writes using Swift completion handlers.

The asyncWrite() API allows for performing async writes using Swift async/await syntax.

The asyncWrite() API suspends the calling task while waiting for its turn to write rather than blocking the thread. In addition, the actual I/O to write data to disk is done by a background worker thread.

Just for clarity.

[davidkessler-ch]

You are totally right, sorry for writing about writeAsync() as well, I am only concerned with asyncWrite() here, since I call only using the async/await syntax.

[davidkessler-ch]

One more thing I just tried:

I made a global actor like so:

@globalActor
actor RealmBackgroundActor {
    static var shared = RealmBackgroundActor()
}

And I marked the class in which I call functions (and the functions itself, just to be sure) with @RealmBackgroundActor. I then used only Realm(actor: RealmBackgroundActor.shared) which in my understanding of the documentation, should eliminate any data races, as all access to the underlying data should happen on this same actor. The functions then look like:

@RealmBackgroundActor func fetchOrCreateUser(forPrimaryKey: String) async throws -> User? {
        let realm = try await Realm(actor: RealmBackgroundActor.shared)
        
        let user = try await realm.asyncWrite {
            let user = realm.object(ofType: User.self, forPrimaryKey: forPrimaryKey)
            if let user = user {
                return user
            } else {
                let newObject = User(value: ["userId": forPrimaryKey])
                realm.add(newObject, update: .modified)
                return newObject
            }
        }
        return user.freeze()
}

and

@RealmBackgroundActor func write(object: LSNObject) async throws {
        let realm = try await Realm(actor: RealmBackgroundActor.shared)
        try await realm.asyncWrite {
            realm.create(type(of: object), value: object, update: .modified)
        }
    }

But still, the user objects gets overwritten (i.e. the userName lost) if two Tasks are created like so:

let senderUserId = "usr1"

Task {
    let senderUserTest = try await fetchOrCreateUser(forPrimaryKey: senderUserId)
    // user is needed for something
}

Task {
    let newUser = User(userId: senderUserId, userName: "THISGETSLOST")
    try await write(object: newUser)
}