Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(operation): Introducing operation entity and thread safe operation queue. #16

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

bisakhmondal
Copy link
Member

Features

  • Entity Operation Schema
  • Thread-safe operation store.
  • Removing logic to sort counter based on hostID (merely for identification purpose, the operation should be independent of hostid and solely dependent on counter and dependency array).

TODO: tests

issue: #10

@bisakhmondal bisakhmondal added the enhancement New feature or request label Oct 30, 2021
@codecov-commenter
Copy link

Codecov Report

Merging #16 (14871c3) into main (ae6941f) will decrease coverage by 3.10%.
The diff coverage is 33.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
- Coverage   76.16%   73.05%   -3.11%     
==========================================
  Files           6        6              
  Lines         193      193              
==========================================
- Hits          147      141       -6     
- Misses         39       45       +6     
  Partials        7        7              
Impacted Files Coverage Δ
operations/lamport/clock.go 73.91% <33.33%> (-26.09%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ae6941f...14871c3. Read the comment docs.


// Operation - individual operation entity
Operation struct {
Id *lamport.Clock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we using pointers here?

Copy link
Member Author

@bisakhmondal bisakhmondal Oct 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To prevent it from unnecessary being copied and to have the atomicity while checking and updating the counter value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at operation level the clock is fixed as is used as an id so no updates.
Suggestion:
don't use pointers here

rest looks good

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It hardly makes any difference. If you insist - value then.

@noob77777 noob77777 linked an issue Oct 30, 2021 that may be closed by this pull request
@athena-crdt athena-crdt deleted a comment from noob77777 Oct 30, 2021
if atomic.LoadUint64(&obj.counter) == atomic.LoadUint64(&clock.counter) {
return clock.hostId < obj.hostId
} else {
return atomic.LoadUint64(&clock.counter) < atomic.LoadUint64(&obj.counter)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

revert

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(counter, hostId) defines total ordering of operations across all replicas

Copy link
Member Author

@bisakhmondal bisakhmondal Oct 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just think logically and tell me, how does it differ when the two operations happing at two different hosts have the same Lamport counter. The ordering of the host id should matter? No. Either the host id is auto-generated at init or randomly given during bootstrapping. So why should we take that into consideration inside the programming logic? Just keep it for the identification purpose - like where the counter was set.

Copy link
Member

@noob77777 noob77777 Oct 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To resolve concurrent operations, For LWW to work we will need someone to win.

If a replica receives an operation with a counter value c
that is greater than the locally stored counter value, the local
counter is increased to the value of the incoming counter.
This ensures that if operation o1 causally happened before
o2 (that is, the replica that generated o2 had received and
processed o1 before o2 was generated), then o2 must have
a greater counter value than o1. Only concurrent operations
can have equal counter values.
We can thus define a total ordering < for Lamport
timestamps:
(c1, p1) < (c2, p2) iff (c1 < c2) ∨(c1 = c2 ∧p1 < p2).
If one operation happened before another, this ordering is
consistent with causality (the earlier operation has a lower
timestamp). If two operations are concurrent, their order
according to < is arbitrary but deterministic. This ordering
property is important for our definition

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I understand. We need to think about this constructively...instead of in such a destructive way like the last writer wins. As CRDT is all about resolving conflict optimistically : )
The topic for discussion on next meet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if both have random ids which are coincidentally equal. We'll end up introducing randomness into the system

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each replica must have a unique id

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can't ensure must. Instead we could focus here

Another frequently-used approach to conflict resolution
is last writer wins (LWW), which arbitrarily chooses one
among several concurrent writes as “winner” and discards
the others. LWW is used in Apache Cassandra, for example.
It does not meet our requirements, since we want no user
input to be lost due to concurrent modifications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

⏩ type defintations required for Operations and Cursor
3 participants