Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharded query with where.not running map/each goes to master shard #515

Open
MrBusch opened this issue Feb 22, 2019 · 0 comments
Open

Sharded query with where.not running map/each goes to master shard #515

MrBusch opened this issue Feb 22, 2019 · 0 comments

Comments

@MrBusch
Copy link

MrBusch commented Feb 22, 2019

I've run into a query not returning the expected result and finally got it down to this minimum working example:

class Testquery
    def self.get_total(id)
        calendar_days = CalendarDay.using(:shard)\
                                   .where(id: id)\
                                   .where.not(price: nil)
        puts "Got #{calendar_days.count} entries"
        total = 0.0
        calendar_days.map { |cd|
            total += cd.price
        }
        puts total
    end

    def self.get_total2(id)
        calendar_days = CalendarDay.using(:shard)\
                                   .where(id: id)\
                                   .where('price IS NOT NULL')
        puts "Got #{calendar_days.count} entries"
        total = 0.0
        calendar_days.map { |cd|
            total += cd.price
        }
        puts total
    end
end
> ActiveRecord::Base.logger = Logger.new(STDOUT)
> Testquery.get_total(id)
Fetching calendar_days
D, [2019-02-22T18:41:19.231971 #20445] DEBUG -- : [Shard: shard]   (1.2ms)  SELECT COUNT(*) FROM "calendar_days" WHERE "calendar_days"."id" = $1 AND "calendar_days"."price" IS NOT NULL  [["id", 564815]]
Got 1929 entries: CalendarDay::ActiveRecord_Relation
D, [2019-02-22T18:41:19.232863 #20445] DEBUG -- : [Shard: master]  CalendarDay Load (0.5ms)  SELECT "calendar_days".* FROM "calendar_days" WHERE "calendar_days"."id" = $1 AND "calendar_days"."price" IS NOT NULL  [["id", 564815]]
0.0
=> nil
> Testquery.get_total2(id)
Fetching calendar_days
D, [2019-02-22T18:41:23.594372 #20445] DEBUG -- : [Shard: shard]   (1.2ms)  SELECT COUNT(*) FROM "calendar_days" WHERE "calendar_days"."id" = $1 AND (price IS NOT NULL)  [["id", 564815]]
Got 1929 entries: CalendarDay::ActiveRecord_Relation
D, [2019-02-22T18:41:23.600031 #20445] DEBUG -- : [Shard: shard]  CalendarDay Load (5.2ms)  SELECT "calendar_days".* FROM "calendar_days" WHERE "calendar_days"."id" = $1 AND (price IS NOT NULL)  [["id", 564815]]
312784.0
=> nil

The difference between the two queries is that the first uses the ActiveRecord way of where.not(price: nil) while the second uses where('price IS NOT NULL'). If I use where.not, the evaluation when invoking map goes to :master even though a shard is specified. Contrary to that the prior call to count correctly goes to the shard. I also tested each in place of map and both had the same result.

So the combination of defining a non-master shard, querying with where.not and then running map/each does not evaluate the query to the correct shard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant