-
Notifications
You must be signed in to change notification settings - Fork 15
Feat/spam digest job #905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Feat/spam digest job #905
Changes from 22 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
7e186fd
- Added SpamDigestGeneratorJob to aggregate spam reports (daily/weekly)
BarbaraOliveira13 f80fe0f
delete event_registration
BarbaraOliveira13 d86bf4c
fix methode error .organization because resource = organization
BarbaraOliveira13 1dbaefc
lint
BarbaraOliveira13 e36b010
lint i18 trad key and rollback resource_locator method
BarbaraOliveira13 2264031
run bundle exec i18n-tasks normalize --locales nl
BarbaraOliveira13 2e80e82
test file for spam digest event
BarbaraOliveira13 178f95a
delete link from the view on decidim-core in the digest summary spam …
BarbaraOliveira13 9028038
patch to stop instant spam mail from messages spams
BarbaraOliveira13 87aa452
stop instant spam mail via decidim-ai also when is a spam user report
BarbaraOliveira13 0886a57
lint spam_digest_event.rb
BarbaraOliveira13 590cc5c
delete unused trad key and clean mail text
BarbaraOliveira13 941c688
add user_spam part logic who was missing
BarbaraOliveira13 6dc2781
add link to moderations into email text
BarbaraOliveira13 4d61e46
add trad key access of organization name for the text mail
BarbaraOliveira13 7bf6b05
don't stop report process when frequency notification is realtime
BarbaraOliveira13 752ae80
Merge branch 'develop' into feat/decidim-ai-digest
BarbaraOliveira13 e08c545
fix CI
BarbaraOliveira13 9ef29b4
test file
BarbaraOliveira13 c472dfc
local from develop merged
BarbaraOliveira13 1d46b3a
normalise trad key
BarbaraOliveira13 03afaf2
Merge branch 'develop' into feat/decidim-ai-digest
luciegrau b53cd9b
change request: avoid js injection with sanitize
BarbaraOliveira13 3a18ccd
change request: avoid js injection with sanitize
BarbaraOliveira13 d960075
change request: refacto with translated attribute
BarbaraOliveira13 a9f80f1
change request: refacto count_spam method
BarbaraOliveira13 891ac3d
change request: .exists? method instead of .any who load all admins...
BarbaraOliveira13 b53fce5
Our resource here is Decidim::Organization but Decidim::Organization …
BarbaraOliveira13 0d96783
refactored the anonymous class into a Struct for simplicity. Removed …
BarbaraOliveira13 7c32feb
fix CI: refacto Struct by a real class with route_name method as expe…
BarbaraOliveira13 2fe3cec
change request: add raise error if frequency is invalid when run the…
BarbaraOliveira13 465363f
fix CI by adding a nil condition argument as expected by decidim-core…
BarbaraOliveira13 dc52e5c
fix the table path for spam report queries
BarbaraOliveira13 1753a33
refacto spam_user_reports_since method
BarbaraOliveira13 f58c191
Merge branch 'develop' into feat/decidim-ai-digest
AyakorK File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
127 changes: 127 additions & 0 deletions
127
app/events/decidim/ai/spam_detection/spam_digest_event.rb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module Decidim | ||
| module Ai | ||
| module SpamDetection | ||
| class SpamDigestEvent < Decidim::Events::BaseEvent | ||
| include Decidim::Events::EmailEvent | ||
|
|
||
| def self.types | ||
| [:email, :notification] | ||
| end | ||
|
|
||
| def resource | ||
| return @resource unless @resource.is_a?(Decidim::Organization) | ||
|
|
||
| OpenStruct.new(organization: @resource) | ||
| end | ||
|
|
||
| def email_intro | ||
| org_name = | ||
| organization.name[I18n.locale.to_s].presence || | ||
| organization.name.dig("machine_translations", I18n.locale.to_s).presence || | ||
| organization.name["en"].presence || | ||
| organization.name.values.compact.first | ||
|
|
||
| I18n.t( | ||
| "decidim.ai.spam_detection.digest.summary", | ||
| count: spam_count, | ||
| frequency_label:, | ||
| organization: org_name, | ||
| moderations_url: | ||
| ).html_safe | ||
BarbaraOliveira13 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| end | ||
|
|
||
| def notification_title | ||
| email_intro | ||
| end | ||
|
|
||
| def email_subject | ||
| I18n.t( | ||
| "decidim.ai.spam_detection.digest.subject", | ||
| count: spam_count, | ||
| frequency_label: | ||
| ) | ||
| end | ||
|
|
||
| def resource_title | ||
| organization.name[I18n.locale.to_s].presence || | ||
| organization.name.dig("machine_translations", I18n.locale.to_s).presence || | ||
| organization.name["en"].presence || | ||
| organization.name.values.compact.first | ||
| end | ||
|
|
||
| def resource_locator | ||
| helpers = Decidim::Core::Engine.routes.url_helpers | ||
| host = organization.host || Decidim::Organization.first&.host || "localhost" | ||
|
|
||
| Class.new do | ||
| def initialize(path, url) | ||
| @path = path | ||
| @url = url | ||
| end | ||
|
|
||
| def path(_params = nil) | ||
| @path | ||
| end | ||
|
|
||
| def url(_params = nil) | ||
| @url | ||
| end | ||
| end.new( | ||
| resource_path, | ||
| helpers.root_url(host:, protocol: "http") | ||
AyakorK marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ) | ||
| end | ||
BarbaraOliveira13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| def resource_path(_organization = nil) | ||
| Decidim::Core::Engine.routes.url_helpers.admin_moderations_path | ||
| rescue NoMethodError | ||
| Decidim::Core::Engine.routes.url_helpers.root_path | ||
| end | ||
|
|
||
| def show_extended_information? | ||
| false | ||
| end | ||
|
|
||
| private | ||
|
|
||
| def moderations_url | ||
| host = organization.host | ||
|
|
||
| if host.blank? | ||
| return "" unless Rails.env.development? || Rails.env.test? | ||
|
|
||
| host = "localhost:3000" | ||
| elsif host == "localhost" && (Rails.env.development? || Rails.env.test?) | ||
| host = "localhost:3000" | ||
| end | ||
|
|
||
| protocol = Rails.env.production? ? "https" : "http" | ||
|
|
||
| "#{protocol}://#{host}/admin/moderations" | ||
| end | ||
|
|
||
| def organization | ||
| if @resource.is_a?(Decidim::Organization) | ||
| @resource | ||
| elsif @resource.respond_to?(:organization) | ||
| @resource.organization | ||
| elsif @resource.respond_to?(:component) | ||
| @resource.component.participatory_space.organization | ||
| else | ||
| Decidim::Organization.first | ||
| end | ||
| end | ||
|
|
||
| def spam_count | ||
| extra[:spam_count] || 0 | ||
| end | ||
|
|
||
| def frequency_label | ||
| I18n.t("decidim.ai.spam_detection.digest.frequency_label.#{extra[:frequency] || "daily"}") | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end | ||
89 changes: 89 additions & 0 deletions
89
app/jobs/decidim/ai/spam_detection/spam_digest_generator_job.rb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| module Decidim | ||
| module Ai | ||
| module SpamDetection | ||
| # This job generates and publishes the AI spam digest event | ||
| # for each organization, either daily or weekly. | ||
| class SpamDigestGeneratorJob < ApplicationJob | ||
| queue_as :mailers | ||
|
|
||
| def perform(frequency) | ||
BarbaraOliveira13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Decidim::Organization.find_each do |organization| | ||
| admins = organization.admins.where(notifications_sending_frequency: frequency) | ||
| next if admins.empty? | ||
|
|
||
| spam_count = count_spam(organization, frequency) | ||
BarbaraOliveira13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| next if spam_count.zero? | ||
|
|
||
| Decidim::EventsManager.publish( | ||
| event: "decidim.events.ai.spam_detection.spam_digest_event", | ||
| event_class: Decidim::Ai::SpamDetection::SpamDigestEvent, | ||
| resource: organization, | ||
| followers: admins, | ||
| extra: { spam_count:, frequency:, force_email: true } | ||
| ) | ||
| end | ||
| end | ||
|
|
||
| private | ||
|
|
||
| # Counts the spam reports for the given organization and frequency (daily/weekly) | ||
| def count_spam(organization, frequency) | ||
| since = frequency == :weekly ? 1.week.ago : 1.day.ago | ||
|
|
||
| general_spams = spam_reports_since(since).count do |report| | ||
| report_belongs_to_org?(report, organization) | ||
| end | ||
|
|
||
| user_spams = spam_user_reports_since(since).where(decidim_users: { decidim_organization_id: organization.id }).count | ||
|
|
||
| user_spams + general_spams | ||
| end | ||
|
|
||
| # Returns all spam reports created since the given time | ||
| def spam_reports_since(since) | ||
| Decidim::Report | ||
| .joins(:moderation) | ||
| .where(reason: "spam") | ||
| .where("decidim_reports.created_at >= ?", since) | ||
| end | ||
|
|
||
| def spam_user_reports_since(since) | ||
| Decidim::UserReport | ||
| .joins(:user) | ||
| .where(reason: "spam") | ||
| .where("decidim_user_reports.created_at >= ?", since) | ||
| end | ||
BarbaraOliveira13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # Determines if a spam report belongs to the given organization | ||
| def report_belongs_to_org?(report, organization) | ||
| reportable = report.moderation.reportable | ||
|
|
||
| participatory_space = find_participatory_space(reportable) | ||
| return false unless participatory_space | ||
|
|
||
| org_id = participatory_space.try(:decidim_organization_id) || participatory_space.try(:organization_id) | ||
| org_id == organization.id | ||
| rescue StandardError => e | ||
| Rails.logger.debug do | ||
| "[Decidim-AI] ⚠️ Could not resolve organization for report ##{report.id}: #{e.class} #{e.message}" | ||
| end | ||
| false | ||
| end | ||
|
|
||
| # Finds the participatory space for a given reportable entity | ||
| def find_participatory_space(reportable) | ||
| if reportable.respond_to?(:component) | ||
| reportable.component.participatory_space | ||
| elsif reportable.respond_to?(:commentable) | ||
| commentable = reportable.commentable | ||
| commentable.try(:component)&.participatory_space | ||
| elsif reportable.respond_to?(:participatory_space) | ||
| reportable.participatory_space | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end | ||
| end | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| # frozen_string_literal: true | ||
|
|
||
| # Patch to stop instant spam report emails from Decidim-AI when frequency is daily, weekly | ||
| # Allows the Spam Summary Digest to manage notifications (daily, weekly) | ||
| # instead of non-configurable real-time emails from Decidim-ai report | ||
|
|
||
| module Decidim | ||
| module InstantSpamMailBlocker | ||
| def send_report_notification_to_moderators | ||
| return if spam_report? && !frequency_notifications_is_realtime?(@report.moderation.participatory_space.organization.admins) | ||
|
|
||
| super | ||
| end | ||
|
|
||
| def send_notification_to_admins! | ||
| return if spam_report? && !frequency_notifications_is_realtime?(@report.moderation.user.organization.admins) | ||
BarbaraOliveira13 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| super | ||
| end | ||
|
|
||
| private | ||
|
|
||
| def spam_report? | ||
| @report.reason.to_s == "spam" | ||
| end | ||
|
|
||
| def frequency_notifications_is_realtime?(admins) | ||
| admins.any? { |a| a.notifications_sending_frequency == "realtime" } | ||
BarbaraOliveira13 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| end | ||
| end | ||
| end | ||
|
|
||
| Rails.application.config.to_prepare do | ||
| Decidim::CreateReport.prepend(Decidim::InstantSpamMailBlocker) | ||
| Decidim::CreateUserReport.prepend(Decidim::InstantSpamMailBlocker) | ||
| end | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.