Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

java.util.concurrent.ScheduledThreadPoolExecutor related to S3 input resulting in all pipeline logstash failure. #247

Open
agrant1 opened this issue Mar 2, 2023 · 1 comment
Open

java.util.concurrent.ScheduledThreadPoolExecutor related to S3 input resulting in all pipeline logstash failure. #247

agrant1 opened this issue Mar 2, 2023 · 1 comment
Labels
bug

Comments

@agrant1
Copy link

agrant1 commented Mar 2, 2023

Logstash information:

Please include the following information:

  1. Logstash version (e.g. bin/logstash --version)
    Using bundled JDK: /usr/share/logstash/jdk
    logstash 8.5.1
  2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
    RPM via yum
  3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
    systemd
  4. How was the Logstash Plugin installed
    Via RPM

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
  3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system):

Description of the problem including expected versus actual behavior:
S3 input causes a complete failure of logstash. Each S3 bucket is configured in a different pipeline, there are thousands of files in some of the s3 buckets.

The issue occurs randomly as far as I can tell. When it occurs the below error shows, logstash will never recover, and the only solution is to restart the service. The error will repeat over and over for each s3 input pipeline however, it is not limited to only an s3 input failure. All other pipelines silently fail with no error and the built in api in logstash do not return data.
pipeline.workers: use in the pipelines.yml seems to limit how often it occurs. However, this is an app breaking issue.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

I do not have a consistent way to reproduce this. The logs do not show anything out of the ordinary even in debug before it occurs.

Example input config:

input {
        s3{
        access_key_id => "<REDACTED>"
        secret_access_key => "${secret}"
        bucket => "<REDACTED>"
        region => "ca-central-1"
        prefix => "<REDACTED>
        "exclude_pattern" => "(\/202[0-1]\/|\/2022\/0\d\/|\/processing\-failed\/)"
        additional_settings => {
          force_path_style => true
          follow_redirects => true
          }
        }
}

Provide logs (if relevant):

[2023-03-02T16:16:33,530][ERROR][logstash.javapipeline    ][pipeline-1][52b512a94aa93bce1b8fc1cbaa5515accf8367fd3435706fa260182bb4dea35b] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:pipeline-1
  Plugin: <LogStash::Inputs::S3 access_key_id=>"<REDACTED>", bucket=>"<REDACTED>", additional_settings=>{"force_path_style"=>"true", "follow_redirects"=>"true"}, prefix=>"<REDACTED>", secret_access_key=><password>, exclude_pattern=>"(\\/202[0-1]\\/|\\/2022\\/0\\d\\/|\\/processing\\-failed\\/)", id=>"52b512a94aa93bce1b8fc1cbaa5515accf8367fd3435706fa260182bb4dea35b", region=>"ca-central-1", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_6debca34-0cd6-48ec-b7a7-1242fbc472f0", enable_metric=>true, charset=>"UTF-8">, role_session_name=>"logstash", delete=>false, interval=>60, watch_for_new_files=>true, temporary_directory=>"/tmp/logstash", include_object_properties=>false, gzip_pattern=>".gz(ip)?$">
  Error: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1c0e5f31[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@37ec1c71[Wrapped task = org.jruby.ext.timeout.Timeout$TimeoutTask@79b15f5b]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@63c57f7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 15]
  Exception: Java::JavaUtilConcurrent::RejectedExecutionException
  Stack: java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
org.jruby.ext.timeout.Timeout.yieldWithTimeout(Timeout.java:145)
org.jruby.ext.timeout.Timeout.timeout(Timeout.java:118)
org.jruby.ext.timeout.Timeout.timeout(Timeout.java:92)
org.jruby.ext.timeout.Timeout$INVOKER$s$timeout.call(Timeout$INVOKER$s$timeout.gen)
usr.share.logstash.vendor.jruby.lib.ruby.stdlib.net.http.RUBY$method$connect$0(/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:949)
usr.share.logstash.vendor.jruby.lib.ruby.stdlib.net.http.RUBY$method$do_start$0(/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:934)
usr.share.logstash.vendor.jruby.lib.ruby.stdlib.net.http.RUBY$method$start$0(/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:929)
usr.share.logstash.vendor.jruby.lib.ruby.stdlib.net.http.RUBY$method$start$0$__VARARGS__(/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:919)
org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:139)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:112)
org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:514)
org.jruby.RubyBasicObject.send(RubyBasicObject.java:1733)
org.jruby.RubyBasicObject$INVOKER$i$send.call(RubyBasicObject$INVOKER$i$send.gen)
org.jruby.ir.targets.indy.InvokeSite.fail(InvokeSite.java:237)
usr.share.logstash.vendor.jruby.lib.ruby.stdlib.delegate.RUBY$method$method_missing$0(/usr/share/logstash/vendor/jruby/lib/ruby/stdlib/delegate.rb:83)
org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:139)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:112)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.net_http.connection_pool.RUBY$method$start_session$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/net_http/connection_pool.rb:307)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.net_http.connection_pool.RUBY$method$session_for$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/net_http/connection_pool.rb:100)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.net_http.handler.RUBY$method$session$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/net_http/handler.rb:128)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.net_http.handler.RUBY$method$transmit$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/net_http/handler.rb:76)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.net_http.handler.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/net_http/handler.rb:50)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.content_length.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/content_length.rb:24)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.request_callback.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/request_callback.rb:85)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.s3_signer.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/s3_signer.rb:132)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.s3_signer.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/s3_signer.rb:63)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.s3_host_id.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/s3_host_id.rb:17)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.xml.error_handler.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/xml/error_handler.rb:10)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.transfer_encoding.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/transfer_encoding.rb:26)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.helpful_socket_errors.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:12)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.s3_signer.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/s3_signer.rb:110)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.redirects.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/redirects.rb:20)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.retry_errors.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/retry_errors.rb:360)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.http_checksum.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/http_checksum.rb:19)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.endpoint_pattern.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/endpoint_pattern.rb:30)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.accelerate.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/accelerate.rb:67)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.checksum_algorithm.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:136)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.dualstack.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/dualstack.rb:41)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.expect_100_continue.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/expect_100_continue.rb:22)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.bucket_name_restrictions.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/bucket_name_restrictions.rb:26)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.arn.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/arn.rb:62)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.rest.handler.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/rest/handler.rb:10)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.recursion_detection.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/recursion_detection.rb:18)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.user_agent.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/user_agent.rb:13)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.endpoint_discovery.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/endpoint_discovery.rb:80)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.endpoint.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/endpoint.rb:47)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.param_validator.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/param_validator.rb:26)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.arn.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/arn.rb:88)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.raise_response_errors.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/raise_response_errors.rb:16)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.sse_cpk.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:24)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.dualstack.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/dualstack.rb:27)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.plugins.accelerate.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/plugins/accelerate.rb:56)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.checksum_algorithm.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/checksum_algorithm.rb:111)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.jsonvalue_converter.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:16)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.idempotency_token.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/idempotency_token.rb:19)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.param_converter.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/param_converter.rb:26)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.request_callback.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/request_callback.rb:71)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.plugins.response_paging.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/plugins/response_paging.rb:12)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.plugins.response_target.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/plugins/response_target.rb:24)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.seahorse.client.request.RUBY$method$send_request$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/seahorse/client/request.rb:72)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.client.RUBY$method$list_objects_v2$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/client.rb:8556)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_s3_minus_1_dot_114_dot_0.lib.aws_minus_sdk_minus_s3.bucket.RUBY$block$objects$1(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-s3-1.114.0/lib/aws-sdk-s3/bucket.rb:912)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.runtime.Block.call(Block.java:143)
org.jruby.RubyProc.call(RubyProc.java:309)
org.jruby.RubyGenerator.each(RubyGenerator.java:99)
org.jruby.RubyGenerator$INVOKER$i$0$0$each.call(RubyGenerator$INVOKER$i$0$0$each.gen)
org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:514)
org.jruby.RubyClass.finvoke(RubyClass.java:502)
org.jruby.runtime.Helpers.invoke(Helpers.java:645)
org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:387)
org.jruby.RubyEnumerator.__each__(RubyEnumerator.java:396)
org.jruby.RubyEnumerator.each(RubyEnumerator.java:392)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.resources.collection.RUBY$block$non_empty_batches$1(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/resources/collection.rb:101)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.runtime.Block.call(Block.java:143)
org.jruby.RubyProc.call(RubyProc.java:309)
org.jruby.RubyGenerator.each(RubyGenerator.java:99)
org.jruby.RubyGenerator$INVOKER$i$0$0$each.call(RubyGenerator$INVOKER$i$0$0$each.gen)
org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:514)
org.jruby.RubyClass.finvoke(RubyClass.java:502)
org.jruby.runtime.Helpers.invoke(Helpers.java:645)
org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:387)
org.jruby.RubyEnumerator.__each__(RubyEnumerator.java:396)
org.jruby.RubyEnumerator.each(RubyEnumerator.java:392)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.resources.collection.RUBY$block$each$1(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/resources/collection.rb:52)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.runtime.Block.call(Block.java:143)
org.jruby.RubyProc.call(RubyProc.java:309)
org.jruby.RubyGenerator.each(RubyGenerator.java:99)
org.jruby.RubyGenerator$INVOKER$i$0$0$each.call(RubyGenerator$INVOKER$i$0$0$each.gen)
org.jruby.RubyClass.finvokeWithRefinements(RubyClass.java:514)
org.jruby.RubyClass.finvoke(RubyClass.java:502)
org.jruby.runtime.Helpers.invoke(Helpers.java:645)
org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:387)
org.jruby.RubyEnumerator.__each__(RubyEnumerator.java:396)
org.jruby.RubyEnumerator.each(RubyEnumerator.java:392)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.aws_minus_sdk_minus_core_minus_3_dot_151_dot_0.lib.aws_minus_sdk_minus_core.resources.collection.RUBY$method$each$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/aws-sdk-core-3.151.0/lib/aws-sdk-core/resources/collection.rb:58)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_integration_minus_aws_minus_7_dot_1_dot_0.lib.logstash.inputs.s3.RUBY$method$list_new_files$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-integration-aws-7.1.0/lib/logstash/inputs/s3.rb:141)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_integration_minus_aws_minus_7_dot_1_dot_0.lib.logstash.inputs.s3.RUBY$method$process_files$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-integration-aws-7.1.0/lib/logstash/inputs/s3.rb:183)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_integration_minus_aws_minus_7_dot_1_dot_0.lib.logstash.inputs.s3.RUBY$block$run$1(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-integration-aws-7.1.0/lib/logstash/inputs/s3.rb:130)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)
org.jruby.runtime.Block.call(Block.java:147)
org.jruby.RubyProc.call(RubyProc.java:272)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.stud_minus_0_dot_0_dot_23.lib.stud.interval.RUBY$method$interval$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/stud-0.0.23/lib/stud/interval.rb:20)
usr.share.logstash.vendor.bundle.jruby.$2_dot_6_dot_0.gems.logstash_minus_integration_minus_aws_minus_7_dot_1_dot_0.lib.logstash.inputs.s3.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/logstash-integration-aws-7.1.0/lib/logstash/inputs/s3.rb:129)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:411)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:406)
org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:139)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:112)
org.jruby.ir.targets.indy.InvokeSite.invoke(InvokeSite.java:208)
usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:402)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:141)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:64)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
org.jruby.runtime.Block.call(Block.java:143)
org.jruby.RubyProc.call(RubyProc.java:309)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:107)
java.base/java.lang.Thread.run(Thread.java:833)
@agrant1 agrant1 added the bug label Mar 2, 2023
@agrant1
Copy link
Author

agrant1 commented Apr 25, 2023

Following up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@agrant1