Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document proper log analysis #834

Open
5 tasks
kltm opened this issue Apr 20, 2022 · 1 comment
Open
5 tasks

Document proper log analysis #834

kltm opened this issue Apr 20, 2022 · 1 comment

Comments

@kltm
Copy link
Contributor

kltm commented Apr 20, 2022

Now that there are rotating apache logs in an S3 bucket, how does the average operator get at them and use them?

  • Have proper AWS permissions, including API keys
  • Get AWS client tools
  • Get logs in obo-service-logs; currently obo/purl/355fea0db941/2022/03/ would be most useful for what we want
    For example: aws s3 sync s3://DESIRED_LOGS . --include "*" --exclude "*error*"
  • Decompress pigz -d purl.obolibrary.org.access.log.202203*.gz && cat purl.obolibrary.org.access.log.202203* > all-log.txt
  • Use your favorite apache2 log tool; for example goaccess all-log.txt --log-format=COMBINED
@kltm
Copy link
Contributor Author

kltm commented Apr 20, 2022

Tagging @cmungall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant