Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

s3 support? #5

Closed
tooptoop4 opened this issue Jul 27, 2019 · 2 comments
Closed

s3 support? #5

tooptoop4 opened this issue Jul 27, 2019 · 2 comments

Comments

@tooptoop4
Copy link

tooptoop4 commented Jul 27, 2019 •
edited
Loading

@arempter is AWS S3 supported?
@arempter
Copy link
Contributor

Hey,
The question is very wide, so not sure what exactly you are looking for.
In general plugin uses AWS SDK so it should work. There may be problem with url endpoint value when constructing S3 client, sice we use host:port and in AWS you just specify https://region_url. I never tested it.
So in summary it may/will with small update...

Plugin is just one part, allows Ranger to understand S3 format.
You will also need authorizer on application side, which will ask Ranger, if resource to be accessed is allowed by policy

@tooptoop4
Copy link
Author

have u plugged pyspark into this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@arempter @tooptoop4