Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature][resource center] Can the resource center support Tencent Cloud COS #16396

Closed
2 of 3 tasks
kiwi-field opened this issue Jul 30, 2024 · 10 comments · Fixed by #16565
Closed
2 of 3 tasks

[Feature][resource center] Can the resource center support Tencent Cloud COS #16396

kiwi-field opened this issue Jul 30, 2024 · 10 comments · Fixed by #16565
Assignees
@Mighten
Labels
feature new feature help wanted Extra attention is needed

Comments

@kiwi-field
Copy link

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

I am currently using HDFS in the resource center and I want to migrate to Tencent Cloud COS uniformly. What changes do I need to make? Tencent Cloud's bucket is public read

Use case

After switching to Tencent Cloud COS, you can perform upload, download, and preview operations in the resource center

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@kiwi-field kiwi-field added feature new feature Waiting for reply Waiting for reply labels Jul 30, 2024
@SbloodyS
Copy link
Member

Currently there are no plans to support it.

@SbloodyS SbloodyS added help wanted Extra attention is needed and removed Waiting for reply Waiting for reply labels Jul 31, 2024
@kiwi-field
Copy link
Author

Can I connect to Tencent Cloud COS through the dependency of S3Utils and AWS Java SDK S3

@SbloodyS
Copy link
Member

You can try it, but cos doesn't work well with S3.

@SbloodyS SbloodyS changed the title [Feature][resource center ] Can the resource center support Tencent Cloud COS [Feature][resource center] Can the resource center support Tencent Cloud COS Jul 31, 2024
@kiwi-field
Copy link
Author

image
The above picture shows our program configuration, and when downloading from the resource center, the following error occurred。
image

@kiwi-field
Copy link
Author

It is obvious that AWS Java SDK S3 is designed for Amazon S3 cloud services, not the S3 protocol

@kiwi-field
Copy link
Author

image
The optional regions are shown in the above figure

@kiwi-field
Copy link
Author

@SbloodyS

@fuchanghai
Copy link
Member

for me ,it work

@Mighten
Copy link
Contributor

Mighten commented Sep 1, 2024 •
edited
Loading

Hi, I am currently working on this feature (#16396) to support Tencent Cloud COS as a storage plugin in DS.

Could you please assign this issue to me?

This work is due by September 30, 2024.

@fuchanghai
Copy link
Member

Hi, I am currently working on this feature (#16396) to support Tencent Cloud COS as a storage plugin in DS.

Could you please assign this issue to me?

This work is due by September 30, 2024.

done

@Mighten Mighten mentioned this issue Sep 2, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Mighten Mighten

Labels
feature new feature help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Feature-16396][storage-plugin] Add Tencent Cloud COS Storage Plugin Mighten/dolphinscheduler
4 participants
@Mighten @SbloodyS @kiwi-field @fuchanghai