Uploading file from the path but not as a string/buffer?

[pubmikeb]

Before you start
Have you checked StackOverflow, previous issues, and Dropbox Developer Forums for help?
Yes.

What is your question?
I want to implement a direct file uploading from a client to the Dropbox without a need of creating a buffer on the application server. This is critical in case of big files, which can't be stored as a buffer on the application server in a whole.

I've found a thread from 2017: Url of the file instead of a buffer in uploadSessionAppendV2?, where it is stated: «No, unfortunately the API v2 JavaScript SDK only supports supplying the file content as string or buffer, but I'll be sure to pass this along as a feature request.» Are there any changes since that time?

As a middleware, I'm using formidable, which contains fileWriteStreamHandler: «If you are looking to write the file uploaded in other types of cloud storages (AWS S3, Azure blob storage, Google cloud storage) or private file storage, this is the option you're looking for.»

Is there any way to forward an uploading file directly to Dropbox and minimize an application server CPU & RAM overloading?

This is especially critically when we're talking about uploading big files, which application server can't handle in the RAM in whole.

Versions

• What version of the SDK are you using? 10.4.3
• What version of the language are you using? ES-2021
• Are you using Javascript or Typescript? JS
• What platform are you using? (if applicable) Node.js

[greg-db]

We don't currently have documentation or an example of this, but I believe the Dropbox JavaScript SDK should accept streams for uploads in Node now. I can't offer help with "formidable", as that's not made by Dropbox, but for example, you should technically be able to pass a ReadStream object to a Dropbox upload method, such as filesUpload or filesUploadSessionAppendV2, to supply the data to upload that way.

If something isn't working for you though, please share the code and output so we can take a look.

By the way, note that this is not exactly tied to whether or not you're running client-side or server-side. The Dropbox API v2 JavaScript SDK, including its upload methods, can be run either client-side (such as in JavaScript in the browser) or server-side (such as in Node.js). Either way, it can produce HTTPS requests to communicate with the Dropbox API servers.

For instance, in JavaScript in the browser, you can upload data from an input file from the browser directly to the Dropbox API, like shown in this example.

Hope this helps!

[pubmikeb]

By the way, note that this is not exactly tied to whether or not you're running client-side or server-side.

The reason the uploading procedure is implemented on a server-side is a willing to not expose the Dropbox API access token on a client-side. Is there any way to use the Dropbox API on a client-side without exposing the API token via DevTools?

[greg-db]

It's not possible to securely hide an access token on the client, so if you're using an access token that the end-user shouldn't have access to, you do need to make sure you do not send it to the client at all. (The typical way of using the Dropbox API is that the end-user would interact with their own account, so it's okay for them to have access to the access token since it's just for their own account.)

In this case though, you should look into the filesGetTemporaryUploadLink method. You can call that on the server, and then pass down the resulting link to the client, which can be used without the access token. It doesn't support large files like upload sessions though, so that would only work for files less than 150 MB in size.

[pubmikeb]

@greg-db, thanks for pointing me to filesGetTemporaryUploadLink().

After several hours, I've figured out the working POC.
I must admit, the current Dropbox SDK documentation is not so helpful in case there is no previous hands-on experience with the Dropbox SDK. The real life full (plug & play, turn-key) samples with the implementation of the common scenarios could improve the readability of the documentation a lot! This is especially valid for filesGetTemporaryUploadLink(), since Google returns just 5 results, which point to the TypeScript definition from the API reference.

[pubmikeb]

It doesn't support large files like upload sessions though, so that would only work for files less than 150 MB in size.

Does it mean that in case I would like to upload files larger than 150 MB, I'll be required to send them through the application server, which leads to extra traffic, CPU, and RAM?

[greg-db]

Thanks for the feedback!

Does it mean that in case I would like to upload files larger than 150 MB, I'll be required to send them through the application server, which leads to extra traffic, CPU, and RAM?

Yes, unfortunately the Dropbox API doesn't offer functionality like filesGetTemporaryUploadLink for larger files, so you'd need to send those through your server to protect your access token in a case like this.

[pubmikeb]

Thanks for the feedback!

One more concrete example of a documentation lack.
FilesCommitInfo contains a property mode of a type FilesWriteMode, which consists of update and .tag. But it's completely unclear what are the possible values of update. Of course, I can digger into source code and make kind of reverse engineering but wouldn't it be more correctly to provide such info in a documentation, ideally, there should be an enum of possibly accepted values for both update and .tag to avoid possible typo errors.

[greg-db]

I see, thanks, that's a helpful example. I've sent this along to the team.