Error: Cannot find module 'protobufjs'

[lounsbrough]

Describe the bug

When deploying an app on the latest Node alpine 22, I get an error Error: Cannot find module 'protobufjs'.

Expected Behavior

I expect to get a functioning package that I can use to connect to Valkey in AWS.

Current Behavior

I get the error: Error: Cannot find module 'protobufjs'.

Reproduction Steps

Build an image on Alpine Linux and then try to create a valkey client.

Possible Solution

No response

Additional Information/Context

No response

Client version used

1.2.0

Engine type and version

AWS Valkey 8.0

OS

Alpine Linux

Language

TypeScript

Language Version

Node 22

Cluster information

Not relevant

Logs

Error: Cannot find module 'protobufjs'

Other information

No response

[avifenesh]

@lounsbrough Hi, thanks for opening the issue.
I tried to reproduce the issue, but I couldn't, I don't get the error you get.
Created a simple example running JS app on a node:22-alpine image as you shared, and I don't get any issue installing or connecting.
I'm running some commands and don't get any issue with interacting with the cluster.
Can you please share more details, that might help me locate the issue?
If you are interested, I can hope on a call when you are free for it, and I can check it with you and maybe give a live support.

[lounsbrough]

@avifenesh, Thanks for the quick reply. I will try to make a minimal repro example that I can share here.

[lounsbrough]

@avifenesh - sorry I just finally got back to this issue. I threw a reproduction up here: https://github.com/lounsbrough/valkey-issue-repro/blob/issue-2812/README.md

It only happens for me on Node 22, Node 20 works fine. Note also that I created this outside of Docker, so that does not appear to be the issue in my case.

Let me know if you can also reproduce the issue. Thanks for your help.

[lounsbrough]

I also found that after I install a couple times (with pnpm i) then the issue goes away. For some reason the first pnpm i doesn't always install the protobuf dependencies. Maybe this is a pnpm issue? I can try to look into it a little more tomorrow.

After first install:

After second install:

[avifenesh]

@lounsbrough yes pnpm can explain it, since our CI run on node alpine latest as well, and teat the packages that been released, and i did very simple app without any workarounds so it was supposed to be reproduce.
Pnpm use different dependency mechanism by default (isolated) while npm use hoisted, so it might be that in our structure we use it from a mutual directory, but using pnpm its separated from the other dependency and its being fix by the reinstall with lock file.
Its just a guess. I will reproduce with pnpm later today.

[Yury-Fridlyand]

@avifenesh please consider adding more details to installation instructions and/or update troubleshooting section

[avifenesh]

I will do it when finishing the research and have a clear understanding, currently those are just assumptions.
@Yury-Fridlyand

[avifenesh]

@lounsbrough Sorry about the delay, had some other duties this week.
Are you also copying in the lock file?

[lounsbrough]

No worries. Yes we copy the pnpm-lock.yml file during the build, and the lock file also did not change locally between installs when I was experiencing the issue. I can mess with it some more tomorrow and see if I find anything else out.

[avifenesh]

So what I managed to see till now is that while not copying the lock, it's working.
If you add protobufjs to your packge.json separately, it's also working.
And if you add .npmrc and set to "hoist=true" that's also working.

But - glide use optional dependencies, and for some reason pnpm download all of them without connection to the environment as npm does. And while doing that, it creates symlink just to one of them, I have hard time understanding the logic behind this design. It shouldn't create slimmer packages?
Why to use pnpm in the first place?

[lounsbrough]

@avifenesh - My team started using pnpm a while back because it is much faster than npm. This does feel more like a pnpm bug than a valkey-glide bug. We have a workaround for now, so I'll leave it up to you how much further to pursue this. Thanks for your effort looking into this.

[avifenesh]

@lounsbrough since there's a workaround and it is not a blocker for you, we will not rush it as a patch.
But for the next release we plan to change the structure of the package and I believe it will solve this issue as well.

It worth opening issue to pnpm since the way they managing platform based optional dependencies end up very expensive, and it would be the same for every package use the same type of structure (for example SWC).

I'll leave the issue open so we can go back to it with the fix.

And we would love to get feedback from you and your team about glide, good or bad, as well as feature requests, issues etc.

We have a discord channel for direct communication, if you need anything or have some feedback 🙏

https://discord.gg/XYgTZnM7

[avifenesh]

Putting all the issues which are relevant to the restructuring under the latest issue on the topic and closing them.
#3195