Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

appendMap Panic #1648

Closed
winey-dev opened this issue Sep 20, 2024 · 4 comments
Closed

appendMap Panic #1648

winey-dev opened this issue Sep 20, 2024 · 4 comments

Comments

@winey-dev
Copy link

Go Version

  • go version go1.22.0 linux/amd64

protoc Version

  • libprotoc 27.3

Log

reflect.mapiterinit(0x4ac5a5?, 0x1b12b80?, 0x7b0e?)
        /home/smlee/.dev-package/go/src/runtime/map.go:1387 +0x13
reflect.(*MapIter).Next(0x1c07380?)
        /home/smlee/.dev-package/go/src/reflect/value.go:2000 +0x57
google.golang.org/protobuf/internal/impl.appendMap({0xc002bc6000, 0x6cb0, 0x7b0e}, {0x1c07380?, 0xc0015a4ba0?, 0x6cb0?}, 0xc0015fa3c0, 0xc00049de28, {0x74?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_map.go:274 +0x1d3
google.golang.org/protobuf/internal/impl.encoderFuncsForMap.func2({0xc002bc6000, 0x6cb0, 0x7b0e}, {0xc0015a51e0?}, 0xc00049de28, {0x50?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_map.go:56 +0x7e
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc000426010, {0xc002bc6000?, 0x5d42c8?, 0x2d8?}, {0xc000795400?}, {0x80?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab578?, 0x5e9345?}, {0xc002bc6000?}, 0xc00154e450, {0x0?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc000426158, {0xc002bc6000?, 0x5bc0aa?, 0x612?}, {0xc002bc6000?}, {0x2?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab638?, 0x41b128?}, {0xc005e30b00?}, 0xc00055a948, {0x0?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc0004262a0, {0xc002bc6000?, 0xc0004262a0?, 0x412c?}, {0x7f7ed79378d8?}, {0x5?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab728?, 0x4113be?}, {0x7f7ed79378d8?}, 0xc001698090, {0x28?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc0004263e8, {0xc002bc6000?, 0x8000?, 0x7f7f1ef92b38?}, {0x8000?}, {0x8?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshal(0x0?, {{}, {0x22901b8, 0xc000778e40}, {0xc002bc6000, 0x0, 0x7b0e}, 0x2})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:107 +0x85
google.golang.org/protobuf/proto.MarshalOptions.marshal({{}, 0xc0?, 0xb8?, 0x7a?}, {0x0, 0x0, 0x0}, {0x22901b8, 0xc000778e40})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:163 +0x24e
google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend({{}, 0x40?, 0xc4?, 0xd3?}, {0x0, 0x0, 0x0}, {0x2264de0?, 0xc000778e40?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0x73
github.com/golang/protobuf/proto.marshalAppend({0x0, 0x0, 0x0}, {0x7f7ed8456a78?, 0xc000778e40?}, 0x0)
        /home/smlee/go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0x97
github.com/golang/protobuf/proto.Marshal(...)
        /home/smlee/go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23
google.golang.org/grpc/encoding/proto.codec.Marshal({}, {0x1d3c440?, 0xc000778e40?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:45 +0x5a
google.golang.org/grpc.encode({0x7f7ed849d260?, 0x323eb40?}, {0x1d3c440?, 0xc000778e40?})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/rpc_util.go:594 +0x3e
google.golang.org/grpc.(*Server).sendResponse(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, {0x1d3c440, 0xc000778e40}, {0x0, 0x0}, 0xc00392cfb8, {0x0, ...})
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1062 +0x167
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, 0xc001089dd0, 0x31b1190, 0x0)
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1318 +0xd75
google.golang.org/grpc.(*Server).handleStream(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, 0x0)
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1619 +0x9da
google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:921 +0x8d
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 1385081
        /home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:919 +0x226

For services that operate on the background, Store/Load is performed through SyncMap, etc

When the gRPC Call comes, the corresponding sync.Map will go around the Range and check it.

I'm getting Panic from AppendMap intermittently when I'm doing that... Is it my code error?
@puellanivis
Copy link
Collaborator

What is the panic message that you’re receiving? Like the panic: foo bar baz line.

@winey-dev
Copy link
Author

@puellanivis .
Sorry .

fatal error: concurrent map iteration and map write

goroutine 1385082 [running]:
reflect.mapiterinit(0x4ac5a5?, 0x1b12b80?, 0x7b0e?)
	/home/smlee/.dev-package/go/src/runtime/map.go:1387 +0x13
reflect.(*MapIter).Next(0x1c07380?)
	/home/smlee/.dev-package/go/src/reflect/value.go:2000 +0x57
google.golang.org/protobuf/internal/impl.appendMap({0xc002bc6000, 0x6cb0, 0x7b0e}, {0x1c07380?, 0xc0015a4ba0?, 0x6cb0?}, 0xc0015fa3c0, 0xc00049de28, {0x74?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_map.go:274 +0x1d3
google.golang.org/protobuf/internal/impl.encoderFuncsForMap.func2({0xc002bc6000, 0x6cb0, 0x7b0e}, {0xc0015a51e0?}, 0xc00049de28, {0x50?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_map.go:56 +0x7e
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc000426010, {0xc002bc6000?, 0x5d42c8?, 0x2d8?}, {0xc000795400?}, {0x80?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab578?, 0x5e9345?}, {0xc002bc6000?}, 0xc00154e450, {0x0?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc000426158, {0xc002bc6000?, 0x5bc0aa?, 0x612?}, {0xc002bc6000?}, {0x2?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab638?, 0x41b128?}, {0xc005e30b00?}, 0xc00055a948, {0x0?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc0004262a0, {0xc002bc6000?, 0xc0004262a0?, 0x412c?}, {0x7f7ed79378d8?}, {0x5?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.appendMessageSliceInfo({0xc002bc6000?, 0xc0017ab728?, 0x4113be?}, {0x7f7ed79378d8?}, 0xc001698090, {0x28?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/codec_field.go:485 +0xf1
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshalAppendPointer(0xc0004263e8, {0xc002bc6000?, 0x8000?, 0x7f7f1ef92b38?}, {0x8000?}, {0x8?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:139 +0x13f
google.golang.org/protobuf/internal/impl.(*MessageInfo).marshal(0x0?, {{}, {0x22901b8, 0xc000778e40}, {0xc002bc6000, 0x0, 0x7b0e}, 0x2})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:107 +0x85
google.golang.org/protobuf/proto.MarshalOptions.marshal({{}, 0xc0?, 0xb8?, 0x7a?}, {0x0, 0x0, 0x0}, {0x22901b8, 0xc000778e40})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:163 +0x24e
google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend({{}, 0x40?, 0xc4?, 0xd3?}, {0x0, 0x0, 0x0}, {0x2264de0?, 0xc000778e40?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0x73
github.com/golang/protobuf/proto.marshalAppend({0x0, 0x0, 0x0}, {0x7f7ed8456a78?, 0xc000778e40?}, 0x0)
	/home/smlee/go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0x97
github.com/golang/protobuf/proto.Marshal(...)
	/home/smlee/go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23
google.golang.org/grpc/encoding/proto.codec.Marshal({}, {0x1d3c440?, 0xc000778e40?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/encoding/proto/proto.go:45 +0x5a
google.golang.org/grpc.encode({0x7f7ed849d260?, 0x323eb40?}, {0x1d3c440?, 0xc000778e40?})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/rpc_util.go:594 +0x3e
google.golang.org/grpc.(*Server).sendResponse(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, {0x1d3c440, 0xc000778e40}, {0x0, 0x0}, 0xc00392cfb8, {0x0, ...})
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1062 +0x167
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, 0xc001089dd0, 0x31b1190, 0x0)
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1318 +0xd75
google.golang.org/grpc.(*Server).handleStream(0xc000c0ce00, {0x228abc0, 0xc000b05a00}, 0xc000c9efc0, 0x0)
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:1619 +0x9da
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:921 +0x8d
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 1385081
	/home/smlee/go/pkg/mod/google.golang.org/[email protected]/server.go:919 +0x226

@puellanivis
Copy link
Collaborator

I don’t know what your problem is, but I know it is going to be somewhere in your own code.

I’m taking a stab here, because I don’t know what your code is, but I think there is a map somewhere in the message itself, which is not concurrent safe like sync.Map is.

Another common read/write race condition can happen when you’re attempting to marshal a message at the same time as accessing it. You need to ensure you are only doing the one or the other at any given time.

@stapelberg
Copy link

As @puellanivis explained, almost certainly the issue is within your code.

You can use Go’s race detector to locate the issue: https://go.dev/blog/race-detector

I’ll close this issue as it’s not a Go Protobuf issue.

@stapelberg stapelberg closed this as not planned Won't fix, can't repro, duplicate, stale Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@stapelberg @puellanivis @winey-dev