new(expr) and its effect on Go Protobuf

[stapelberg]

Go 1.26 (not yet released) will allow the new built-in to be called on expressions: https://antonz.org/accepted/new-expr/

Therefore, the following Go Protobuf example:

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/descriptorpb"
)

func main() {
	b, err := proto.Marshal(&descriptorpb.DescriptorProto{
		Name: proto.String("hoi"),
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protobuf wire format:\n% x\n", b)
}

…can be changed to use new("hoi"):

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/descriptorpb"
)

func main() {
	b, err := proto.Marshal(&descriptorpb.DescriptorProto{
		Name: new("hoi"),
	})
	if err != nil {
		panic(err)
	}
	fmt.Printf("Protobuf wire format:\n% x\n", b)
}

This means that the need for the helper functions in the proto package like proto.String is going away over time.

We will not deprecate these functions because that causes unnecessary churn (and there’s nothing wrong with using the functions).

We should probably update the examples to use the new() syntax throughout, but not sure at which point in time. Immediately after the Go 1.26 release would probably be a little bit early.

I’m filing this issue for discussion. If there are any considerations with regards to how Go Protobuf should change with regards to the new built-in, please share them here.

[znkr]

proto.String and friends are functions that need to be learned to understand what they are doing and when they need to be used. Also, I am pretty sure that people will start using new("hoi") almost immediately after the release of go1.26. Additionally, I believe that Go has modernizer analyzer as part of gopls that will rewrite usages of proto.String and friends to new(expr). That is, in a few years, it's likely going to be rare to see proto.String and friends in the wild and I don't think it makes sense to not go with the flow.

[lfolger]

One advantage the proto.Int32 function has it always makes a copy of the underlying value. Users are naturally guided towards using these functions when populating their protos. If we don't offer such functions, users need to come up with their own way to provide a pointer. new(expr) is one such way, &mystruct.Value is another which might lead to aliasing issues.

proto.String and friends are functions that need to be learned to understand what they are doing and when they need to be used.

I'm not sure I follow. Why do I have to learn what they are doing to use them? They take a value and return something I can store in a proto. I don't need to look at their implementation to use them.

[znkr]

I'm not sure I follow. Why do I have to learn what they are doing to use them? They take a value and return something I can store in a proto. I don't need to look at their implementation to use them.

Most of the open source world uses proto3 where the proto.String, etc, functions are only necessary for optional fields. However, you don't see if a field is optional or not by looking a proto with an LSP, you'll only see that this field requires a pointer. So you need to know that there is a proto.String function that you are supposed to use them instead of, say, new(expr).

[adonovan]

You might want to wait a cycle or two so that the examples don't advise use of language features that are not yet available to many users.

[puellanivis]

Oof, var i int32 = new(42) still fails, which means we would need Int32Field: new(int32(13)) to set a value that could probably be type inferred. I get ne(expr) is great if expr is already signed, but for constants it would still be nicer to have something like &int32(42) matching up with &myStruct{} syntax. Though, of course, I know this is the wrong place to be airing such a disappointment…

But this really is already such a big improvement for composite literal protobufs for proto2 (lots of legacy Google protos) and editions (the future).

[adonovan]

Oof, var i int32 = new(42) still fails, which means we would need Int32Field: new(int32(13)) to set a value that could probably be type inferred. I get ne(expr) is great if expr is already signed, but for constants it would still be nicer to have something like &int32(42) matching up with &myStruct{} syntax. Though, of course, I know this is the wrong place to be airing such a disappointment…

This was litigated in golang/go#76122 and the decision was that such cases should be expressed as new(int32(42)).

[puellanivis]

This was litigated in golang/go#76122 and the decision was that such cases should be expressed as new(int32(42)).

It doesn’t seem to cover the &int32(3) idea, but it does cover the Int32Field: new(13) discussion… since it’s semantically identical to a new[T any] function, that’s pretty easy, and it makes a lot of sense, as mentioned in the linked issue, it would require a really big change of type inference to hoist this type information around like that.

The discussions in golang/go#45624 does seem to note that new[T](x) and new(T(x)) are the same number of characters, so preferring the later over the former is fine, and doesn’t require the builtin new to suddenly become a generic function. And that’s all good, but it doesn’t cover that &T(x) is shorter. Though the later certainly would require a deeper change than the new(expr), which simplified a lot of implementing the change.

All said, as I mentioned, this is a good change. It’s like to see the semantic sugar version at some point still, but perfect is the enemy of good, and this is good.