Pinning peer public keys for client?

[dhoodx64]

I was able to deploy this on a K3S cluster and it runs as expected so far. However I cant get around how to apply my clients public key in the server configuration. I did try to use the publicKey item under spec, but that does not seem to change anything except borking the private key. is there any way to pin the public key to the peer? Below is what I had tried.

kind: WireguardPeer
metadata:
  name: test
  namespace: "wireguardvpn"
spec:
  wireguardRef: "wgserver"
  publicKey: "cBaucnK3ufdDm4/UGP/+d4WETwj1+E/xsVIAsiLyVzc="


`` kubectl get wireguardpeer test --template={{.status.config}} -n wireguardvpn | bash ``
Error from server (NotFound): secrets "test-peer" not found 

[Interface]
PrivateKey = 
Address = 10.8.0.3
DNS = 10.43.0.10, wireguardvpn.svc.cluster.local
MTU = 1380

[Peer]
PublicKey = qn+nbV9R3onKixAoXKcwLlPz+f/vW3O5nOZpVB3SjSQ=
AllowedIPs = 0.0.0.0/0
Endpoint = 192.168.3.237:51820 >

[Cavaler]

From what I could understand, this is not possible, since it seems server want to know the peer private key to be able to display its config.
This does seem to be a somewhat weird choice, but more or less OK for the centralized key management case.

However, I was not able to make it use a pre-existing secret with a private key either. And that is a serious problem.