Steps to reproduce:

1. Setup voice pipline
2. Add a smart light to a zone
3. Add a voice device to HA and set it to the same zone as the light
4. Say to the voice device Set the lights to 50%

I did notice in my testing that Set the brightness to 50% works.