Different approach to enum generation

[jakubdulas]

Adds enum generation by allowing to generate only tokens from enums. Created with @trebacz626

[wassname]

So the original method would add each enum to the prompt, and see what hindsight probability was given to it's tokens. This method generates them uses a logits transform that restricts generation to the tree of tokens in the enum values.

I get slightly different probs with the different methods, this one giving more extreme (and probably more correct since forward prediction is the task it was trained on) probs.

Check out my measurements here https://github.com/wassname/prob_jsonformer/blob/main/prob_dist.ipynb