Skip to content

Conversation

@dmjoy
Copy link
Contributor

@dmjoy dmjoy commented Nov 6, 2025

@eveenhuis @jadie1 @barry-ravichandran FYSA

This is a WIP, but wanted to draw attention to some of the changes that need to be made for the latest outlines version. Specifically:

  • The outlines.pompt decorate has been removed, now the guidance is to use the outlines.Template.from_string function.
    • Copied this decorator in from the older outlines version for near term "compatiblity" rather than require changes to all of these old templates (using outlines.Template.from_string we also need to change how these templates are invoked wherever they're used, and that's scattered around several algorithms; not worth) future templates should strongly considering using this new approach though
  • The notion of a sampler has been removed from outlines, instead inference control parameters are just passed through to whatever backend is loading / serving the model (so for determinstic sampling using transformers as the model provider we would set do_sample=False in the generation_kwargs)

Copy in old outlines.prompt decorator for compatibility
@dmjoy dmjoy force-pushed the dev/update-outlines-and-transformers branch from bf92614 to 542d191 Compare November 6, 2025 21:21
@dmjoy
Copy link
Contributor Author

dmjoy commented Nov 7, 2025

Alright so functionally this all seems to work, but when running the integration tests, there are several differences (mostly that makes sense as enough pieces have changed here that I wouldn't expect to get the exact same output). However, it seems like there's some non-determinism even with do_sample=False, and for the comparative regression ADMs I'm getting negative regression values in some cases, which I don't think we've seen before. Digging into the negative regression values though and it seems like maybe this was never enforced with JSON schemas?
dottxt-ai/outlines#215
dottxt-ai/outlines-core#147

I need to spend some more time with this to understand what's really going on before feeling confident in merging. Anecdotally it does seem a bit faster now, but if it's not deterministic and doesn't enforce the entire schema what's the point.

@barry-ravichandran
Copy link
Contributor

@dmjoy Tagging this package update PR here in case it's ok with you to merge it into your PR to fix some issues with installing via poetry: #250

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants