Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Let's remove the unused comments in this file
There was a problem hiding this comment.
its autogenerated by jest initialization. They are config options with description of what they do, it looks it will be helpful for any future changes
There was a problem hiding this comment.
We can always get them from the docs. No need keeping them if we aren't using it
|
|
||
| on: | ||
| pull_request: | ||
| branches: [main] |
There was a problem hiding this comment.
shouldn't we test on staging as well?
There was a problem hiding this comment.
This gh-action runs the external-dependencies tests directory.
The external dependencies test directory should be limited to only PR to main because we call openai chat completion in there and that has a cost/million token.
I'm looking at this directory to hold tests that requires API calls or dependencies you wouldn't want running too often, and on PR to main seems like a good approach
Emmanuel-Develops
force-pushed
the
staging
branch
from
93b6351 to
2236327
Compare
Emmanuel-Develops
force-pushed
the
staging
branch
from
464db84 to
54db3e9
Compare
0tuedon
force-pushed
the
staging
branch
2 times, most recently
from
b4f139b to
db3c1fb
Compare
2 participants
This PR implements the groundwork for unit testing
Adds
extractor.test.ts
Ensures the gpt extractor output adheres to some predefined structure. I included some mock data (mocking different chat history context) and the tests do infact make calls to openAI against this mock data. (useful if we change models or prompt, so we have an objective test)
apiMessageSeparator.test.ts
This tests for the function that separates the output into the 3 components. (body, follow-up questions, and links).
This is a unit functional test that should be part of our CI/CD.
A consequence of these tests is that they make our prompts better and outputs relatively deterministic (at least for the extractor). For example, usually for every model change we edit the prompts for that particular model, using the extractor test we have a objective measurement that informs us if the prompt works, and consequently makes the prompts robust as switching between models and running the tests can highlight vague instructions in the prompts.
note: as at the time of writing, all mainstream models used as
OPENAI_EXTRACTOR_MODELpasses the extractor test