Context limit does not work

[konard]

2024-02-19 10:07:18,498 - __main__ - ERROR - OpenAI completion error: This model's maximum context length is 128000 tokens. However, your messages resulted in 130444 tokens. Please reduce the length of the messages.
2024-02-19 10:07:18,499 - __main__ - ERROR - This model's maximum context length is 128000 tokens. However, your messages resulted in 130444 tokens. Please reduce the length of the messages.

[FreePhoenix888]

What is the reason?

russian-laws-bot/main.py

Lines 25 to 26 in 149a8f3

	MAX_TOKENS = 128 * 1024
	MAX_PROMPT_TOKENS = MAX_TOKENS * 0.8

russian-laws-bot/main.py

Lines 214 to 217 in 149a8f3

	tokens_count = len(encoding.encode(prompt))
	if tokens_count > MAX_PROMPT_TOKENS:
	raise ValueError(f'{tokens_count} tokens in promt exceeds MAX_PROMPT_TOKENS ({MAX_PROMPT_TOKENS})')

[konard]

@FreePhoenix888 calculation is wrong, because now only single message content tokens number is calculated. To fix that bug, we have to calculate tokens of the entire context (all messages that will be sent to GPT-4). It will require logic similar to what we have in our ChatGPT package in Deep. Or we can just ignore that issue if we will use actual ChatGPT deep package.

As a workaround of that and other issues, I now send only single message to GPT-4 API.

[FreePhoenix888]

@FreePhoenix888 calculation is wrong, because now only single message content tokens number is calculated. To fix that bug, we have to calculate tokens of the entire context (all messages that will be sent to GPT-4). It will require logic similar to what we have in our ChatGPT package in Deep. Or we can just ignore that issue if we will use actual ChatGPT deep package.

As a workaround of that and other issues, I now send only single message to GPT-4 API.

Should we prefer chatgpt deep package ?