Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

max_tokens is replaced into max_completion_tokens in the http request #30113

Open
5 tasks done
memorylorry opened this issue Mar 5, 2025 · 1 comment
Open
5 tasks done
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@memorylorry
Copy link

memorylorry commented Mar 5, 2025

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langchain_openai import ChatOpenAI
from langchain.schema import SystemMessage, HumanMessage

API_KEY = 'sk-key'
BASE_URL = 'https://api.deepseek.com'
MODEL = 'deepseek-chat'

MAX_TOKENS = 8888

chat = ChatOpenAI(
base_url=BASE_URL,
model=MODEL,
api_key=API_KEY,
max_tokens=MAX_TOKENS, # not useful
temperature=0.664
)

messages = [
SystemMessage(content="你是一个乐于助人的 AI。"),
HumanMessage(content="请写一首关于春天的诗。")
]

response = chat.invoke(messages)

print(response.content)

Error Message and Stack Trace (if applicable)

max_tokens is replaced into max_completion_tokens in the http request. But real useful parameter is max_tokens in our models.

Description

When we use ChatOpenAI to utilize ourselves' deployed model in inner network, I try to set max-tokens parameter of this class, and I find this parameter is replaced by max_completion_tokens. I cannot understand which reason it is. Was max_completion_tokens parameter used before?
But the fact is that our model use max-tokens to set models' the length of real input token.
So, Dose these mean that ChatOpenAI is deprecated?

System Info

System Information

OS: Linux
OS Version: #52~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Dec 9 15:00:52 UTC 2
Python Version: 3.10.16 (main, Dec 11 2024, 16:24:50) [GCC 11.2.0]

Package Information

langchain_core: 0.3.41
langchain: 0.3.10
langchain_community: 0.3.10
langsmith: 0.1.147
langchain_anthropic: 0.3.0
langchain_astradb: 0.5.2
langchain_aws: 0.2.7
langchain_chroma: 0.1.4
langchain_cohere: 0.3.3
langchain_elasticsearch: 0.3.0
langchain_experimental: 0.3.4
langchain_google_calendar_tools: 0.0.1
langchain_google_community: 2.0.3
langchain_google_genai: 2.0.6
langchain_google_vertexai: 2.0.7
langchain_groq: 0.2.1
langchain_milvus: 0.1.7
langchain_mistralai: 0.2.3
langchain_mongodb: 0.2.0
langchain_nvidia: Installed. No version info available.
langchain_nvidia_ai_endpoints: 0.3.5
langchain_ollama: 0.2.1
langchain_openai: 0.3.7
langchain_pinecone: 0.2.0
langchain_text_splitters: 0.3.5
langchain_unstructured: 0.1.5
langchainhub: 0.1.21

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.9.5
anthropic: 0.45.0
anthropic[vertexai]: Installed. No version info available.
astrapy: 1.5.2
async-timeout: 4.0.3
beautifulsoup4: 4.12.3
boto3: 1.34.162
chromadb: 0.5.23
cohere: 5.13.11
dataclasses-json: 0.6.7
db-dtypes: Installed. No version info available.
defusedxml: 0.7.1
elasticsearch[vectorstore-mmr]: Installed. No version info available.
fastapi: 0.115.7
filetype: 1.2.0
gapic-google-longrunning: Installed. No version info available.
google-api-core: 2.24.0
google-api-python-client: 2.154.0
google-api-python-client>=2.104.0: Installed. No version info available.
google-auth-httplib2: 0.2.0
google-auth-oauthlib: 1.2.1
google-auth-oauthlib>=1.1.0: Installed. No version info available.
google-cloud-aiplatform: 1.78.0
google-cloud-bigquery: 3.29.0
google-cloud-bigquery-storage: Installed. No version info available.
google-cloud-contentwarehouse: Installed. No version info available.
google-cloud-core: 2.4.1
google-cloud-discoveryengine: Installed. No version info available.
google-cloud-documentai: Installed. No version info available.
google-cloud-documentai-toolbox: Installed. No version info available.
google-cloud-speech: Installed. No version info available.
google-cloud-storage: 2.19.0
google-cloud-texttospeech: Installed. No version info available.
google-cloud-translate: Installed. No version info available.
google-cloud-vision: Installed. No version info available.
google-generativeai: 0.8.4
googlemaps: Installed. No version info available.
groq: 0.15.0
grpcio: 1.70.0
httpx: 0.27.2
httpx-sse: 0.4.0
jsonpatch<2.0,>=1.33: Installed. No version info available.
langchain-core<1.0.0,>=0.3.39: Installed. No version info available.
langchain>=0.0.335: Installed. No version info available.
langsmith-pyo3: Installed. No version info available.
langsmith<0.4,>=0.1.125: Installed. No version info available.
numpy: 1.26.4
ollama: 0.4.7
openai<2.0.0,>=1.58.1: Installed. No version info available.
orjson: 3.10.0
packaging: 24.2
packaging<25,>=23.2: Installed. No version info available.
pandas: 2.2.2
pillow: 10.4.0
pinecone-client: 5.0.1
protobuf>=4.25.0: Installed. No version info available.
pyarrow: 17.0.0
pydantic: 2.10.6
pydantic-settings: 2.4.0
pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
pymilvus: 2.4.9
pymongo: 4.10.1
pytz>=2023.3.post1: Installed. No version info available.
PyYAML: 6.0.2
PyYAML>=5.3: Installed. No version info available.
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.37
tabulate: 0.9.0
tenacity: 8.5.0
tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
tiktoken<1,>=0.7: Installed. No version info available.
tokenizers: 0.20.3
types-requests: 2.32.0.20241016
typing-extensions>=4.7: Installed. No version info available.
unstructured-client: 0.25.9
unstructured[all-docs]: Installed. No version info available.

@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Mar 5, 2025
@andrasfe
Copy link

andrasfe commented Mar 9, 2025

@memorylorry , FYI: this? In OAI max_tokens has been deprecated in favor of max_completion_tokens. Also, have you tried using ChatDeepSeek instead of ChatOpenAI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

2 participants