getting null data in input and output columns 😬

[jankigabani]

so, i'm trying trace using start_trace() and i'm doing it on huggingface model llm with langchain's LlamaCPP framework. it is working and i'm getting raw json data but in table i'm not able to get input , output , Total tokens, and status

this type of raw json i'm getting:
{
name: "RunnableSequence.langchain.workflow"
context: {
trace_id: "0xee53aa1411a59b6933cafeb1da95724d"
span_id: "0x7a5a731b42cb0060"
trace_state: ""
}
kind: "1"
parent_id: null
start_time: "2024-06-10T06:43:07.740289"
end_time: "2024-06-10T06:43:39.589881"
status: {
status_code: "Unset"
description: ""
}
attributes: {
traceloop.span.kind: "workflow"
traceloop.entity.name: "RunnableSequence.langchain.workflow"
traceloop.entity.input: "{"args": [], "kwargs": {"input": "What is the difference between llm and slm?"}}"
traceloop.entity.output: "LLM stands for Large Language Model, a type of artificial intelligence model that was trained on huge amounts of text data. LLMs can now write code just like humans. SLM stands for Stop Length Model, another type of AI language model that is used to generate human-like text output in a smaller forma...
view all
"
}
events: [
]
links: [
]
resource: {
attributes: {
service.name: "promptflow"
collection: "trace-llm"
}
schema_url: ""
}
external_event_data_uris: [
]
}

this is the code i'm working on :
from promptflow.tracing import start_trace

start_trace()

from opentelemetry.instrumentation.langchain import LangchainInstrumentor

instrumentor = LangchainInstrumentor()
if not instrumentor.is_instrumented_by_opentelemetry:
instrumentor.instrument()

import os
from langchain_community.llms import LlamaCpp
from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler

callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
llm = LlamaCpp(
model_path = "D:/RAG/ragas/models/tinyllama-1.1b-chat-v0.3.Q4_K_S.gguf",
temperature = 0.75,
max_tokens = 2000,
top_p = 0.95,
top_k = 40,
callback_manager = callback_manager,
verbose = True
)
from langchain.prompts import PromptTemplate

Custom_Prompt_Template = """
You are world class documentation writer.
input: {input}
output:
"""

prompt = PromptTemplate(input_variables= ["input"], template= Custom_Prompt_Template)

llm_chain = prompt | llm

input = "What is the difference between llm and slm?"
llm_chain.invoke({"input": input})

how to trace properly with above huggingface model

[brynn-code]

@thy09 Could you please take a look at this? Thanks.

[thy09]

This is by design that the traces are produced by "LangchainInstrumentor" which is a 3rd party contract. So PF only show the parent-child structure in UI but no more information.
It would be a feature ask to show detailed langchain information in UI, it requires the traces data with PF's data contract.

[jankigabani]

is the feature that shows detailed langchain information is working or present right now?

[thy09]

See this issue: #3080

[jankigabani]

Got it