Skip to content

modify value_outputs_unwrap method - #61

Open
whatchang wants to merge 1 commit into
princeton-nlp:masterfrom
whatchang:master
Open

modify value_outputs_unwrap method#61
whatchang wants to merge 1 commit into
princeton-nlp:masterfrom
whatchang:master

Conversation

@whatchang

Copy link
Copy Markdown

First, thank you for releasing your awesome paper and code.

I found one problem when I ran your Quick Start code that is written in README.
(That is below)

import argparse
from tot.methods.bfs import solve
from tot.tasks.game24 import Game24Task

args = argparse.Namespace(backend='gpt-4', temperature=0.7, task='game24', naive_run=False, prompt_sample=None, method_generate='propose', method_evaluate='value', method_select='greedy', n_generate_sample=1, n_evaluate_sample=3, n_select_sample=5)

task = Game24Task()
ys, infos = solve(args, task, 900)
print(ys[0])

The problematic line of code I found is shown below.
(Thas is in src->tot->tasks->game24.py line 91)

sum(value * value_names.count(name) for name, value in value_map.items())

In the above code, the problem was caused by the value_names being in the list type rather than a string.

As a result, we can see that the first string of value_names does not count even though it has 'sure' in it, as shown in the WATCH part of the figure below.

오류 n이 3일때

This also happens when n_evaluate_sample=1.

오류 n이 1일때

So, I modified the code to look like this.

sum(value * target_value.count(name) for name, value in value_map.items() for target_value in value_names)

Thank you for reading and have a great day!

@ysymyth

ysymyth commented Jan 16, 2025

Copy link
Copy Markdown
Collaborator

thanks! i think the problem is, what if target_value is a string like "sure sure likely impossible" or "i am sure i do not know"? exact match is more conservative but maybe less risky

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants