Skip to content

Compiler in the loop integration (anthropic)#33

Open
devanshanker wants to merge 17 commits intomainfrom
citl
Open

Compiler in the loop integration (anthropic)#33
devanshanker wants to merge 17 commits intomainfrom
citl

Conversation

@devanshanker
Copy link
Copy Markdown
Collaborator

  • add max_retries parameter for claude executions
  • automatically retry using 1. compilation then 2. execution errors

future work

  • create similar prompts and integration for openAI and other approaches (should be quick)
  • consider adding anthropic rewrites to all models instead? for efficiency
  • might be worth augmenting compilation and execution errors

)


retries, retries_dir, fstub = max_retries, os.path.join(os.curdir, 'retries', 'retries'), 'temp.scenic'
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup this looks way better

verbose: bool,
max_retries: int
) -> str:
verbose_retry = True
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can leave it all under the same verbose argument

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think it might be better to separate out so we can watch it retry as it runs without all the other verbose output it was hard to see what's happening in real time when I tried verbose=true and these runs take 30-60m+

with open(PromptFiles.AST_FEEDBACK_CLAUDE.value) as f:
prompt = f.read()

prompt = prompt.format(
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would bubble this up into the caller so the anthropic _ast_prompt function. The reason being is that when we do this for gpt + others, they will use different formatting and so this would get pushed up anyways

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smart will do this and add gpt and rerequest

… split reasoning, program combining - add intelligent program reduction/redundancy removal
…tion occasionally has repeat sections -> need debug output
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants