Conversation
|
|
||
| def __init__(self): | ||
| super().__init__() | ||
| openai.api_base = "http://localhost:8080/v1" |
There was a problem hiding this comment.
I will hate myself for starting like this but I am just trying to go ><
If we do grammar constraints this wont work. If we do LMQL constraints, this might work?
There was a problem hiding this comment.
Can you explain the distinction when we meet
| continue | ||
|
|
||
| # we could not compile the program so we just return our best effort | ||
| # TODO: do a GPT-Judge thing here and ask which was the best then return that one? |
There was a problem hiding this comment.
Good TODO want to make sure it doesn't get lost in the PR
devanshanker
left a comment
devanshanker
left a comment
There was a problem hiding this comment.
Good looks, we should make sure the methodology and instructional/ToT Q/A process is consistent across local models and API (at least to the extent that context lengths / LMQL allow)
| questions = None | ||
| with os.open(PromptFiles.COMPOSITIONAL_GBNF.value) as f: | ||
| prompt = f.read() | ||
| questions = yaml.safe_load(prompt) |
There was a problem hiding this comment.
|
Current state of affair's is that I can generate constants section with 100% compilation. I am splitting inference by reasoning with mixtral and coding with claude. I could do all with claude but this way "saves money" |
2 participants
No description provided.