Skip to content

feat: copilot chat with custom questions#5

Open
Maximvdw wants to merge 1 commit intodsp05:mainfrom
Maximvdw:main
Open

feat: copilot chat with custom questions#5
Maximvdw wants to merge 1 commit intodsp05:mainfrom
Maximvdw:main

Conversation

@Maximvdw
Copy link
Copy Markdown

This feature adds a keyboard shortcut (default Ctrl+Shift+I) to open Copilot chat to ask a specific question.

Important changes:

  • Added chat panel to avoid clutter in the "side panel"
  • Added permission for active tab. This is required because the shortcut is a background process
    that sends a "chat:open" command to the current overleaf tab page to open the chat window.

Design choices:

  • Ctrl+Shift+I is chosen as a similar command in VSCode for GitHub Copilot (CTRL+I). Since CTRL+I is already used by overleaf for \textit I've added shift. But users can configure this themselves by going to extension shortcuts
  • I auto focus on the chat text input after opening the chat window
  • I currently do not provide context of the paper in the question. Preferably this could be a toggle/switch where users can opt-in to ask a question while also providing the last paragraph
  • When selecting text and pressing CTRL+SHIFT+I, the question is asked "about" the selection and the button will say "Replace" instead of "Insert"

Images:
chrome_X1VZs24EqK
chrome_8vTP7M1Qqq

Optional todos:

  • I've added a "copilot:editor:append" event to insert new data. Preferably the onAcceptSuggestion function could also make use of this.

Closes #4

Comment thread src/utils/chat.ts
`${question}`
);
} else {
return (
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no content from the paper is included in the prompt right?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, currently it only includes the question

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this expected? GPT doesn't have the context to continue the "academic paper" in the Overleaf editor.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and no. In my examples I usually selected something and asked a question to modify that selected text. Or I asked for non-contextual question such as "create a table to do this or that" which did not require input about the paper. But I agree it could be useful to add context.

Comment thread src/utils/chat.ts

function buildAskPrompt(question: string, selection?: string) {
if (selection) {
return (
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this prompt seems a bit unclear to me, would it be more intuitive to ask GPT to answer the question based on the selection?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that the reply is often:

"Sure thing! In order to solve xxxx you write:"

Where the answer includes a human response and explanation instead of simply giving the answer

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hence why the prompt is forced to only provide the output

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part makes sense to me. I'm a bit confused about how the user is expected to use this feature. I think when a user selects some text, they may want to:
1 rewrite the selection maybe with some specific instructions, or
2 ask a question regarding the selection
It looks like the prompt tries to do 1 but it also tries to answer a question.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I aim to get an improvement/latex as a response rather than an explanation. When selecting text it aims to answer the question with the selection as context.
Other than the existing "improvement" button in this extension the user can ask more specific improvements such as "make this shorter", "change the color of the headings in this table to red",...

When not selecting text it just tries to answer the question with a response that can be inserted into the paper.

So your 1. point is correct. Answering questions about a selection is not my aim with this PR.

Github copilot solves this ambiguity using commands such as /explain and /fix that indicate what you want to do with a selection. Future changes could incorporate something similar.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... Or we could add a "improve" and "explain" button?

Comment thread src/contentMainScript.ts
maxPromptWords = e.detail[CONFIG_MAX_PROMPT_WORDS] || maxPromptWords;
}

function onEditorAppend(
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's use "insert", since it's not always adding content to the end of the document.

Comment thread src/background.ts
chrome.commands.onCommand.addListener((command) => {
if (command === "chat") {
chrome.tabs.query({ active: true, currentWindow: true }, function (tabs) {
if (tabs && tabs[0] && tabs[0].id) {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: tabs?[0]?.id

@Maximvdw
Copy link
Copy Markdown
Author

I was a bit busy with other work. But I've tested the extension and copilot chat for a month and have some updates pending to this PR that I will push soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Ask specific questions

2 participants