feat(websearch): search suggestion and various reworks

[Ori-Riaru]

This primarily adds the ability to get suggestions for queried engine while in single mode or when using an engine prefix. Behavior should remain mostly the same when using an old / default config to see changes you will need to configure the new options (see provider readme).

Changes

• Add previously missing development dependencies for nix devshell
• Add direnv to auto start nix devshell
• Add ability to configure engines with any custom suggestions api using suggestions_url and suggestion_path
• Allow url protocols other than https
• Allow querying "Search: " results for default_single engines when searching in single mode
• Display suggestions for queried engines while in single mode or when using engine prefix
• Add various config options for engine_finder_* which can be used to disable the ability to search for/find engines when in single mode
• Show engine prefix in UI similar to providerlist
• Show query text instead instead of engine name when querying an engine
• Refactor

TODO

• Find some way to asynchronously stream / request new items on client mitigating slow api results

Also

Unrelated but the update-nix-hash action is failing because it needs "Read and write permissions" to create a PRs. The option should be here in the repo settings

[Ori-Riaru]

This should be good now. I didn't do async suggestions yet since it seems like it will be a bigger change for another pr.

Here is an example config for testing.

websearch.toml

max_api_items = 10
engine_finder_default_single = false

# name and url are required 
[[entries]]
default = true
name = "Example"
url = "https://www.example.com/search?q=%TERM%"

[[entries]]
default = true
default_single = true
name = "DuckDuckGo"
icon = "duckduckgo"
prefix = "@d"
url = "https://duckduckgo.com/?q=%TERM%"
suggestions_url = "https://ac.duckduckgo.com/ac/?q=%TERM%"
suggestions_path = "#.phrase"

[[entries]]
name = "Google"
icon = "google"
prefix = "@g"
url = "https://www.google.com/search?q=%TERM%"
suggestions_url = "https://suggestqueries.google.com/complete/search?client=firefox&q=%TERM%"
suggestions_path = "1"

[[entries]]
name = "Crunchyroll"
prefix = "@anime"
url = "https://www.crunchyroll.com/search?q=%TERM%"
# Suggestions from MyAnimeList for Crunchyroll search
suggestions_url = "https://myanimelist.net/search/prefix.json?type=all&keyword=%TERM%&v=1"
suggestions_path = "categories.#(type==\"anime\").items.#.name"

[[entries]]
name = "Amazon"
icon = "amazon"
prefix = "@shop"
url = "https://www.amazon.ca/s?k=%TERM%"

[[entries]]
name = "Newegg"
prefix = "@shop"
url = "https://www.newegg.ca/p/pl?d=%TERM%"
suggestions_url = "https://www.newegg.ca/api/SearchKeyword?keyword=%TERM%"
suggestions_path = "suggestion.keywords.#.keyword"

[abenz1267]

For disclosure: how much of this is vibe coded?

[Ori-Riaru]

This was mostly written by myself. I use AI for review f.e. "How can I make <thing> better" or "Is there any bugs in this code".

Is there something I've done wrong / can do better? I am a student so your feedback is appreciated.

[abenz1267]

There's always something everyone can do better :P.

Would you mind if I just refactor as i see fitting?

Technically everything seems to be working.

[Ori-Riaru]

Yes that's ok. If you are going to change things there are some thing didn't do because they were breaking or required larger changes.

The current and updated provide emulates a fallback a system by always returning results. However I think it would be better to implement a fallback system in walker similar to the already existing empty providers system. This would remove the need for distinguishing between default and default_single for websearch and also be more configurable.

For the async results it seems the current system only allows for updating existing items and not amending new items to previous queries this makes it difficult to de-bounce queries or add results as they become available. I was going to add this but I don't really want to mess with the protocol as this is beyond my abilities.