How to reproduce for Webshop

[Ashwini4869]

As mentioned in the paper, the authors have evaluated AgentKit on two environments: Crafter and Webshop. As of last week, you have added the code in the examples/crafter folder to run crafter. I want to run for Webshop. My initial thought was to replace the environment and Graph with that of Webshop instead of crafter. However, exploring further, I found that the other files like compose_prompt, post_processing, all were very much specific to crafter.

So my question is how can I run Webshop?

Again, thumbs up for the great work with this framework and paper. Thank you in advance.

[Ashwini4869]

@Holmeswww , Can you please provide some insights on this?

[Holmeswww]

Hi,

Thanks for your insights in our work. We attached the prompts for webshop in paper. If you just use the prompts for webshop and then use the same planner post-processing, it should get you quite far already. We didn't make much customizations apart from removing all crafter related stuff.

I'm quite busy with work related problems. I will come back to upload the webshop code in a week or so when I'm finished. Apologies for the delay.

[Holmeswww]

match_threshold=80
fallback_steps=15
max_length=35

import re

def parse_obs(observation: str, env, max_length, match_threshold, step, fallback_steps):
    observation = observation.split(" [SEP] ")
    observation = observation[observation.index("Instruction:"):]
    if step <= max_length - fallback_steps:
        note = """
- Try to make a purchase in {} steps.
- Puchase a product with no less than {}% match with the task. No need to look for a perfect match. If multiple products match the task, purchase the one with the highest match.
- The search engine uses bag-of-words. Optimize the search terms accordingly and don't put price in search. If you cannot find relevant items, consider going to the next page of the search results.
- The search result page only contain short product descriptions. Click on the product to reveal other customizable attributes/options.
""".format(max_length - fallback_steps - step, match_threshold)
    else:
        note = """
- Your *highest priority goal* is to make a purchase reasonablely close to the task specification within a HARD LIMIT of {} steps, and you currently have {} steps remaining.
- First, attempt the search using the full task request without the price as the search-term, then purchase the closest item on the first three pages of the search results, don't worry about perfect match.
- The search result page only contain short product descriptions. Click on the product to reveal other customizable attributes/options.
- If none of the products match the task specification, don't worry about the details and just purchase an item of the same general category as the request.
""".format(fallback_steps, max_length - step)

    general_note = """
- If there are any customizable attributes/options on the product page, ALWAYS select all available customizations before placing the order.
- All customizations are available immediately after clicking on the product. If you cannot find the customization, then it is not available.
- The customizations may not match the task exactly, so allow for some flexibility.
- Do not click on the description, features, or reviews page (They offer no information).
"""
    note = note.strip() + "\n" + general_note.strip()
    # task_str = "# Task\n{}\n\n# Notes\n{}".format(observation[1], note.strip())
    task_str = observation[1]
    obs = str(observation[2:])

    available_actions = env.get_available_actions()

    available_actions_list = []
    for btn in available_actions['clickables']:
        if btn == 'search':
            continue # skip the search button
        available_actions_list.append("- click[{}]".format(btn))
    if available_actions['has_search_bar']:
        available_actions_list.append("search[{}]".format("$search_term"))

    obs_str = "## Webpage content\n{}\n\n## Permitted actions\n{}".format(obs, "\n".join(available_actions_list))
    return task_str.strip(), note.strip(), obs_str

def match_act(action: str, allowed_actions: dict):
    """
    Required action formatting:
    - search[string] (search for a string, only available if 'has_search_bar': True)
    - click[subject] (click on a clickable element)

    Example of allowed actions:
    {'has_search_bar': False,
    'clickables': ['back to search',
    '< prev',
    'description',
    'features',
    'reviews',
    'buy now',
    'black',
    'bodycon',
    'fluff black',
    'polka dot',
    'red',
    'red a',
    'small',
    'medium',
    'large',
    'x-large']}
    """
    if 'search' in allowed_actions["clickables"]:
        allowed_actions["clickables"].remove('search')
    # use regex to ensure that the action is formatted correctly
    if re.match(r"search\[.+\]", action):
        if allowed_actions['has_search_bar']:
            return action, None
        else:
            return None, "The action 'search[]' is not allowed, try to go back to search first."
    elif re.match(r"click\[.+\]", action):
        if action[6:-1] in allowed_actions['clickables']:
            return action, None
        else:
            return None, "The target '{}' is not clickable this page.".format(action[6:-1])
    else:
        return None, "The action '{}' is not allowed on the current page.".format(action)

Here's the snippet that's most different from crafter.

[Ashwini4869]

Hey! Thanks for the reply. No problems for the delay.
Thanks for providing the idea and for the code snippet. I can now try running for webshop. Also, will be waiting for the webshop example upload.
Thanks again.