v0.7.0

What's Changed

• CLI Example for Windows by @programminx-askui in #68
• Experimental: Chat Web App by @adi-wan-askui in #76
• Refactor tool integration with models by @adi-wan-askui in #77
• Feat/add android agent by @adi-wan-askui in #79
• Fix android agent by @mlikasam-askui in #80
• refactor: OpenRouter model integration and settings by @onur-askui in #75

🚀 Features

• OpenRouter Improvements: Response schema support and enhanced, more flexible configuration through OpenRouterSettings (corresponding environment variables with OPEN_ROUTER_ prefix)
• Android Agent Support: New AndroidVisionAgent class for Android device automation with corresponding tools using Pure Python ADB
• Exception Tool: act() raises exceptions and fails tests automatically if it runs into issues

🐞 Bug Fixes

• Fix nested and recursive schema handling with get()
• Fix telemetry flushing by removing it to prevent hanging up within test or experimental chat api

🚨 Breaking Changes

• OpenRouter Model Naming: OpenRouterGetModel renamed to OpenRouterModel

Full Changelog: v0.6.0...v0.7.0

v0.6.0

What's Changed

• Update openrouter example by @onur-askui in #71
• fix(PynputAgentOs): Fix missing run_command by @programminx-askui in #70
• fix: Change private variable _tools to public tools for accessibility by @onur-askui in #73
• docs(agent): fix type annotation of VisionAgent.act() by @adi-wan-askui in #74
• docs(agent): update docs + tests with regards to nested/recursive response schemas by @adi-wan-askui in #72

🚀 Features

• Use recursive response schemas with VisionAgent.get()

🐞 Bug Fixes

• Use nested response schemas with VisionAgent.get()
• Response type of VisionAgent.get() was ResponseSchemaBase instead of class extending ResponseSchemaBase when passing that class with the response_schema arg

🚨 Breaking Changes

• Renamed VisionAgent._tools to VisionAgent.tools as tools should be public

Full Changelog: v0.5.3...v0.5.4

v0.5.3

What's Changed

• Add "switch" as element class locator by @adi-wan-askui in #66
• Open up underlying messages of VisionAgent.act() and extract api from chat (experimental) by @adi-wan-askui in #64
• OpenRouter Integration by @onur-askui in #67

🚀 Features

• Add OpenRouter integration
• Allow passing messages and callbacks to VisionAgent.act()
• Add "switch" element class locator

🐞 Bug Fixes

• Fix logging and reporting of act()
• Fix askui.tools.anthropic.computer.ComputerTool, e.g., hard coding of height and width of screen and that height and width were not set when no screenshot was taken before clicking, mouse moving etc. raising an exception

Other

• Update README to include OpenRouter integration and usage examples
• Improve docstrings for new features
• Structure deps better, e.g., making pynput optional
• Add cursor rules

🚨 Breaking Changes

• ActModel.act() signature changed, original goal parameter is now in messages[0]["content"]
• Rename AgentOs.set_display's displayNumber parameter to display (snake_case, consistent with AskUiControllerClient constructor parameter)
• Rename AskUiControllerClient.set_display's displayNumber parameter to display (snake_case, consistent with AskUiControllerClient constructor parameter)
• Rename AgentOs.mouse to AgentOs.mouse_move (including implementations PynputAgentOs and AskUiControllerClient)
• Remove unused image API in chat

Full Changelog: v0.5.2...v0.5.3

v0.5.2

Rerelease of v0.5.1 as something went wrong releasing.

Full Changelog: v0.5.1...v0.5.2

v0.5.1

🐞 Bug Fixes

• Locate retry to include screenshot by @onur-askui in (#65)

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

• Retry on ElementNotFoundError by @onur-askui in (#54)
• Resolve issues where OS agent fails to stop on close by @mlikasam-askui and @onur-askui in (#58)
• Introduces mouse_up and mouse_down to VisionAgent by @onur-askui in (#60)
• Run cli commands via agent os by @onur-askui and @adi-wan-askui in (#61)
• Processing UI-TARS model coordinate outputs by @zenjieli in (#56)
• Report AskUI computer use by @onur-askui in (#57)
• Register your own models by @adi-wan-askui in (#62)
• Set mause move delay to 500ms by @onur-askui in (#63)

🚀 Features

• New models parameter in VisionAgent constructor
• Retry mechanism into VisionAgent for locate method
• Add mouse_up and mouse_down methods to VisionAgent
• Forward cli command to agentos
• Introduce abstract retry and move default retry to ConfigurableRetry class

🐞 Bug Fixes

• Typing & linting errors
• Some typing issues (mypy) across codebase
• Some examples in docstrings
• Update reporter message from "Anthropic Computer Use" to "AskUI Computer Use"
• Set mouse move delay to 500ms
• Resolve issue where OS agent fails to stop on close
• Fix invalid variable access
• Fix code example of using custom models
• Add validation to ConfigurableRetry

Other

• Add integration tests for custom models
• Make examples in docs clearer
• Add Retry utility class for configurable retry mechanisms
• Add return types in ConfigurableRetry
• Add debug message to _locate
• Rename timeout parameter and remove telemetry for run_command

🚨 Breaking Changes

• Changed order of arguments of VisionAgent.get() to fix typing errors (image moved to last position)
• model_router parameter removed from VisionAgent constructor --> replaced by models parameter
• No default routing to Anthropic models as a fallback to AskUI models (routing only to "askui" if model parameter not set)
• Raising exceptions.ModelNotFoundError and exceptions.ModelTypeMismatchError that may be raised by
  • VisionAgent.act()
  • VisionAgent.click()
  • VisionAgent.get()
  • VisionAgent.locate()
  • VisionAgent.mouse_move()

Full Changelog: v0.4.7...v0.5.0

v0.4.7

Rerelease of v0.4.6 as something went wrong releasing.

Full Changelog: v0.4.6...v0.4.7

v0.4.6

What's Changed

• feat: retry act on 529 and 429 errors by @onur-askui in #55

Full Changelog: v0.4.5...v0.4.6

v0.4.5

What's Changed

• increase the default timeout of act request to AskUI api to increase robustness

Full Changelog: v0.4.4...v0.4.5

v0.4.4

What's Changed

• Cl 1388 act anthropic via askui by @onur-askui in #53

New Contributors

• @onur-askui made their first contribution in #53

Full Changelog: v0.4.3...v0.4.4