Skip to content

cache grounding model results to avoid redundant API calls - #218

Open
dajiaohuang wants to merge 1 commit into
simular-ai:mainfrom
dajiaohuang:fix/issue-178-grounding-cache
Open

dajiaohuang wants to merge 1 commit into
simular-ai:mainfrom
dajiaohuang:fix/issue-178-grounding-cache

Conversation

@dajiaohuang

Copy link
Copy Markdown

Summary

Cache grounding model results in OSWorldACI.generate_coords() to avoid redundant API calls when the same (ref_expr, screenshot) pair is queried multiple times.

Problem

The grounding model is called in generate_coords() every time coordinates are needed, even when the same (reference expression, screenshot) pair has already been queried in the current execution cycle. This leads to redundant API calls and increased latency.

Fix

  • Added _grounding_cache dict to OSWorldACI.__init__ to store computed coordinates
  • In generate_coords(), check cache before calling the grounding model
  • Use (ref_expr, hash(screenshot_bytes)) as the cache key
  • Store computed coordinates in cache after each model call

Verification

  • python3 -m py_compile gui_agents/s3/agents/grounding.py passes

Closes #178

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce grounding model call frequency

1 participant