아래 프롬 넣어보고 싶은데 해줄사람 혹시 있나
# Request for a Multimodal Web Automation Framework Based on Playwright
## Context
I'm currently using Microsoft's Playwright for web automation, but I'm facing a significant challenge: whenever a website's structure changes, I need to rewrite large portions of my code to adapt to the new selectors and elements. This is time-consuming and inefficient.
## What I'm Looking For
I'm seeking a framework or solution that builds on top of Playwright with the following key features:
1. **Multimodal Understanding**: The ability to understand websites visually (like a human would) rather than solely relying on DOM selectors. The tool should "see" the page the way I do.
2. **Resilient Element Identification**: When website elements change positions or attributes, the tool should still be able to find them based on their visual characteristics or context.
3. **Adaptive Code Generation**: The ability to automatically adjust automation sequences/routines when the website structure changes, minimizing manual intervention.
4. **Visual Element Selection**: A way to visually select elements during automation routine creation/maintenance, rather than having to inspect the DOM and write selectors manually.
5. **Practical & Production-Ready**: Must be a practical solution that can be used in real-world projects, not just an experimental or research tool.
## What I Already Know
I'm aware of Playwright's codegen capability, but I'm looking for something more advanced that incorporates visual understanding. I believe there must be some robust solutions built on top of Playwright that solve this exact problem.
## Specific Questions
- Are there any frameworks that combine computer vision with Playwright for more resilient web automation?
- Do any tools exist that can "understand" a website's UI components contextually rather than just by their DOM position?
- Are there solutions that can automatically adapt to website changes without requiring me to manually update selectors?
- What are the most mature and well-maintained projects in this space?
## Use Case Examples
- If a button changes its class or ID but visually remains the same, the framework should still be able to find and interact with it.
- If a form is redesigned but maintains the same overall purpose, the automation should adapt with minimal changes.
- If elements are rearranged on a page, the framework should understand the new layout without requiring a complete rewrite.
I appreciate specific recommendations of tools that actually exist and have been successfully used in production environments. Theoretical approaches or automation solutions still in early development stages would be less helpful for my current needs.
Your request is quite detailed, which is great! Before I begin my research, could you clarify a couple of things: Are you specifically looking for open-source frameworks, commercial tools, or both? Do you have a preferred programming language for integration, or should the solution be Playwright-compatible regardless of language? Would you be open to hybrid solutions that use AI-driven OCR (e.g.,
Would you be open to hybrid solutions that use AI-driven OCR (e.g., Tesseract, OpenCV) along with Playwright for enhanced element detection? Once I have this information, I’ll find the best available tools and frameworks that align with your needs.
헐 해줄수 있음? 답해서 덧글달게
1) I'm open to commercial tools as long as costs are minimal or controllable. Free options are preferred, or solutions that can leverage existing API balance (OpenAI, etc.). My main concern is avoiding significant additional expenses. 2) I'm working with Python 3.11.11 in a conda environment, so compatibility with this setup would be ideal
3) I'm comfortable with solutions using Tesseract or OpenCV, but I'm looking for well-developed, ready-to-use frameworks rather than having to build something from scratch. My priority is getting quick results without spending excessive time on implementation details.
이렇게 해주면 됨!! 고맙
https://chatgpt.com/share/67c44b44-57f0-8001-afc5-8d3099a4ff75
쓸모있는지만 알려줘. 얼마나 쓸모있는지는 문제 해결에 몇 프로 정도 됐는지 정도 간단하게 ㅎ
다른 모델에선 추천 안해준게 몇개 있는듯 실제로 써봐야겠지만 한 70 80 정도 되는거 같음 ㄱㅅㄱㅅ