This recorder is still at a very early stage of development.
“Pywinauto recorder” records user interface actions and saves them in a Python script. The saved Python script can be run to replay the user interface actions previously recorded.
“Pywinauto recorder” is a unique “record-replay” tool in the open source field because it generates reliable scripts without hard-coded coordinates thanks to Pywinauto.
Pywinauto is a library that uses accessibility technologies allowing you to automate almost any type of GUI: MFC, VB6, VCL, simple WinForms controls and most of the old legacy apps, WinForms, WPF, Store apps, Qt5, browsers.
Pywinauto_recorder.exe for Windows¶
“pywinauto_recorder.exe” is a standalone application, it’s the compiled version for Windows.
- Double click on “pywinauto_recorder.exe” to start the recorder.
- When the recorder is started, it is in “Pause” mode.
- Press CTRL+SHIFT+f to copy the code that finds the element colored green or orange to the clipboard.
- Press CTRL+ALT+r to switch to “Record” mode.
- If the element below the mouse cursor can be uniquely identified, it will turn green or orange.
- You can then click or perform another action on the user interface and it will be recorded in the generated Python script.
- Repeat this process performing a few actions on the user interface and when you’re done press CTRL+ALT+r to return to end recording.
- Eventually, press CTRL+ALT+q to exit the recorder.
- The generated Python script is saved in the “Record files” folder and copied in the clipboard.
- To replay a Python script, you can drag and drop it to “pywinauto_recorder.exe”
- Two transparent icons are displayed at the top left of the screen:
- the first icon corresponds to Record/Pause mode. Press CTRL+ALT+r to switch.
- the second icon displays a green bar at each iteration of the loop. It allows you to see how fast the loop is running.
- The main of “Pywinauto recorder” is an infinite loop where at each iteration it:
- finds the path of the element under the mouse cursor. The path is formed by the window_text and control_type pair of the element and all its ancestors.
- searches for an unambiguous path, if found, it colors the element region green or orange.
- records a user action in a file involving the last recognized unique path.
To reflect the position of the mouse cursor as closely as possible, an offset is added to the user actions recorded in the generated Python script. This offset is proportional to the size of the element and relative to the center of the element.
- If the path of the element under the mouse cursor is not ambiguous, the region of the element is colored green. Otherwise two strategies are used to try to disambiguate the path in the following order:
- All elements having the same path are ordered in a 2D array. The path of the element region under the mouse cursor is disambiguated by adding a row index and a column index so that it is colored orange. The other element regions are colored red
- An element whose path is unambiguous is searched on the same line on the left, if found its region is colored blue and the element under the mouse cursor is colored orange.
Pywinauto recorder API¶
pip install -U pywinauto_recorder
Contributing to Pywinauto recorder¶
You are here to help on Pywinauto recorder? Awesome, feel welcome and read the following sections in order to know how to ask questions and how to work on something.
All members of our community are expected to follow our Code of Conduct. Please make sure you are welcoming and friendly in all of our spaces.
Get in touch¶
Contributing to development¶
If you want to deep dive and help out with development on Pywinauto recorder, then first get the project installed locally. After that is done we suggest you have a look at tickets in our issue tracker that are labelled Good First Issue. These are meant to be a great way to get a smooth start and won’t put you in front of the most complex parts of the system.
If you are up to more challenging tasks with a bigger scope, then there are a set of tickets with a Feature or Improvement tag. These tickets have a general overview and description of the work required to finish. If you want to start somewhere, this would be a good place to start (make sure that the issue also have the Accepted label). That said, these aren’t necessarily the easiest tickets. They are simply things that are explained. If you still didn’t find something to work on, search for the Sprintable label. Those tickets are meant to be standalone and can be worked on ad-hoc.
You can read all of our /development/index to understand more the development of Pywinauto recorder. When contributing code, then please follow the standard Contribution Guidelines set forth at contribution-guide.org.
Code of Conduct¶
Like the technical community as a whole, the Pywinauto recorder team and community is made up of a mixture of professionals and volunteers from all over the world, working on every aspect of the mission - including mentorship, teaching, and connecting people.
Diversity is one of our huge strengths, but it can also lead to communication issues and unhappiness. To that end, we have a few ground rules that we ask people to adhere to. This code applies equally to founders, mentors and those seeking help and guidance.
This isn’t an exhaustive list of things that you can’t do. Rather, take it in the spirit in which it’s intended - a guide to make it easier to enrich all of us and the technical communities in which we participate.
This code of conduct applies to all spaces managed by the Pywinauto recorder project. This includes IRC, the mailing lists, the issue tracker, and any other forums created by the project team which the community uses for communication. In addition, violations of this code outside these spaces may affect a person’s ability to participate within them.
- Be friendly and patient.
- Be welcoming. We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, colour, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.
- Be considerate. Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions. Remember that we’re a world-wide community, so you might not be communicating in someone else’s primary language.
- Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It’s important to remember that a community where people feel uncomfortable or threatened is not a productive one. Members of the Pywinauto recorder community should be respectful when dealing with other members as well as with people outside the Pywinauto recorder community.
- Be careful in the words that you choose. We are a community of
professionals, and we conduct ourselves professionally. Be kind to
others. Do not insult or put down other participants. Harassment and
other exclusionary behavior aren’t acceptable. This includes, but is
not limited to:
- Violent threats or language directed against another person.
- Discriminatory jokes and language.
- Posting sexually explicit or violent material.
- Posting (or threatening to post) other people’s personally identifying information (“doxing”).
- Personal insults, especially those using racist or sexist terms.
- Unwelcome sexual attention.
- Advocating for, or encouraging, any of the above behavior.
- Repeated harassment of others. In general, if someone asks you to stop, then stop.
- When we disagree, try to understand why. Disagreements, both social and technical, happen all the time and Pywinauto recorder is no exception. It is important that we resolve disagreements and differing views constructively. Remember that we’re different. The strength of Pywinauto recorder comes from its varied community, people from a wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesn’t mean that they’re wrong. Don’t forget that it is human to err and blaming each other doesn’t get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.