“Pywinauto recorder” records user interface actions and saves them in a Python script. The saved Python script can be run to replay the user interface actions previously recorded.
“Pywinauto recorder” is a unique inspect/record/replay tool in the open source field because it generates reliable scripts without hard-coded coordinates thanks to Pywinauto.
Pywinauto is a library that uses accessibility technologies allowing you to automate almost any type of GUI: MFC, VB6, VCL, simple WinForms controls and most of the old legacy apps, WinForms, WPF, Store apps, Qt, and browsers.
Pywinauto_recorder for Windows
“pywinauto_recorder.exe” is a standalone application, it’s the compiled version of “pywinauto_recorder.py” for 64-bit Windows.
“pywinauto_recorder.py” is the main source, you will find the Pyhon code downloading the github clone.
The recorder is started in display information mode, a tray icon is added in the right-side of the Windows Taskbar.
Press CTRL+SHIFT+f to copy the code that finds the element colored green or orange to the clipboard.
Press CTRL+ALT+r to switch to “Record” mode.
If the element below the mouse cursor can be uniquely identified, it will turn green or orange.
You can then click or perform another action on the user interface and it will be recorded in the generated Python script.
Repeat this process performing a few actions on the user interface and when you’re done press CTRL+ALT+r to end recording.
The generated Python script is saved in the “Pywinauto recorder” folder in your home folder and copied in the clipboard.
Click on “Quit” in the tray menu.
To replay a Python script, you can drag and drop it to “pywinauto_recorder.exe”
Pywinauto_recorder_exe does not work on all PCs. If it doesn’t work use the Python version. Help me to find a solution by filling this form.
- Some transparent icons are displayed at the top left of the screen:
an icon corresponds to Record/Stop mode. Press CTRL+ALT+r to switch.
a magnifying glass icon corresponds to Display information mode.
a bulb icon corresponds to Smart mode. Press CTRL+ALT+S to activate it.
another icon displays a green bar at each iteration of the loop. It allows you to see how fast the loop is running.
- The main of “Pywinauto recorder” is an infinite loop where at each iteration it:
finds the path of the element under the mouse cursor. The path is formed by the window_text and control_type pair of the element and all its ancestors.
searches for an unambiguous path, if found, it colors the element region green or orange.
records a user action in a file involving the last recognized unique path.
To reflect the position of the mouse cursor as closely as possible, an offset is added to the user actions recorded in the generated Python script. This offset is proportional to the size of the element and relative to the center of the element.
- If the path of the element under the mouse cursor is not ambiguous, the region of the element is colored green. Otherwise two strategies are used to try to disambiguate the path in the following order:
All elements having the same path are ordered in a 2D array. The path of the element region under the mouse cursor is disambiguated by adding a row index and a column index so that it is colored orange. The other element regions are colored red
When Smart mode is enabled, an element whose path is unambiguous is searched on the same line on the left, if found its region is colored blue and the element under the mouse cursor is colored orange.
Pywinauto recorder tests
The tests are good examples of how to use the player API. The tests are grouped by application:
Pywinauto recorder API
pip install -U pywinauto_recorder
The following table lists the useful methods and classes for the ‘Player’ and ‘Recorder’ modules:
This module contains functions to replay a sequence of user actions automatically.
This module contains functions and classes allowing to record a sequence of user actions.
Contributing to Pywinauto recorder
You are here to help on Pywinauto recorder? Awesome, feel welcome and read the following sections in order to know how to ask questions and how to work on something.
All members of our community are expected to follow our Code of Conduct. Please make sure you are welcoming and friendly in all of our spaces.
Get in touch
Contributing to development
If you want to deep dive and help out with development on Pywinauto recorder, then first get the project installed locally. After that is done we suggest you have a look at tickets in our issue tracker that are labelled Good First Issue. These are meant to be a great way to get a smooth start and won’t put you in front of the most complex parts of the system.
If you are up to more challenging tasks with a bigger scope, then there are a set of tickets with a Enhancement tag. These tickets have a general overview and description of the work required to finish. If you want to start somewhere, this would be a good place to start.
When contributing code, then please follow the standard Contribution Guidelines set forth at contribution-guide.org.
Contributing to documentation
Documentation for Pywinauto recorder itself is hosted at https://pywinauto-recorder.readthedocs.io.
At https://docs.readthedocs.io there are guidelines around writing and formatting documentation for the project.
Code of Conduct
Like the technical community as a whole, the Pywinauto recorder team and community is made up of a mixture of professionals and volunteers from all over the world, working on every aspect of the mission - including mentorship, teaching, and connecting people.
Diversity is one of our huge strengths, but it can also lead to communication issues and unhappiness. To that end, we have a few ground rules that we ask people to adhere to. This code applies equally to founders, mentors and those seeking help and guidance.
This isn’t an exhaustive list of things that you can’t do. Rather, take it in the spirit in which it’s intended - a guide to make it easier to enrich all of us and the technical communities in which we participate.
This code of conduct applies to all spaces managed by the Pywinauto recorder project. This includes IRC, the mailing lists, the issue tracker, and any other forums created by the project team which the community uses for communication. In addition, violations of this code outside these spaces may affect a person’s ability to participate within them.
Be friendly and patient.
Be welcoming. We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, colour, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.
Be considerate. Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions. Remember that we’re a world-wide community, so you might not be communicating in someone else’s primary language.
Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It’s important to remember that a community where people feel uncomfortable or threatened is not a productive one. Members of the Pywinauto recorder community should be respectful when dealing with other members as well as with people outside the Pywinauto recorder community.
Be careful in the words that you choose. We are a community of professionals, and we conduct ourselves professionally. Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behavior aren’t acceptable. This includes, but is not limited to:
Violent threats or language directed against another person.
Discriminatory jokes and language.
Posting sexually explicit or violent material.
Posting (or threatening to post) other people’s personally identifying information (“doxing”).
Personal insults, especially those using racist or sexist terms.
Unwelcome sexual attention.
Advocating for, or encouraging, any of the above behavior.
Repeated harassment of others. In general, if someone asks you to stop, then stop.
When we disagree, try to understand why. Disagreements, both social and technical, happen all the time and Pywinauto recorder is no exception. It is important that we resolve disagreements and differing views constructively. Remember that we’re different. The strength of Pywinauto recorder comes from its varied community, people from a wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesn’t mean that they’re wrong. Don’t forget that it is human to err and blaming each other doesn’t get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.