Ceci est une ancienne révision du document !
It’s unusual for me to get questions more than once a month. However, today, while I was thinking about what I was going to do for my Python article for December, I got a question from a friend about my suggestions for doing automated testing of his program. Immediately, I suggested pyautogui. I passed a copy of my article from FCM#151, which came out in November 2019. A few hours later, another friend and client contacted me and asked what would be a good library for mouse and keyboard emulation. Once again, I immediately suggested pyautogui and sent him a copy of the article as well. Once things quieted down a bit around the old homestead, I realized that the Cosmic Muffin was trying to tell me something. So I started looking at the latest documentation for pyautogui and found that Al Sweigart has really updated things quite a bit. You can find that documentation at https://pyautogui.readthedocs.io/en/latest/index.html. You might want to get some more background by looking at his online version of Automating The Boring Stuff with Python at https://automatetheboringstuff.com/ and jumping to chapter 20. I STRONGLY suggest that you purchase this book for your library and support Al for all his wonderful work over the years. (Can you tell he’s one of my favorite authors?).
The first thing to do is to install or update pyautogui. Some of my friends and readers seem to have more than one version of Python (and pip) installed. So let’s use some simple commands from the terminal to determine the proper commands for your system. Open a terminal and type python -V and pip –version greg@earth:~$ python -V Python 3.8.10 greg@earth:~$ pip –version pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8) This will give you the current versions of Python version and pip, which should be the same. You might also want to try python3 -V and pip3 –version to verify that the versions are the same. greg@earth:~$ python3 -V Python 3.8.10 greg@earth:~$ pip3 –version pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)
If the versions of python and pip are not the same, pick one of the versions and use the proper python and pip combination that you wish to install pyautogui into. Since mine are the same, I just used the pip command. Next page, on the right, is an abbreviated version of the terminal output from my system when I installed pyautogui on my system (your terminal output will probably be a bit different)… Now that pyautogui is installed, you might have to install one more thing. If you try to run pyautogui right now, you might not have a package and will get an error. You need to be sure that the package scrot is installed on your Linux system. You can do this by using the still open terminal to verify it. $ sudo apt install scrot Scrot stands for SCReenshOT. If you are running a Windows system, you might have to use a different screenshot package. Pyautogui will tell you what else is needed. Assuming everything is installed correctly, we can move forward.
With the terminal still open, try the following command… $ python -m mouseinfo If everything is working, you should see a window open that looks something like this… This is a new utility that comes with pyautogui that makes automation of the mouse and keyboard really simple. It will show you the X and Y positions on the screen where you need to direct the mouse pointer to allow for text or mouse clicks. Notice the four buttons on the right (upper) of the window. The Log All (F5) and Log XY (F6) will allow you to save the mouse positions to a file, along with the Save Log button on the lower right of the window. Now, let’s write a simple program to move the mouse around the screen just to verify everything is working. Save the program as “test1.py”, but before you run the program, let’s look at a few of the lines. The first line we will examine is… sw, sh = pyautogui.size()
This will return the width and height of your screen in pixels. I have two monitors hooked to my system, so Linux combines the width of both and reports that as the width. It takes the height of the primary monitor. Be sure to take these values into consideration when you use the moveTo functions. Speaking of the moveTo function, the next line we need to look at is the first moveTo line. pyautogui.moveTo(10, 10, duration=0.50) The parameters are (Xposition, Yposition, Duration). Be sure you use proper values for your system. Remember that Xposition is the horizontal or right/left position on the screen and the Yposition is the vertical or top/bottom position on your screen. The Duration setting is how many milliseconds the movement will take. It’s completely optional, but by setting the value at 0.50, we can actually watch the mouse cursor move on the screen. In your future use, you might want to not include the duration parameter, to speed things up.
We will move to near the upper left of the monitor, then the upper right of the monitor, then to near the lower right of the monitor then to somewhere around the center of the monitor. Next, the mouse will move to position 139, 250 and perform a mouse click in the text editor to make sure it’s selected as the target for the text we will eventually send. pyautogui.click(139, 250) You will want to open a text editor and make sure the entry area of the editor will be available for this, since we are about to enter some text into the text editor, using the write function. pyautogui.write('Hello from Python!!!') As you might guess, being the observant reader that you are, the write function emulates “standard” keyboard keypresses, and writes whatever is set as the parameter to the function to the current mouse position. The final thing that gets done in our test program is that we show a message box from pyautogui saying the program has finished and will then wait for you to manually click the OK button. pyautogui.alert(text='Finished!', title='PYAUTOGUI TEST', button='OK')
Hopefully, you have the mouseinfo program still running. You can move the mouse to the positions that are coded in the demo program, just to make sure that they fit for your system. If not, all you have to do is modify the positions in the moveTo or click functions to make sure everything works. Now, making sure that the text editor is open and in the proper place, you can simply open another terminal in the folder where you saved your python test1.py program, and run it. $python test1.py If everything worked correctly, you can watch the mouse pointer move around the screen, eventually moving into the text editor and sending the message. Now, while this isn’t the most useful example of the capabilities of pyautogui, our next project should give you a really good one.
I’ve been working on a new tutorial for the next version of PAGE for Don Rozenberg. The first project in the tutorial is to create a login form which has an entry box for Username and Password, and a button to “submit” the information. I won’t go into the code very much and the actual project can be found on the repository (mentioned at the end of this article). The program looks something like this… The reason I chose this was that I know exactly where the form will appear on the screen, every time it runs, so it will make an easy demo for you to follow. You can use the MouseInfo program to verify the positions of the Entry widgets on your system and the button by running the login.py program, then watching the XY Position as you hover over the widgets. If they are off a great deal, you can modify it in our test2 program, which is presented next page, top right.
You can see that everything in the code of the test program has already been talked about from our test1 program. I set the time.sleep() call to 10 seconds, so I could take screenshots for the article. You can set it to a value that works for you. Now you will hopefully have downloaded all the code from the repository and unpacked it into a convenient folder then open two terminals within that folder. The first terminal is to run the login.py program and the second is to run the test2.py program. So, open your first terminal and type… $python login.py In the second terminal, making sure that nothing is obstructing the login form, you want to type… $python test2.py The first thing you should see is the cursor jump to the first Entry widget of the login form and the username Greg should appear.
After the sleep time has elapsed, you should see the word “password” appear, followed by another sleep time. Finally, another sleep period should expire and the Continue button should get “pressed”, resulting in a message box from the program. Immediately after that, the message box from pyautogui should pop-up as well. Click the OK buttons on both of the message boxes, and the programs should end. There is a lot more that pyautogui can do, we’ve just scratched the surface. Please take some time to read through the documentation, and Al’s chapter 20, to take advantage of this tremendous resource. The repository for the code is at https://github.com/gregwa1953/FCM-176 I hope that you found this revisitation of pyautogui helpful, and that it encourages you to try automated testing for your future projects. Until next time, as always; stay safe, healthy, positive and creative!