How to control the perspective of a unity game using python?

Question

I want to simulate mouse movement through Python, so that the perspective of the Unity game rotates accordingly.

I used pydirectinput.moveTo() to move mouse. Used pydirectinput.keyDown() and pydirectinput.keyUp() to input key.

It can work in 《Borderlands 2》. I can move forward and backward and turn the camera.

But it can't work in 《Aim Hero》, which is a unity game. I can also move forward and backward. The characters in the game move with my control. But the character's perspective doesn't move and shoot.

I execute the command to move the mouse, and the mouse actually does move. However, the mouse will move out of the game's window, and the in-game perspective does not move with it.

Initial suspicion is the difference between Unity games and DirectX games.

This is my code:

import pydirectinput
import pyautogui
import time
def func1():
 # ------------this can work in borderlands 2 and Aim Hero(unity)
 time.sleep(4)
 pydirectinput.keyDown('w')
 time.sleep(1)
 pydirectinput.keyUp('w')
 time.sleep(1)
 pydirectinput.keyDown('d')
 time.sleep(0.5)
 pydirectinput.keyUp('d')
 time.sleep(1)
 # ------------
 
 # ------------this all can't work in Aim Hero(unity)
 pos = pyautogui.position()
 pydirectinput.moveTo(pos.x + 100, pos.y) # can't work in borderlands 2
 # pos = pyautogui.position()
 pydirectinput.moveTo(pos.x + 200, pos.y) # can work in borderlands 2
 time.sleep(0.5)
 pydirectinput.click()
 # -------------
def func2():
 time.sleep(4)
 # in borderlands 2:
 # If the command to move the mouse is executed once, the in-game camera does not move.
 # If the command to move the mouse is executed n times, the in-game camera will move n-1 times
 # in Aim Hero(Unity Game):
 # The mouse keeps moving and eventually moves out of the game window.
 # But the in-game perspective has never changed.
 for i in range(2):
 pos = pyautogui.position()
 pydirectinput.moveTo(pos.x + 10, pos.y)
 # pydirectinput.click()
 print(pos, '\n')
 time.sleep(0.1)
if __name__ == '__main__':
 print("Start!\n")
 func1()
 # func2()
 print("Done.")

• Peter: Could be worth trying to click before moving to make sure the window has focus

Answer 1

Can you try

# ------------this all can't work in Aim Hero(unity)
 pos = pyautogui.position()
 pydirectinput.moveTo(900, 600) # estimated center position of screen
 time.sleep(1.5) #'cause this code working slow
 pydirectinput.click()
 pydirectinput.click()
 pydirectinput.click()
 # -------------

instead of

# ------------this all can't work in Aim Hero(unity)
 pos = pyautogui.position()
 pydirectinput.moveTo(pos.x + 100, pos.y) # can't work in borderlands 2
 # pos = pyautogui.position()
 pydirectinput.moveTo(pos.x + 200, pos.y) # can work in borderlands 2
 time.sleep(0.5)
 pydirectinput.click()
 # -------------

EDIT

Okay I found the problem! It's not about Python, it's about Unity. Let me show you a video from my computer. As far as I understand, if you want to use the pydirectinput.moveTo() method in Unity game, you should not lock your cursor. Be careful about your cursor state and use CursorLockMode.None instead of CursorLockMode.Locked. But it wont be effective for real game :( And I don't know any alternative for now

• Aehcaoiggc: Thanks a lot for your answering. Obviously, that doesn't work. When I restarted my computer and re-run my program, I found that if I run the second function in AimHero, the mouse position output remains the same and is the position of the center of the window relative to the computer monitor.
• Aehcaoiggc: Also, I tried moving the mouse myself and returning my mouse position via pyautogui.position(). The result is that no matter how my mouse moves, the returned coordinate values are always the same. It may be because the coordinates returned by pyautogui.position() are always the position coordinates of the mouse relative to the computer monitor, which will not change when the game window is not moved.
• Emirhan Soylu: I'm sorry but I don't understand what you mean :( Because both functions are working very well for me. It doesn't matter in Unity game or desktop.
• Aehcaoiggc: I'm so sorry for replying you so late. I have uploaded the video I recorded to Google Drive, hope you can see it. This is my first time using stackoverflow and I don't know how to provide my video to the responders. If you can't see it, I can upload it to youtube. The link to the video is link
• Aehcaoiggc: Another thing to say is that it is normal for the video to not be able to shoot when func1 is running, because the game has not started, but it is not normal for the camera to not turn.