Skip to content

Macro Maker

Description

The Macro project was my final project for Java Data Structures (H) in 10th grade, of which the requirements can be found here. The final product allows the user to start and end a global recording of all inputs to the computer. The global recording will inputs including mouse movements; mouse dragging; mouse clicking, releasing, and holding; key presses, releases, and hold times; and any simultaneous combination of events. The actions are also logged in the logger.out file. After the user begins recording, (CTRL + SHIFT+ R), all henceforth input actions are recorded until the user stops recording (CTRL + SHIFT + S). Upon replaying, the program will execute the actions exactly as recorded. The press() methods also delays the action's execution as to match the delays between actions of the original recording. The program does not interfere with input actions or other applications in any way.

Demonstration

This is a macro made to play the video game Bloons Tower Defense 6 by infinitely beating one of the levels to farm as much in-game currency and in-game player experience as desired. This macro uses a slightly modified version of the Macro Maker source code that allows specification of how much in-game currency or experience the user wants.

Inspiration

I've done a lot of scripting before, both static and dynamic through OCR. My goal with this project was to make a program that could record your actions and replay them with no need for hard coding or manual key press/mouse coordinate tracking. In the future, I plan to make make the project's logging function more useful, where the logging of actions into a seperate .txt document can function as a saved user macro sequence. Eventually I want to add a feature where the program is able to ingest a log file and replay the actions stored there, allowing users to save a specific macro multiple times past the closure of the JFrame.

Method

The main library that enabled the creation of this project is JNativeHook, which allows the global tracking of all actions that are sent into a computer, as opposed to only being able to detect inputs inside a specific window. In this project, JNativeHook's GlobalScreen and adapters were used, but I have overriden the action listeners and adapters with my own code to suit the needs of this project, specifically storage and replaying of inputs.

When the program is running and recording is true (the user has started recording), every input action triggers a specific handler method which is able to encapsulate the event as an object to store in the recordedActions stack. During recording, a global variable of time is used to keep track of delays between actions. Every time a new action is recorded, the variable is then updated to the time in milliseconds when the action was performed. When the next action is then recorded, subtracting the last recorded time from the current system time gives us the delay, which is stored as part of the action object. Then, the time variable is updated. Thus, we are able to track the delays between actions in order to replay them at the right times.

The different action classes (ClickPress, ClickRelease, KeyPress, KeyRelease, MouseMove) allow the program to store every input as an individual object. All these classes implement the Action interface, giving them all a press() method which is called when replaying. Here is an example of the handler method creating a new object:

   @Override
    public void nativeKeyPressed(NativeKeyEvent e) {    // This method is called when a key is pressed; other methods handle other events.
        recordedActions.add(new KeyPress(e, Math.abs(System.currentTimeMillis()-time)));    // Creates a new KeyPress object in the stack, parsing in the NativeKeyEvent generated by JNativeHook/GlobalScreen and the time since last recorded action
        time = System.currentTimeMillis();  // Updates variable time
    } 


    // For context, the constructor for KeyPress is: 
    public KeyPress(NativeKeyEvent ke, long delay) {...}


The other adapter and conversion classes (SwingKeyAdapter, MouseCoordinateConverter, KeyLocationLookup, KeyCursorLookup) allow for conversion between different keycodes and screen sizes. The SwingKeyAdapter, KeyLocationLookup, and KeyCursorLookup classes convert VC keycodes used by NativeKeyEvent into VK keycodes used by KeyEvent. The MouseCoordinateConverter class allows for conversion between physical mouse cursor coordinate positions and scaled mouse cursor coordinate positions, allowing MouseMove to move the cursor to the correct point for all screen sizes on different computers. For example, the KeyPress class uses SwingKeyAdapter to translate a NativeKeyEvent keycode integer value to a KeyEvent keycode, which is what the robot from java.awt.Robot can interpret and use. 

   @Override
    public void press() {
        robot.delay((int) delay); // Waits the millisecond amount as parsed from the constructor
        robot.keyPress(ska.getJavaKeyEvent(e).getKeyCode()); // Uses SwingKeyAdapter to convert keycode values
    }


During recording, all input actions are created as new objects of their respective class and are stored in a stack until the user stops. Upon replaying the stored actions, the program will iterate through the stack and call each object's press() method to execute the action exactly as performed. 

   for(Action i : recordedActions) {
        i.press(); /* The implementation of the Action inferface by all the input action classes 
                      allows me to call a generic .press() function */
    }


For more information, the full project can be found open-sourced on the Github repo. The downloadable jar file is not obfuscated and can be decompiled.


Download

The github repo can be found here.
The jar executable can be downloaded here.

It appears as of November 2024 that an update to the JVM has caused an issue with the JNativeHook library, likely due to increased keylogger malware protections. I will debug this issue if I find the time.


Instructions for Use

Start Recording Keystrokes: CTRL + SHIFT + R

Stop Recording Keystrokes and Save: CTRL + SHIFT + S

Replay Saved Keystroke Sequence: CTRL + SHIFT + 1



Initial Planning Chart

MacroMaker flow design chart

Key Elements

  • Action interface implemented by all the action/event classes: ClickPress, ClickRelease, KeyPress, KeyRelease, MouseMove, etc.
    • Every action class follows the command archetype, allowing the encapsulation of a request as an object.
  • KeyLocationLookup
    • Singleton - single instance of the lookup object that is referenced every time a key conversion is needed
    • Hashtable inside KeyLocationLookup to store NativeKeyEvent KeyLocations and corresponding KeyEvent integer values for conversion
  • Mouse and Keyboard Adapters to listen for actions within the global screen. Those adapters also function as observers for the GlobalScreen.
  • Stack to store the recorded actions
  • Custom MouseCoordinatesConverter component that will convert true x, y mouse coordinates to relative x, y mouse coordinates scaled to each computer's unique screen size. Also functions as an adapter.
  • State machine under GUILogger handling action transitions and entries when a key/mouse is pressed
  • TreeMap under KeyCursorLookup to store NativeKeyEvent VC values and corresponding KeyEvent VK values