Hi I'm Nigel. I'm the developer of AudioSwift.
About the issue with the Magic Mouse or working with two trackpads, I've released a new version 1.2.0 that has a workaround for this. Because of limitations with the macOS framework used to make AudioSwift work properly, a trackpad can't be used solely for the MIDI controller without moving the mouse pointer. That's why the mouse pointer is frozen when the console is on, to avoid accidental clicks and drags over other apps.
In the new version however, the first trackpad that is touched when the console is on will be the MIDI controller. Then, when the second device surface is touched (another trackpad or a Magic Mouse), the console will be turned off automatically so you can get control of the mouse pointer again without the need of pressing the escape key or the hotkey.
To download the new version, just click "Check for updates" at the main menu.