Yea I actually remembered and figured it out before coming back here, but I'm not sure which version I used to sideload but the magisk on my phone now says 24.3.Magisk zip. This method is technically depricated and I've seen some say they had problems with the latest versions. So maybe use 24.1 or 24.2. You can update it once installed.
Get it from GitHub and rename from .apk to .zip.
We've discussed this method and have steps on the TWRP thread:
How would I make a NANDroid backup? the TWRP fails when I try to backup