Just to clear things up in case there’s someone out there who’s having the same trouble:
I ended up using the template Rex Rainbow is suggesting, which is this template made by Yannick.
I was having a hard time grasping the concept of webpack, but in the end, after you copy-pasted the entire folders of your game into ‘src’ folder, just type npm run build into the terminal.
Following that, /dist folder will be compiled by the webpack, and the only folder you want to publish to your GitHub Page is that /dist folder only.
And that’s it! Your game is now online.
If a 404 page shows up, try disabling cache in your browser, that might help.
Huge thanks to superTommy, RexRaimbow, and Yannick who are willing to help me. 