No. Launching as you are is the correct way that is:
Any other way simply will not work because that isn't how the program is built.
Ultimately, this is exactly why we standardise an install through Conda. Sure, it isn't for everyone, and has some major issues, but it standardises the install making these kinds of problems far easier to solve.
As it is, I have no idea about your individual Python setup, what's in your base environment, what site-packages you have, and which are available in the virtual environment, how your Cuda is installed, etc. etc. etc.
So, unfortunately, we have to take the approach that you are entirely entitled (and indeed capable) to install Faceswap whichever way you please, but at that point you are, unfortunately, on your own.
You are welcome to open a thread re: manual installs to see if anyone can help, but that's the best I can offer. As it is, this thread is for installing on Linux through the installer, so isn't the best place for this dicussion.