Try booting into OS 9, rather than booting into OS X and selecting Classic mode. That might help your problem #2. Have no idea why your resolution is stuck; I think it should be 640 x 480 for RealMyst, although I'm not sure. (Have you checked your RealMyst user manual?)
Fear leads to anger; anger leads to hate; hate leads to SUFFERING! - Yoda
I don't remember having to boot into OS 9 or start Classic for RealMyst. As far as I know it ran under OS X. Riven I had to boot into OS 9 or run in Classic.
As for your resolution, is your monitor capable of displaying 640x480? If it's not than the software won't be able to adjust it. When you say "I set the resolution at start of game", are you setting it through your display panel or using RealMyst's config?
I don't have any suggestions for your movement lag issue though. I've never heard of that one. I wonder if it's something to do with either being in classic mode or using a dual processor? I have a single 1.25Ghz G4 and didn't experience anything like that.