Well that's a weird, yet probably valid question. However, I don't believe it would take longer for a web page to load using "color safe" codes other than white.
Other things you should be aware of is things like specifying the sizes of tables and images so the browser does not have to figure it out, watching the file sizes of your images, watching how many database queries you have on one page.
Of course there are a lot of other things you can do to make your pages load quickly, but those are just some that come to mind for me.