Only DDS will decrease loading time. Once it's in memory it doesn't matter. However, if you have a large project loading time becomes important. Loading a .bmp, .jpg, or .png can take forever, I've seen as long as 4 seconds, while a .dds of the same resolution loads in a few milliseconds.
Quote: "it might but the jpgs will make it look horrible"
No it won't, unless you use a large compression ratio. JPGs get good compression at lower ratios so there's not much benefit to hurting quality for that reason.
Come see the WIP!