Depends what kind of speed you refer to. There's different types.
I still can't hold the XC B grade bunches on long flats, and tend to fall back over long climbs, but can outsprint virtually everyone I race against off the start and up short hills, but can't put in high power finish sprints. On any track which involves lots of starting and stopping, or short sprinting between trees I've just unconsciously put in a lot of training (ie, STXC).
I saw the biggest improvement in straight-line in-the-saddle power after riding single-speed for a few weeks/months. OTOH I'm now not as quick up hills since I changed my riding style to be a lot more in-the-saddle, but can hold that pace longer. All that's just about trying to develop more power through my glutes - which at my current point I don't need to get out of the saddle anymore accelerating out of corners in C grade road bunches whereas others need that extra power for those accelerations.
You'll notice where you need to improve when you try to sit on fast bunches or crits - eg, few weeks ago I rode a crit in Elite Men (way out of my league), and could really feel it in my glutes. Guess what area needs improvement for that kind of riding?
Also remember that if your position on your road and XC bikes aren't matched, then you're using different muscle groups. Effectively, if they're not the same, you end up needing to re-train yourself for each bike, and if they're too far different effort on one won't help (much) effort on the other.
My point is here, there's different kinds of speed. Pick one.