If you are worried about being jumped by a crack head or someone else a bit off the planet, just go riding with mates...
If you go alone, and for some reason the guy living next to the track has bad intentions, no-one is there to back you up.
If you ride with mates, at least someone is...