Where I grew up in northern Wisconsin, nearly any rock I saw, especially a rounded one, had started out hundreds of miles to the north in Canada and had been moved to where I found it by Quaternary (Pleistocene) glaciation. Many rocks found in northern Europe were also emplaced by glaciers and any rock at the bottom of a mountain or in a stream bed has traveled a long distance.
I have been contacted by numerous farmers and contactors from midwestern and northeastern states who found a big rock in their field or 10 feet deep at a building site and who conclude that it must have fallen from space. “How else could it get there?” The rocks came from above, but only because the ice under them melted. Also, when all that ice melted there were tremendous amounts of silt and water that moved from here to there, carrying and burying rocks with them. See Lake Agassiz and Lake Missoula.
In addition to glaciers and running water, people pick up and move rocks for many different reasons. In the days when the U.S. was settled, farmers would weight their plows with large rocks just so the plow would dig deeper. Eventually the rock was discarded “where it doesn’t belong.” Old building sites are a great place to find misplaced rocks.