Robert Silverberg 
Lest We Forget Thee Earth 

A hundred thousand years ago, there had been a planet called Earth. It had been a proud world ruling a thousand vassal stars, but its stellar empire had turned upon and annihilated their conquerors, and wiped the name of Earth from the maps of space.But Earthmen still survived . . . a strange race of worldless men and women, by tradition advisers to rulers, but never themselves ruling. Wanderers through myriad planets, their origin was a half-forgotten legend.That was the situation when a strange quirk of fate sent Earthman Hallam Navarre on an interstellar wild goose chase. He had to bring back a strictly mythical treasure to his alien ruler, or die.