Since ancient times people have known that some rocks were magnetic. These rocks are called lodestones.
If a thin sliver of such a rock is hung from a thread, it will always turn to point the same way.
A magnet only moves if it's close to another magnet, so where is the other magnet? It is the Earth itself. The core of the Earth is made from iron and this makes the Earth into a giant magnet.