Waves can pick out even the tiniest weaknesses in rocks. For example, waves can enlarge hairline cracks until a block falls away.
All rocks have weaknesses like this and the sea carries away the weakened rock, forming caves.
In a headland a cave can form on both sides. When the backs of two caves meet you get an arch. When the roof of the arch falls in, a pillar called a stack is left.