I was told by my auto body specialist that those swirls occur due to dust or dirt being on the car. when you wipe the dust off they can create the swirl in your clear coat or even your paint. To avoid this be sure to thoroughly apply water and soap to your car when cleaning it. Do not drag a dry cloth across your car to knock dust or dirt off, and wax your car regularly.
If it is in the clearcoat, wax should help get rid of most of it, but if its to the primer, im affraid you will have to do what I did and take it to the shop for professional work.
I recommend turtle wax ice. It goes on easily, spreads evenly, and due to its liquid state, fills in the scratches and specs verry well on older cars with blemished paint.