Four possible ways that a cap can cause a headache:
This is what is called an external compression headache. Perhaps that cap is not quite as loose as you think it is. Compression is by far the most common reason for cap headaches.
Some medical conditions make you more sensitive to an external compression headache, or make them far worse. For example, if you are already prone to migrane headaches, you need to be much more cautious about caps and external compression headaches. If the headache persists even after you remove the cap, then go see a doctor.
This is speculative, but does this occur on a really hot day when a cap can interfere with the convective and evaporative cooling of your head?
You are wearing a “Make America Great Again” trucker cap and people are dope-slapping you.