A study of 1,000 people found that staying active nearly halved the odds of catching cold viruses and, failing that, made the infection less severe.
Experts told the British Journal of Sports Medicine that this could be because exercise helps bolster the immune system to fight off bugs.
But you may not have to actually do much exercise - those who merely think they are fit enjoy the same lower risk.
Adults can expect to suffer two to five colds per year. This latest research suggests there are lifestyle choices you can make to improve your odds of either avoiding them, or suffering too badly from them.
For their study, US researchers asked the healthy volunteers to keep a record of any coughs and sniffles they experienced over a three-month period during the autumn and winter.
The volunteers were also asked to say how frequently in any given week they would do exercise lasting at least 20 minutes and intensive enough to break a sweat.
And they were questioned about lifestyle, diet and recent stressful events, as these can all affect a person's immune system.
Being older, male and married seemed to reduce the frequency of colds, as did eating plenty of fruit.
But the most significant factors that cut colds was how much exercise a person did and how fit they perceived themselves to be.
Feeling fit and being active cut the risk of having a cold by nearly 50%.
People who were physically active on five or more days of the week were unwell with a cold for about five days of the three-month period, compared to nine days for those who did little or no exercise.
And even when they were ill, they suffered less with their symptoms.
The severity of symptoms fell by 41% among those who felt the fittest and by 31% among those who were the most active.
Lead researcher Dr David Nieman and his team, from Appalachian State University in North Carolina, say bouts of exercise spark a temporary rise in immune system cells circulating around the body that can attack foreign invaders.
Although these levels fall back within a few hours, each session is likely to provide an immune boost to fight off infections like the common cold.
Professor Steve Field, chairman of the Royal College of General Practitioners, said: "This is yet more evidence for doing exercise. It reflects what we have believed for some time.
"Exercise makes us feel better and now here's more evidence that it is good for us."