Officials at Public Health England said the amount of activity people did started to tail off from the age of 40.
They are urging those between the ages of 40 and 60 to start doing regular brisk walks.
They say just 10 minutes a day could have a major impact, reducing the risk of early death by 15%.
But PHE estimates four out of every 10 40- to 60-year-olds take a brisk 10-minute walk less frequently than once a month.
To help, the government agency is promoting a free app - Active 10 - which can monitor the amount of brisk walking an individual does and provide tips on how to incorporate more into the daily routine.
PHE deputy medical director Dr Jenny Harries said: "I know first hand that juggling priorities of everyday life often means exercise takes a back seat.
"But walking to the shops instead of driving, or going for a brisk 10-minute walk on your lunch break each day, can add many healthy years to your life."