The Chartered Institute of Housing (CIH)/Ipsos MORI poll questioned adults aged 16-75 online across Great Britain.
Some 20% agree they might have to leave their local area in the future because the cost of housing is too high.
Young people are particularly pessimistic, with more than a third (36%) of 16-24-year-olds and a quarter (26%) of 25-34-year-olds agreeing that they might have to move.
Half (52%) of 16-24-year-olds say that rising house prices would be a very or fairly bad thing for them personally, compared with 32% across Great Britain as a whole.
The survey also showed that 60% of Britons believe it is harder for them to buy or rent a home than it was for their parent’s generation when they were the same age.
And 75% believe it will be harder for the children of today to buy or rent a home than it is for them.
A third (32%) say their housing costs are causing them a great deal or fair amount of stress.
A quarter (24%) of people – and a third (32%) of 16-24-year-olds – are concerned about being able to pay the rent or mortgage right now, while 29% of Britons are concerned about being able to meet their monthly payments in 12 months’ time.
CIH released the figures ahead of Housing 2014, its annual conference and exhibition, which takes place at Manchester Central from 24-26 June.
Chief executive Grainia Long said: “These figures are further confirmation that our housing crisis is causing misery for millions of people across Great Britain and fuelling fears for the future – particularly among younger people. We must take action now for the sake of future generations.
“Our survey showed that four in ten people (40%) would support more homes being built in their local area. We think they are absolutely right – to have any hope of tackling our housing crisis we must drastically increase the number of new homes we build across all tenures.”
She added: “With the general election approaching, our survey also showed that all parties must do much more to convince the people of Great Britain that they can address this issue.
“When asked which party has the best policies on housing, 52% say ‘none’ or ‘don’t know’, while 41% don’t think that political parties pay a lot of attention to housing. Now is the time for action – we’re challenging all parties to put tackling the housing crisis front and centre in their manifestos.”