Between 2004 and 2012, student loan balances nearly tripled, according to a new survey from the Federal Reserve Bank of New York.
What's more, one-third of student loan borrowers are delinquent on their debt, according to the Federal Reserve report. This will impact their credit rating and possibly keep them out of the mortgage market much longer.
"Short term, you see a decrease in the number of first-time homebuyers," Brian Coester of Coester Valuation Management told CNBC. "You're going to see somebody who would have been able to afford a more expensive house maybe go for the lower version or the downgraded version."
Potential buyers with heavy student debt burden have been forced to rent or even move back in with their parents as they chip away at their debt.
"Long term, it's going to really affect especially the upper end, because people aren't going to have the excess income to buy the jumbo property or buy that high end property," says Coester. "It' s going to affect home prices as a negative, as more of a cap, because it's really debt that they are servicing."