This is going to be really tough for the housing market, expert says