The Abbey owner says restaurant business has been a slow return