An RRSP is definitely worth it for someone who wants to withdraw money during retirement. If you withdraw money when your income is very low, then you end up taking deductions from your income when your tax rates are high, and adding them back when your tax rates are low. You can also save on capital gains taxes while your money resides inside your RRSP.
That said, you raise some very good points about the use of RRSPs for the purposes of making down payments. I would actually not use an RRSP (beyond the $25k allowed under the Home Buyer's Plan) for anything that requires withdrawing money while your tax rates are high. Instead, it would make sense to withdraw money from TFSA or build savings in fully taxable accounts.
Hope that helps,