Does ‘Buy, Borrow, and Die’ Work for Physicians?

By Dr. Jim Dahle, WCI Founder

Every now and then, you will see an article in the financial media complaining that very wealthy people are using the “Buy, Borrow, and Die” strategy to get out of paying their fair share of taxes. High earners like physicians sometimes wonder if this strategy would work well for them. The truth is that it can work for physicians but probably only among the wealthiest and least risk-averse among them. Once they understand how it works, most physicians will likely choose to simply sell taxable assets instead of borrowing against them.

How Does Buy, Borrow, and Die Work?

Imagine you have some assets in a taxable account—perhaps mutual fund shares—that you bought a long time ago. You now need some additional money to spend. You have two choices. The first is to sell those shares, pay any taxes due, and spend what is left. The second option is to borrow against those shares, avoid paying any taxes, and pay interest on the loan (until the day you die) instead. While the second strategy definitely has a lower tax bill, that doesn’t necessarily mean the investor ends up with additional money to spend or leave behind later.

More information here:

Buy, Borrow, Die Strategy Examples

The Upsides of Buy, Borrow, and Die

There are three main upsides of this strategy.

1. You don’t have to pay any capital gains or depreciation recapture taxes.
2. Your heirs will get a step up in basis at death, so potentially nobody will ever pay capital gains or depreciation recapture taxes.
3. You still own the asset and receive any income it produces or future appreciation that occurs with it.

Sounds appealing, right? Well, there is no free lunch. Check out the substantial downsides.

The Downsides of Buy, Borrow, and Die

The problems with the strategy are:

1. You have to pay interest to access your own money.
2. You don’t know how long you will live (and you may be paying interest for decades on money spent years ago).
3. You wouldn’t owe any taxes on basis anyway, but you pay interest on everything.
4. You are increasing your leverage risk, and you may eventually be taking on a lot more leverage risk than you would prefer.
5. You may have to pay fees in addition to interest to get the loan you want.
6. Margin calls can happen, forcing you to pay interest AND pay those taxes anyway if your shares are sold by the broker to meet a margin call.
7. You will need to have a relatively high-risk portfolio. It probably doesn’t make any sense to hold low-risk assets like bonds and CDs AND leverage up the portfolio by borrowing against it, as the interest rate is typically higher than bond yields.
8. Interest rates change, and the benefits of the strategy are very dependent on interest rates.

Buy, Borrow, and Die Means Leveraged Investing

The truth is that Buy, Borrow, and Die is all about leverage, or investing using borrowed money. Some people are more comfortable with this than others. If you’re the type of person who pays off their 3% mortgage, this probably isn’t for you. If you’re the type of person who regularly does cash-out refinances on their rental properties in order to buy more rental properties and you already have a margin account or a HELOC, you should pay attention to this technique.

Any time you are borrowing money to invest, the devil is in the details. The details generally boil down to three things:

1. The amount of debt used
2. The terms of the debt
3. The interest rate of the debt

Investing using borrowed money has been discussed extensively elsewhere, but suffice it to say that if you’re taking on a huge amount of debt at a high interest rate on poor terms, Buy, Borrow, and Die is a bad idea. On the other hand, a small amount of debt with great terms and a low interest rate can make it work out very well. If you get serious about Buy, Borrow, and Die, make sure you read, digest, understand, and reread our How to Think About Debt article. At a minimum, try to keep any debt used to less than 25% of your assets, particularly if using margin loans, so that margin calls don’t start kicking in until the value of your assets drops at least 50%.

More information here:

How to Leverage Debt. Best Ways to Use Debt to Your Advantage

How Does This Work with Mutual Funds?

Imagine you own $3 million worth of highly appreciated stock mutual funds in a taxable account at Interactive Brokers, and you want to spend $500,000. The interest on a $500,000 margin loan at Interactive Brokers as I write this post is 5.33%. You borrow against $500,000 at 5.33%. You’ll owe 5.33%*$500,000 = $26,650 in interest per year until you pay the loan back or die. Presumably, you don’t wish to ever pay back the loan, so over 30 years, the interest on that $500,000 could add up to as much as $800,000, which seems less than ideal.

However, you have two other factors working for you.

First, the interest may be tax-deductible. You have to itemize, of course, and have $30,000 (MFJ) worth of other deductions for it to be fully deductible, AND only a portion of the interest equal to your investment income is actually deductible. But if you had a 45% marginal tax rate, your actual interest cost could be as low as 2.93%.

Second, that investment will presumably continue to generate dividends and appreciation. Let’s say its total after-tax return is 8%. After 30 years, that $500,000 investment is now worth $5 million. Now that $800,000 (or perhaps as little as $440,000) doesn’t seem so large, although, to be fair, you should also apply compound interest to the interest paid. That would turn that $800,000 into a little over $3 million. Still, you’re theoretically coming out pretty far ahead—a couple of million dollars on that $500,000 loan.

You’re probably not going to have a margin call issue when you have only borrowed $500,000 against $3 million. But what about next year? And the year after that? And the year after that? What if you’ve borrowed $1.5 million against that now $3.5 million portfolio? And then there’s a big bear market, and the value of your investments drops 50% (to $1.75 million)? Now you’re getting margin calls. So, there is a limit to how much you can do this that is based on the size of that taxable account. And it’s not like changing strategies to sell the assets helps a lot with this margin call issue since that also increases the ratio of loan to total assets.

Maybe this wouldn’t be such a big deal if you were borrowing $50,000 a year against a $3 million portfolio instead of $500,000. Even after 10 years, your loan is only up to $500,000, and even if the portfolio has been paying the interest, the portfolio has likely grown by at least $500,000. The larger your taxable account and the smaller your loan, the better this strategy becomes.

However, by borrowing against your portfolio, the amount of leverage risk you are taking on increases. Instead of decreasing investment risk as you go through retirement like most people do, you’re actually increasing your risk, all in the name of trying to save some tax dollars you can easily afford to pay. One nice thing about margin loans is that there generally aren’t any fees associated with them, unlike the other two options we’ll discuss below.

Note that margin loans at Vanguard, Fidelity, and Schwab are significantly more expensive than at Interactive Brokers. As I write this, advertised rates at Vanguard range from 10.75%-12.75% and at Fidelity range from 8.25%-12.575%. Needless to say, borrowing at 10%-12% doesn’t usually work out well given typical long-term after-tax returns.

How Does This Work with Real Estate?

Imagine your portfolio is mostly composed of paid-off rental properties. They provide you some income, but you want to spend even more money than the income provided. You can sell a rental property worth $500,000, or you can do a cash-out refinance of $500,00o from a separate $800,000 rental property. If you sold the rental property, there may be $75,000 in capital gains taxes to pay and perhaps another $50,000 in depreciation recapture taxes. The new mortgage will be 7% on a $500,000 loan ($35,000 per year) plus fees of $10,000.

It won’t take very many years for that $35,000 in interest to add up to more than the $125,000 tax bill, but the rental property may still be providing some positive cash flow ($10,000 per year?) and hopefully continues to appreciate (3%, so maybe $15,000 per year?). Plus, that new mortgage is still being paid down, adding some value each year.

But what if you want to do this again? You’re not going to take $500,000 out of this same property next year. You’ll need another property. And if you’re only taking out $50,000 at a time, do you really want to pay for an appraisal and other closing costs for that refinance each year? Maybe you look into a Home Equity Line of Credit (HELOC), but the interest rate on that may be 1% higher and adjustable.

How Does This Work with Whole Life Insurance?

Those who sell whole life insurance as a retirement investment or buffer asset or whatever are actually counting on you wanting to Buy, Borrow, and Die. Whole life insurance policies can be surrendered, but then one must pay taxes on any gains at ordinary income tax rates. Partial surrenders up to basis can come out tax-free, but after that, it’s either pay taxes or pay interest.

Ideally, if you are going to use whole life insurance cash value for retirement spending, you have a policy that offers both non-direct recognition loans AND wash sales. The first means the policy continues to pay dividends based on the value before you borrowed the money instead of after. The second means the loan interest rate is equal to the dividend interest rate. If your policy is designed this way—and this is a big if, since most are not—then you can do the classic Buy, Borrow, and Die approach.

Let’s say you have a whole life policy with $1 million in cash value, and you want to spend $500,000 of it. You borrow $500,000 against it, but the interest rate is 6% and the non-direct recognition dividend rate is 6%. Essentially, the $500,000 of cash value remains static until you die. When you die, your heirs get $500,000 (instead of $1 million, because you already spent $500,000).

This is a much worse deal on a typical whole life policy (or worse, most universal life policies) as the cost of insurance and/or interest costs start eating up that cash value, and it can even eventually cause the policy to collapse.

More information here:

Should You Borrow Money from Your Whole Life Insurance Policy?

Who Should Consider It? 

Buy, Borrow, and Die is generally NOT a great way to spend money throughout your retirement. However, it can work very well near the end of retirement. Early in retirement, you likely have some assets that have relatively high basis. By selling those high-basis shares first, you can spend a lot of money while paying very little in taxes. Later in retirement, most of your taxable assets will have relatively low basis, and the tax bill to spend them will be higher. Also, if you borrow against your assets at age 60 but don’t die until 85, that’s 25 years’ worth of interest you will be paying. You may have been better off paying some capital gains taxes instead. The closer you get to death, the sooner it will be when the estate gets that step up in basis at death. So, borrowing instead of selling makes a lot of sense in the last year or two of life but not nearly as much a decade or more out. You’re really banking on that investment keeping up with the cost of the interest as you go along.

The following criteria make it more likely that this is a good option for you:

1. You actually want more money to spend above and beyond sources you’re already paying taxes on, like Social Security, pensions, SPIAs, dividends, interest, and real estate cash flow.
2. You’re in a high tax bracket, both for ordinary income and long-term capital gain tax rates.
3. You can get very good terms and a low interest rate on the loan.
4. You have a lot of money compared to what you are going to spend/borrow. A total debt-to-asset ratio of 5% is probably fine; 50% is surely not. But if you have $30 million and spend $300,000 a year, you’re not going to get into trouble.
5. You are older, meaning the interest won’t add up to much compared to the cost of taxes, and the risk of the investment dramatically underperforming the interest rate is minimized. This is a particularly good strategy for a wealthy person in the last year or two of life.
6. You invest very aggressively, meaning you don’t have a lot of money sitting around in cash, bonds, CDs, or other investments with low expected returns.
7. You (or your money manager) is comfortable with the additional complexity of managing both the investments and the debt.
8. The basis on your mutual funds is very low, such that the tax cost of selling would be very high.
9. Your properties are fully depreciated, such that a great deal of depreciation recapture tax would be applicable if you sold.
10. Your whole life insurance policy features non-direct recognition and wash loans.
11. Your assets are not in an irrevocable trust, meaning they will still get a step up in basis at death.

If you meet most of those criteria, Buy, Borrow, and Die may work out very well for you. Still, this is a very small percentage of white coat investors we’re talking about.

What do you think? Do you plan to buy, borrow, and die? Why or why not?