I hope this will help those considering giving up their U.S. Citizenship or U.S. Green card or for those whom are already a U.S. Covered Expatriate. There's a ton of misinformation out there even from professional CPA's.
U.S. Citizens whom renounce their citizenship or a Long Term Resident whom give up their green card can become a U.S. Expatriate if other criteria are met, one of the more common being having worldwide assets that exceed $2 million and another is not meeting U.S. tax laws in the last 5 years before expatriation. Long Term Resident is a person whom had been a lawful permanent resident for at least 8 out of the last 15 years.
As a U.S. Covered Expatriate, eligible deferred compensation (such as the 401(k)) plan becomes subject to 30% U.S. tax withholding and you waive the rights to reduce the withholding rate by tax treaty.
However, you actually can claim other tax treaty including filing a Form 1040 NR and claiming a full refund for the 30% withheld by filing Form 8833 to disclose your treaty position. In the case of U.S. Japan tax treaty, article 17 paragraph 1 and 2 will apply making the 401(k) taxable only in Japan if you are a resident of Japan even if you are a Covered Expatriate. After expatriating, you are neither a U.S. Citizen nor U.S. Green card holder so U.S. is no longer allowed to tax the 401(k) distribution if you're a resident of Japan.
Some people point to article 1 paragraph 4b, savings clause for U.S. Expatriate:
"Notwithstanding the other provisions of this Convention, a former citizen or long-term resident of the United States may, for the period of ten years following the loss of such status, be taxed in accordance with the laws of the United States, if the loss of such status had as one of its principal purposes the avoidance of tax (as defined under the laws of the United States)"
This according to technical explanation from the U.S. Treasury is about Section 877 of the Internal revenue code. Section 877 indeed is referring to U.S. Covered Expatriate but applies only to those whom expatriated before June 2008. Since more than 10 years have passed, effectively article 1 paragraph 4b no longer applies to anyone. It's obsolete.
Those expatriating after June 2008 is covered under Section 877A. In Section 877A, there are no mention of principal reasons of "avoidance of tax" or anything similar to it. There's also no 10 year time limit in Section 877A so you are subject to 30% tax withholding until the entire account is withdrawn. Therefore, paragraph 1b in article 4 does not apply to those whom have expatriated after June 2008.
There's other drawbacks when becoming a covered expatriate. One is the potential of an exit tax. All of your appreciated assets outside of your eligible deferred compensation are deemed to be sold the day before expatriating. There's a fair amount of exemption for the deemed sold capital gain so it may or may not trigger an immediate tax. Other accounts such as IRA and HSA are also deemed sold as specific account the day before expatriation. But as noted, I would not be concerned about the 30% tax withholding on 401(k) plan because it is money you can get refunded when you file your 1040NR every year until you withdrawn all of your eligible deferred compensation accounts.