The American subsidiary of a Japanese company has agreed to pay a $6.3-million criminal fine for illegally shipping equipment to Iran and trying to shield the deal from regulators.