Real answer: The sheer amount of neutron radiation thrown off by fusion would mechanically erode the panels. This is why the Lockheed Martin fusion reactor they claimed to have built is complete BS - their design ignored the requirement to shield their superconductors from the neutron radiation, allowing them to be placed far closer to the reaction (and thus vastly lower the power requirements). While it could have theoretically worked briefly, it would have eaten itself into radioactive dust astoundingly quickly.