Goldstone, We Had a Problem
Getting Light from the Sun
Called SOHO, the Solar and Heliospheric Observatory had proven to be a huge hit on the internet, as millions could view real-time images of the Sun, discover comets, or check space weather for flares, prominences, or sunspots.
With its twelve instruments, SOHO had also become a crucial part in the scientific plans to study our star. SOHO is a project of international cooperation between ESA and NASA to study the sun, from its deep core to the outer corona, and the solar wind. It was launched in December 1995 on an Atlas IIAS/Centaur rocket. Besides watching the sun, SOHO had become the most prolific discoverer of comets in astronomical history. As of May 2003, more than 620 comets have been found by SOHO.
Blackout for Predictor of Blackouts?
SOHO observations are irreplaceable for scientists who provide advance warning of space storms. Such unpredictable bursts of electromagnetic energy can cripple satellites, if their orbits or positioning are not protected. During SOHO's blackout periods, space weather forecasting will be set back 20 years, said Joe Kunches, the lead forecaster at NOAA's Space Environment Center.
In total, the real possibility was an inability to monitor or predict solar storms for about a third of each year.
The anomaly in the HGA was first discovered when engineers detected a discrepancy between the commanded and measured antenna position. In normal conditions, the antenna must be able to move along two axes, vertical and horizontal. The horizontal movement was no longer taking place properly. The last time such a problem wrinkled foreheads was 1998, when a navigation error hindered picture transmissions from the billion dollar observatory. Until SOHO could be restored to its full capabilities two months later, NASA and ESA considered a replacement spacecraft as their only viable alternative, particularly given the proven importance of monitoring the Sun continuously.
Last month as an interim measure to safeguard the spacecraft, its operations were put into a safe-mode until the pointing problem could be looked at in more detail. The science teams began safing their instruments for an off-time of 4-6 weeks. Its low gain antenna, which does not need to be pointed in a specific direction, was used in this protected way to control SOHO, monitor the spacecraft and maintain instrument health. But the pictures were lost.
The higher-rate transmissions from the Solar and Heliospheric Observatory (SOHO) were initially interrupted, since these large data sets depended on the movement of the pointing antenna to stay on track. The loss of signal occurred on a 26-meter station of NASA's Deep Space Network (DSN). Until June 30, 2003, however, the spacecraft continued beaming down its science data, which were successfully picked up by larger 34-meter DSN stations (when available). On June 30, 2003, the 70-meter DSN station in Madrid, Spain, successfully received high-rate science data through SOHO's omnidirectional on-board low-gain antenna. SOHO normally uses this antenna only for low-rate telemetry in emergencies, and the antenna does not need to be repointed.
Staring at the Sun
European Space Agency (ESA) and NASA engineers asessed several options to recover the situation, or minimize the scientific data loss. The international team of SOHO designers continued to troubleshoot the locked HGA motor. There were several attempts in high speed mode and one attempt in low speed mode to move the antenna by 30 steps, just enough to get another encoder pulse. None was successful.
The next lead the engineers began investigating centered on one puzzling fact: whenever the antenna was not used for more than 24 hours, i.e., when there were only minimal thermal gradients inside the motor, it moved by about 30 steps (for approx. 5 min in low speed mode) before sticking. Temperature problems in some previous planetary probes, like the 1991 Galileo mission to Jupiter, also led to a locked antenna and lessened communication lines back to the DSN. For the SOHO engineers, the hypothesis was that when the motor was uniformly cool enough, a single set of windings may be enough to drive the gears. It was decided to increase the temperature of the antenna slightly to about 10C.
After a spacecraft offpoint maneuver late on the morning of June 18 confirmed that the high gain antenna (HGA) had in fact not moved more than a small amount (in comparison to the steps commanded) since the first offpoint on June 4, the SOHO team decided to switch to the redundant HGA electronics.
The system team commanded an HGA motion about the spacecraft Z axis (i.e., east-west antenna motion) of roughly one degree. Less than one-fifth of that motion took place, as determined from automatic gain control readings at the ground station. The engineers commanded another 131 steps, without any noticable antenna movement.
The similarity of the behavior on both horizontal sides indicated the likelihood of a mechanical problem with the antenna drive motor or mechanisms. In response to their quandary, the team held a teleconference with the manufacturer of the motor (Moog) and, following their recommendation, tried to command the antenna for a rapid thousand-fold boost, in high speed (100 Hz instead of 0.1 Hz) in both directions. This burst ended without success.
Sun Back Online
The solution to bring the Sun back into its continuously monitored status is still ongoing, but likely will combine a clever package of navigation tricks and a greater use of ground station availability to conquer what appears to be a stuck motor gear.
The second patch depends on rolling the spacecraft itself, to mimic how the antenna used to swivel horizontally. SOHO scientists expect full high-rate telemetry coverage, even on 26-meter stations, to resume on or about July 14. To achieve this, they will make the spacecraft roll 180 around its Sun-pointing axis in a maneuver currently planned for July 8.
After a number of tests and new insights, SOHO engineers now say there will be no 'blackout' periods for SOHO science data. "We're now talking only moderate fractions per day every day during the 2-3 week periods," says Bernhard Fleck, ESA's SOHO Project Scientist.