STORY WRITTEN FOR CBS NEWS & USED WITH PERMISSION
Two software errors detected after launch of a Boeing Starliner crew ship during an unpiloted test flight last December, one of which prevented a planned docking with the International Space Station, could have led to the destruction of the spacecraft had they not been caught and corrected in time, NASA said Friday.
A joint NASA-Boeing investigation team “found the two critical software defects were not detected ahead of flight despite multiple safeguards,” according to an agency statement. “Ground intervention prevented loss of vehicle in both cases.”
The Boeing CST-100 Starliner was launched from Cape Canaveral atop a United Launch Alliance Atlas 5 rocket on Dec. 20. The goal of the flight was to put the commercial crew ship through its paces, from launch through rendezvous and docking with the space station to re-entry and splashdown, to clear the capsule for a piloted test flight.
But when the spacecraft separated from the Atlas 5’s Centaur second stage, it immediately deviated from the expected flight plan, unexpectedly firing thrusters and re-orienting itself, wasting propellant needed to finish the climb to orbit.
After struggling with communications glitches, engineers finally managed to regain control and put the spacecraft in a safe orbit. But by then, too much propellant had been wasted to press ahead with a planned rendezvous with the International Space Station.
Instead, flight controllers focused on carrying out as many other tests as possible before bringing the ship down for landing in New Mexico.
But the timing problem wasn’t the only bug lurking in the Starliner’s computer code. Before re-entry, engineers discovered another software error that affected the thruster firings needed to safely jettison the Starliner’s service module.
Telemetry showed the timing issue cropped up before launch when the Starliner’s flight computer incorrectly read the time from a clock in the Atlas 5’s flight control system. The Atlas 5 was blameless, but the Starliner’s computer thought it was 11 hours earlier than it actually was. Once away from its booster, the Starliner, thinking it was in the wrong place at the wrong time, began firing its thrusters to correct a non-existent problem.
The service module software error “incorrectly translated” the jettison thruster firing sequence. Both errors could have led to “loss of vehicle.”
While the timing problem was widely known during the Starliner test flight, the service module issue was not revealed in any detail until a meeting of the NASA Aerospace Safety and Advisory Panel Thursday, setting off widespread social media calls for more information and “transparency” from NASA’s Commercial Crew Program.
NASA’s statement Friday was to be followed by a media teleconference later in the day with top space agency and Boeing managers.
“Breakdowns in the design and code phase inserted the original defects,” NASA said in the on-line statement. “Additionally, breakdowns in the test and verification phase failed to identify the defects preflight despite their detectability.
“While both errors could have led to risk of spacecraft loss, the actions of the NASA-Boeing team were able to correct the issues and return the Starliner spacecraft safely to Earth.”
Engineers are still investigating what caused the communications glitches that initially prevented flight controllers from quickly correcting the timing issue.
Software defects, particularly in complex spacecraft code, are not unexpected,” NASA continued. “However, there were numerous instances where the Boeing software quality processes either should have or could have uncovered the defects.
“Due to these breakdowns found in design, code and test of the software, they will require systemic corrective actions. The team has already identified a robust set of 11 top-priority corrective actions. More will be identified after the team completes its additional work.”
Since the space shuttle’s retirement in 2011, NASA has been forced to buy seats aboard Russian Soyuz spacecraft to ferry U.S. and partner astronauts to and from the International Space Station.
To end sole reliance on Russia for transportation to and from the space station, NASA announced in 2014 that Boeing and SpaceX would share $6.8 billion to develop independent space taxis, the first new U.S. crewed spacecraft since the 1970s.
Under a $2.6 billion contract, SpaceX is building a crewed version of its Dragon cargo ship that will ride into orbit atop the company’s Falcon 9 rocket. Boeing’s Starliner is being developed under a $4.2 billion contract.
SpaceX carried out a successful unpiloted flight to the space station in March 2019, but suffered a major setback the following April when that same Crew Dragon capsule was destroyed during a ground test. The California rocket builder has recovered from that incident and carried out a successful in-flight abort test in January.
It is widely expected that SpaceX will be ready to launch a Crew Dragon carrying two NASA astronauts — Douglas Hurley and Robert Behnken — sometime this spring.
Boeing’s unpiloted test flight in December was only partially successful because of the two software errors and the communications glitch. It’s not yet clear whether NASA will order a second unpiloted test flight or whether Boeing will be cleared to press ahead for a piloted mission after corrective actions are implemented.
“It’s still too early for us to definitively share the root causes and full set of corrective actions needed for the Starliner system,” NASA said. “We do expect to have those results at the end of February, as was our initial plan.
“Most critically, we want to assure that these necessary steps are completely understood prior to determining the plan for future flights. Separate from the anomaly investigation, NASA also is still reviewing the data collected during the flight test to help determine that future plan. NASA expects a decision on this review to be complete in the next several weeks.”
