Risk and Advancement: SpaceX Anomaly Investigation Makes Progress

Credit: U.S. Launch Report

Late last week, SpaceX released a new update regarding the ‘anomaly’ which destroyed a Falcon 9 rocket and its satellite payload during pre-flight testing on September 1st. It is included in its entirety below. As suggested by previous updates, as well as comments both on and off the record from company officials, the overall cause was a breach in one of three composite over-wrapped pressure vessels, or COPVs, which are located within the Falcon 9’s upper stage liquid oxygen tank.

Under normal conditions, the COPVs contain super cold high pressure helium which is used to maintain the Falcon 9’s propellant tanks at operating pressure during flight. Composite materials are used in conjunction with the spherical vessel’s thin metal liner in order to provide the strength necessary to contain pressure without imposing the extra weight of a conventinal fully metal pressure tank.

COPVs are used in other launch vehicles, including the now-retired Space Shutte, which employed 24. While considered an important advance in space technology, COPVs bring with them an element of risk. In the case of the Shuttle, the Return to Flight campaign following the loss of Columbia in 2003, charged with examining all aspects of the system, found that the failure of a COPV represented the 5th highest total risk to the orbiter.

In the upgrade from the original Falcon 9 to the V1.1, SpaceX took the somewhat unusual step of moving the COPVSs from their location near the base of the second stage tank, to actually locating them within the rocket’s liquid oxygen tank in order to de-clutter the immediate area around the Merlin 1-D engine, while taking advantage of the super cold bath provided by the LOX.

That change came under scrutiny last year when a Falcon 9 exploded during first moments of flight as a part of the NASA CRS-7 resupply mission to the International Space Station. In that case however, the cause of that accident was traced not to a failure of the bottle itself, but instead to a strut which held it in place. According to SpaceX, the strut failed well under its specified operating load, turning the COPV into projectile which shot through the LOX tank, destroying the vehicle. That analysis was backed up by testing of other struts in inventory which found a small percentage of those failed as well.

When SpaceX returned to flight following the CRS-7 failure, it was with a newly upgraded version of the Falcon booster which incorporated another change, the use of super-chilled ‘densified’ liquid oxygen which operating near the freezing point, allowed the company to load more lox into a given tank volume, providing a significant performance increase. Now, and under specific conditions which are still imperfectly understood, it appears that by lowering the termperature of the liquid oxygen in which the tanks are submerged, the company may have inadvertently created conditions which allowed solid oxygen ice to form within the COPV during helium loading. The working theory appears to be that the oxygen ice reacted with the carbon composite to create a rupture.

If confirmed, the finding is good news in the sense that it reflects a problem in operations rather than a design fault in the Falcon 9 itself, and as the update notes, is one which can be fixed or prevented through a change in loading procedures. The result will be return to flight sooner than later, something the company has suggested would be the case since shortly after the incident.

The loss of the Falcon 9 and Amos-6 satellite during the September test has predictably added fuel to the ongoing rivalry between SpaceX and its chief domestic competitor, United Launch Alliance, each of which has brought friendly members of Congress into the fray. It also raises a point which is not a negative to either side, but more of a statement on the nature of pushing the envelope in what is already a cutting edge industry.

ULA has achieved a truly remarkable record of launch success, in part due to the fact that the company has been loathe to make major changes to its Atlas and Delta rockets once they were determined to be reliable. The downside is that its costs have remained very high, with the result being a near total retreat from the commercial launch market and heavy criticism of the burden which is placed on the taxpayer to support its national security applications.

SpaceX on the other hand, following a policy of continual iteration, has brought the U.S. roaring back into the commercial market to the point of nearly completely disrupting it, while at the same time offering a plausible alternative to risk averse DOD mission planners.  Undermining the SpaceX effort is the fact that it has suffered two high profile incidents in the last two years, oddly enough with a vehicle which had appeared to outgrow its adolescence.

The challenge for both companies, as well as those who would regulate them, and the general public which cheers them, is to understand that while carelessness is never acceptable, real progress in engaging the space frontier is going to have its share of failures, particularly where new technologies, or applications for existing technologies are concerned. Both companies, as well as Blue Origin, will sooner or later be introducing new booster families based on methane or LNG technology which has the added benefit of being autogenous, or self-pressurizing. While that will decrease the likelihood of failures related to helium pressurization which has plagued SpaceX in particular, other, as yet undiscovered areas of risk will no doubt arise for all three. Accepting that fact is a necessary step in moving forward.

SpaxeX update:

The Accident Investigation Team continues to make progress in examining the anomaly on September 1 that led to the loss of a Falcon 9 and its payload at Launch Complex 40 (LC-40), Cape Canaveral Air Force Station, Florida.

Since the incident, investigators from SpaceX, the FAA, NASA, the US Air Force and industry experts have been working methodically through an extensive fault tree to investigate all plausible causes. As part of this, we have conducted tests at our facility in McGregor, Texas, attempting to replicate as closely as possible the conditions that may have led to the mishap.

The investigation team has made significant progress on the fault tree. Previously, we announced the investigation was focusing on a breach in the cryogenic helium system of the second stage liquid oxygen tank. The root cause of the breach has not yet been confirmed, but attention has continued to narrow to one of the three composite overwrapped pressure vessels (COPVs) inside the LOX tank. Through extensive testing in Texas, SpaceX has shown that it can re-create a COPV failure entirely through helium loading conditions. These conditions are mainly affected by the temperature and pressure of the helium being loaded.

SpaceX’s efforts are now focused on two areas – finding the exact root cause, and developing improved helium loading conditions that allow SpaceX to reliably load Falcon 9. With the advanced state of the investigation, we also plan to resume stage testing in Texas in the coming days, while continuing to focus on completion of the investigation. This is an important milestone on the path to returning to flight.

Pending the results of the investigation, we continue to work towards returning to flight before the end of the year. Our launch sites at Kennedy Space Center, Florida, and Vandenberg Air Force Base, California, remain on track to be operational in this timeframe.