The Science of Duo in Space, Pt. 5: Up, Up & Away
Previously: Duo in Space is in the final stages of preparation before the big show: launching a payload to the edge of space in hopes of performing the first Push authentication from high into Earth's atmosphere, streaming live during DEF CON 24. Armed with new insights from the test launch flight data and on-board camera footage, Duo in Space architect Mikhail Davidov returned to his lab to make one more round of refinements to the payload and telemetry stack before the real-deal launch.
The Home Stretch
Scotland and I broke down the payload into what seemed like a thousand pieces, packed our gear bags and headed to Las Vegas.
Out in Vegas, the Duo team set up shop in the kitchen of our Director of Research, Steve Manzuik. We unpacked the pieces on the table and got to work putting together the payload, testing the various subsystems and gearing up the chase vehicle.
It's ready! #DuoInSpace pic.twitter.com/vnQaQymqNS
— Duo Labs (@duo_labs) August 6, 2016
After 10 hours of work, the payload was done! We repacked everything in the chase truck and headed back to our hotels. We’d only get about four hours of sleep before we had to set off. I plugged in the rechargeable batteries top off over night (and hoped a fire wouldn’t start in the middle of the night) and tried to get some sleep.
Launch Day
At 4 a.m. everyone managed to peel themselves out of bed. We rigged the vehicles with cameras and started the live stream as we drove out to the launch site. Once we arrived, thanks to knowing what to expect the launch prep progressed smoothly. We laid out a ground tarp and helium tank and set up the rigging. While Scotland worked on payload closure and launch prep, I got down to filling the balloon.
To reach our target ascent rate, we had to fill the balloon precisely. To help with this, I used a calibrated dummy weight. When the weight started to just lift off of the ground, we knew we had our desired neck lift of 14 pounds. The balloon had swelled and taken on an oblong shape. We were definitely pushing it to its limits, but it held.
Like the test launch, I enabled the trackers and verified they were working. The cameras were sped, and I made sure the smartphone screen was actually on. We tied the balloon to the payload train and slowly raised it into the air, arming the cut-down timer and and buzzer along the way.
We wished the payload well and let it go. Almost immediately, things went wrong. The payload started spinning, accelerating faster and faster. Within seconds, it was rotating at more than 70RPM. My heart sank. The payload was going to tear itself apart.
What I hadn’t noticed while checking all the electronics was that the fake solar panel had been mounted at an off angle. Instead of helping steady the payload as intended, it turned into a prop and wound up the payload like a top. Eventually, the force of the air pushing on the panel reached the point that it broke its internal connections and got blown flat, greatly reducing the spin rate. Now, the payload was flying in a small circles around its center line.
With thoughts of losing the payload fresh on my mind, I ran to my chase vehicle, grabbed my laptop and looked at the radio gear. Were the trackers still working? An APRS ping came in: 4,000 feet. The chase, though off to a rough start, was on!
Pushing On
Everyone packed up the launch site as quickly as possible, and at T+26 we hit the road. We were late. It was already past the time for the first push window. I connected to the ground station and ran the script. The moment of truth came: I called the satellite phone on speaker. As I waited through the first ring, every potential failure raced through my head. Did the satphone even make it? Could it talk to the satellite constellation at all?
The short DTMF tone blared out of the phone. The satphone made it and was working! So was the Raspberry Pi! The 15-minute connection window was open. I started my stopwatch. Back on the ground, it took the Raspberry Pi on average a minute and a half to establish a connection and notify my ground station, assuming the data call goes through the first time. At this point. the payload was close to crossing the 40,000ft mark. The tone itself was shorter than it should have been. Some of the call had dropped out.
Another minute passed before “Downlink established” came across my screen. The Raspberry Pi managed to get online! Instantly, the ground control script generated a 2FA request and dispatched it. Everything was on rails from this point forth. Did the smartphone make it through the spin? Not wanting to waste whatever precious throughput we had, I didn’t dare try to SSH into the payload to check. At that point, all I could do was sit and watch the poll requests come back one at a time, as the seconds ticked by.
My watch hit 15 minutes. The connection window was closed; the push request timed out, and the Raspberry Pi had reset state. I called the satphone and started another connection window as we pulled up to our rally point. This is where we’d wait for the balloon to pop directly overhead. Once more, the window closed. Pete, Creative Director at Duo, came by and we discussed how this was all supposed to work for the live stream, calling the satphone again as the we passed the 84,000ft mark. This would be the last shot. The call rang a couple of times and ended without playing a tone at all. We thought the call had dropped altogether. As we tried to redial it, “Downlink established” came across the screen again.
Again, we waited. Ten minutes went by before Pete returned with the latest altitude reading that my receiver had missed. We were falling. There was no way the satphone would stay connected now. That was officially the end of the push attempts. We had to refocus on following the payload and watching the telemetry. It would be a violent 20 miles back down to the ground.
The flight models were spot-on; the payload was heading straight for Jean, Nevada. We were right under it. We got off the freeway next to the local prison and headed behind the mountains. We got the first APRS packet from the ground! The payload had made it! I checked the SPOT tracker’s location. It had a fix, too, and they matched! We knew where the payload was, so we started heading in its direction. The paved roads soon gave way to dirt tracks.
Due to flooding in Vegas a week earlier, the dirt tracks gave way to impassible storm washes. We were still about a mile out and had to hike it. Before we set out, I grabbed my portable radio direction finding gear just in case and punched the coordinates into the GPS. The hike itself was pretty treacherous with the blistering desert heat and attack helicopters bearing down on us.
Within an hour, we could hear the buzzer in the distance! Everyone hurried over and surveyed the damage. The payload had clearly landed pretty hard, but everything was in close proximity.
We rushed to check the cameras. All three were there, even the boom! I was certain it would’ve been flung off, but it held on! I could still the record light illuminated on one camera, but two of them were off. Uh oh. One that was still displayed a timecode of where it was in a recording, frozen at 10 minutes and a few seconds. It too had failed for new and mysterious reasons.
We powered on the second camera and looked at how much it captured: 30 minutes. Last but not least was the boom camera, the one subjected to more than 4Gs of force from spinning at the start of the launch and broke off on landing. We powered it on and discovered it recorded more than two hours — it captured the whole flight!
Final Thoughts
We gathered the pieces and went back to the hotel to review the footage and data to piece together what happened.
Reviewing the footage showed that even though the flight was so rough, nothing flew off as a result of the 70 RPM spin at the start.
Over time, a circular, gliding motion developed, starting to wind up the the rigging line onto itself and build energy like a spring. After a while, the atmosphere thinned to the point of not pushing against the surface area, and the the winding motion stopped giving us an amazing view of the Mojave Desert and Las Vegas before reversing direction.
This spin continued until the balloon burst at a maximum altitude of 92,927 feet!
On the way back down, the camera on the boom flipped backward and got an awesome shot of our rigging. As the payload descended, it again began to spin and wind up the rigging. This caused the parachute deploy improperly, setting the stage for the rough landing that we saw.
After reviewing the logs on the Raspberry Pi, a clearer picture came together of what happened with the Duo Push attempts. The smartphone had survived the ascent and was responding to the Raspberry Pi for the entire trip. The connection log showed several timeouts due to high loss and forced reconnections. Essentially, the connection quality ended up being too low to load the push request, let alone approve it.
In the end, while we couldn’t get the Push to work, it was quite an amazing journey and we all learned a lot in the process. We wanted to do something challenging and captivating while avoiding the fear, uncertainty and doubt that plagues our industry. By that metric, this was a monumental success.
I couldn’t be more proud of what we accomplished and the footage we captured. You can explore Pushing the Boundary, our brand-new documentary about this project; watch our final payload’s full flight in 4K; and relive our journey during the livestream, all on the Duo in Space website.
