Historic Information Breakdowns

Risk managers study causes of tragedies to identify control measures in order to prevent future tragedies.  “There are no new ways to get in trouble, but many new ways to stay out of trouble.” — Gordon Graham

Nearly every After Action Report (AAR) that I’ve read has cited a breakdown in communications.  The right information didn’t get the right place at the right time.  After hearing Gordon Graham at the IAEM convention , I recognized that the failures stretch back beyond just communications.  Gordon sets forth 10 families of risk that can all be figured out ahead of an incident and used to prevent or mitigate the incident.  These categories of risk make sense to me and seemed to resonate with the rest of the audience too.

Here are a few common areas of breakdowns:

Standards: Did building codes exist?  Were they the right codes?  Were they enforced?  Were system backups and COOP testing done according to the standard?

Predict: Did the models provide accurate information?  Were public warnings based on these models?

External influences: How was the media, public and social media managed?  Did add positively or negatively to the response?

Command and politics: Does the government structure help or hurt?  Was Incident Command System used?  Was the situational awareness completed?  Was information shared effectively?

Tactical: How was information shared to and from the first responders and front line workers?  Did these workers suffer from information overload?


“Progress, far from consisting in change, depends on retentiveness. When change is absolute there remains no being to improve and no direction is set for possible improvement: and when experience is not retained, as among savages, infancy is perpetual. Those who cannot remember the past are condemned to repeat it.”  — George Santayana

I add that in since few people actually know the source and accurately quote it.  Experience is a great teacher.  Most importantly, remembering the past helps shape the future in the right direction.

Below are a list of significant disasters that altered the direction of Emergency Management.  Think about what should be remembered for each of these incidents, and then how these events would have unfolded with today’s technology – including the internet and social media.

Seveso, Italy (1976).  An industrial accident in a small chemical manufacturing plant.  It resulted in the highest known exposure to 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) in residential population.  The local community was unaware of the risk.  It was a week before public notification of the release and another week before evacuations.

Bhopal Methyl Isocyanate Release (1984).  An industrial accident that released 40 tones of MIC.  There was no public warning.  The exact mixture of the gas was not shared so the first responders did not know how to treat the public.

Chernobyl Nuclear Disaster (1986).  An explosion at the plant and subsequent radioactive contamination of the surrounding geographic area. Large parts of Europe and even North America were contaminated.  The Communistiic regime hid the initial information and did not share information until another country detected it.

Hurricane Hugo (1989).  At the time, this was the costliest hurricane disaster.  There was an insufficient damage assessment that lead to wrong resource allocation.  The survivors in rural communities were not located and responded to for many days.  Much of the response was dependent on manual systems.

Loma Prieta (1989).  A M7 earthquake that injured around 3800 in 15 seconds.  Extensive damage also occurred in San Francisco’s Marina District, where many expensive homes built on filled ground collapsed and / or caught fire. Beside that major roads and bridges were damaged.  The initial response focused on areas covered by the media.  Responding agencies had incompatible software and couldn’t share information.

Exxon Valdex (1989).  The American oil tanker Exxon Valdez clashed with the Bligh Reef, causing a major oil leakage.  The tanker did not turn rapidly enough at one point, causing the collision with the reef hours. This caused an oil spill of between 41,000 and 132,000 square meters, polluting 1900 km of coastline.  Mobilization of response was slow due to “paper resources” that never existed in reality.  The computer systems in various agencies were incompatible and there was no baseline data for comparison.

Hurricane Andrew (1993).  Andrew was the first named storm and only major hurricane of the otherwise inactive 1992 Atlantic hurricane season. Hurricane Andrew was the final and third most powerful of three Category 5 hurricanes to make landfall in the United States during the 20th century, after the Labor Day Hurricane of 1935 and Hurricane Camille in 1969.  The initial response was slowed due to poor damage assessment and incompatible systems.

Northridge Earthquake (1994).  This M6.7 earthquake lasted 20 seconds.  Major damage occurred to 11 area hospitals.  The damage made FEMA unable to assess the damage prior to distributing assistance.  Seventy-two deaths were attributed to the earthquake, with over 9,000 injured. In addition, the earthquake caused an estimated $20 billion in damage, making it one of the costliest natural disasters in U.S. history.

Izmit, Turkey Earthquake (1999).  This M7.6 earthquake struck in the overnight hours and lasted 37 seconds.  It killed around 17,000 people and left half a million people homeless.  The Mayor did not receive a damage report until 34 hours after the earthquake.  Some 70 percent of buildings in Turkey are unlicensed, meaning they did not get approval on their building code.  In this situation, the governmental unit that established the codes was separate from the unit that enforced the codes.  The politics between the two units caused the codes to not be enforced.

Sept 11 attacks (2001).  The numerous intelligence failures and response challenges during these three events are well documented.

Florida hurricanes (2004).  The season was notable as one of the deadliest and most costly Atlantic hurricane seasons on record in the last decade, with at least 3,132 deaths and roughly $50 billion (2004 US dollars) in damage. The most notable storms for the season were the five named storms that made landfall in the U.S. state of Florida, three of them with at least 115 mph (185 km/h) sustained winds: Tropical Storms Bonnie, Charley, Frances, Ivan, and Jeanne. This is the only time in recorded history that four hurricanes affected Florida.

Indian Ocean Tsunami (2004). With a magnitude of between 9.1 and 9.3, it is the second largest earthquake ever recorded on a seismograph. This earthquake had the longest duration of faulting ever observed, between 8.3 and 10 minutes. It caused the entire planet to vibrate as much as 1 cm (0.4 inches) and triggered other earthquakes as far away as Alaska.  There were no warning systems in the Indian Ocean compounded by an inability to communicate with the population at risk.

Hurricane Katrina and Rita (2005).  At least 1,836 people lost their lives in the actual hurricane and in the subsequent floods, making it the deadliest U.S. hurricane since the 1928 Okeechobee hurricane.  There were many evacuation failures due to inadequate considerations of the demographic.  Massive communication failures occurred with no alternatives considered.


Additional resources


Autonomous systems and robotics

An autonomous system is a system with a trigger that causes an action without the involvement of a person.  Two well known examples are tsunami buoys and earthquake sensors.  These continuously monitor a series of sensors.  When the sensors register a reading that exceeds a predefined threshold (the trigger), a signal is sent to a computer to generate warning alerts (the action).  These systems can range from the very simple to the extremely complex.

One could argue that the backup sensors on cars are an autonomous system: the trigger is the driver shifting into reverse, the computer turns on the back up lights, activates the camera, turns on the internal screen, and activates the sensors to start beeping.  I think it is a bit of a stretch but it does provide a simple example.

The other side of this spectrum is the Mars Rovers.  Due to the time delay in communications between Earth and Mars, the rover cannot be directly controlled.  NASA gives a general command for the rover to head to a new destination.  The rover acts independently to drive over the terrain, and makes decisions to avoid obstacles.  On-board the rover is the Autonomous Exploration for Gathering Increased Science (AEGIS) software — part of Onboard Autonomous Science Investigation System (OASIS) —to automatically analyze and prioritize science targets.  This allows more work to be done without human intervention.

Somewhere between the two is the Roomba.  This autonomous vacuum moves around the house cleaning up.  It will learn the layout of a room to be more effective in future runs.  When complete, it docks to recharge.  At a set time interval, it does this again.  Now don’t laugh about the Roomba; it is a very commonly hacked robot for people interested in DIY robotics.  Microsoft Robotics Developer Studio software has built in modules specific to controlling the Roomba.  That gives mobility and a way to control it.  Microsoft has released additional modules for the Robotics software which adapts the Kinect sensor to sense the environment.  This addition provides vastly better sensors over the traditional range-finder and light sensors.  Microsoft isn’t the only player in this market; Lego Mindstorms markets robotics as a toy for children ages eight and up.  Robotics isn’t just for engineering students in high-end tech universities.

There is enough technology in existence today to make huge leaps in the use of robotics.  The main challenge of robotics is the acceptance by the general public.

Watch the videos about the DARPA autonomous car, Urban Search and Rescue robots, BigDog robotic transport and Eythor Beder’s demonstration of human exoskeletons.  Combine these and we can vision some major transformations.

Take the computational guts from the autonomous car and put them into a fire engine.  Now the fire engine can be dispatched and navigate to a fire scene independently.  Once on-scene, sensors can detect the heat from the fires – even if they are in the walls and not visible to the human eye.  Robots from the fire engine can be sent into the structure.  Heavier duty robots can pull hoses and start to extinguish the fire.  Other robots can perform a search on the house to assist survivors out, and rescue those unable to escape.  Communication systems will link together all the sensors on the robots to generate a complete common operating picture that all robots use in decision making.

A similar thing can be done with an ambulance.  Focus just on the stretcher.  Imagine if the stretcher would automatically exit the ambulance and follow the medic with all the necessary equipment.  The stretcher could be equipped to up and down steep stairs, make tight turns and remain stable.  Automating the lifting and bending done by EMS workers handling patients on stretchers would reduce the number of back injuries caused by improper lifting.  This would keep more EMS workers in better health which reduces the sick leave and employee compensation claims.

Robotics in search and rescue could be the one thing that saves the most number of lives, both in the victims and the rescuers.  A building collapses and the SAR team responds.  On the scene, the workers setup specialized radio antennas at different points around the building site.  They release dozens, if not hundreds, of spider or snake like robots.  Each robot has the ability to autonomously move through the rubble.  They are light enough to not disturb as a human would do.  They are numerous enough to more quickly cover the building as a human would do.  The combined sensor data of their location and scans would quickly build a three dimensional model of the rubble.  They are location aware of each other so they don’t bunch up in one area or miss another.  Heat, sound and motion sensors could detect people.  Once this initial scan is done, the SAR team will be able to know where the survivors are and communicate with them through the robots.  The team will evaluate the 3D model for the best entry paths to get to and rescue the survivors.  If the situation is unstable, larger robots can be used ahead of the team to provide additional structural support to reduce the risk of collapse.  If a robot can’t communicate with the antennas outside, the robots can do mesh networking to pass information along.

My bag of holding

From time to time, I pull something out of my bag and folks wonder just what I carry in it.  So here is the contents of my bag that I carry with me nearly everywhere.  It is my home, commute, work, disaster and COOP bag.

Picture of the contents in my every day laptop bag.

I start with a Timbuk2 messenger bag.  Unlike most messenger bags that are horizontal (wider than tall), this bag is vertical (taller than wide).  It is TSA friendly too with a separate laptop compartment.  I looked at their site and it might not be made anymore.

Top left of the table is a USB clip extender.  This handy doodad clips the USB aircard to the top of the monitor for better signal reception.

Two power supplies for the laptop.  The left one is 12v DC and the right one is 110v 90w.

The silver thing in the middle of the top row is Imodium.  Because when it is needed, it is needed right away.

Two standard micro-USB chargers.  All my USB chargeable devices are standardized on the micro USB.

Near the standard charges are two USB blocks.  The bullet shaped one is for 12v to USB.  The square one is 110v to USB.

The far right of the table is a couple micro USB cables, and one mini USB cable.

Next row start at the business cards.  Note the high quality business card holder.

Pens, assorted.  One of those is really a pencil.

Nail clippers.  Also cuts cable ties, hanging threads, and anything else that needs a nip.

USB aircard.  This one happens to be a 4G Verizon card.

Surge protector.  Three 110v outlets and two USB outlets.  Handy when the hotel or airport only has space for one plug.  The short extension cord goes with this so the other wall outlet isn’t blocked.

Bluetooth mouse.  There is only so much of a touch pad that one can tolerate.  Honestly though, my wife uses that more then I do.

USB sticks.  The black one is an IronKey for sensitive data.  The other two are for file movement only.  I don’t store data on unprotected USB sticks.  Risk of theft/loss is too great.  IronKey moved all the secure USB products to Imation.

Finally, the bottom left of the picture: headsets.  Two are simple listen only.  One has a microphone for phone calls.  They break or get lost so often, that I keep stashing more in the bag.

Not shown:



Extra laptop battery

Note pad … I mean paper, not electrified or anything.



The possibility of 4G

It is important to take note of the possibility of 4G.  A T1 circuit is 1½ Mbit/s.  The minimum 4G standard of 100 Mbit/s is 66 times larger.  Take a look at the graphic posted on my blog at http://keith.robertory.com/?p=560 for a better understanding of this.  A cell phone running true 4G will have more bandwidth then an entire site serviced by a T1.  We are right on the verge of a major cellular service shift.  When setting up a site during a disaster, it is common to use one cellular data card (aka aircard) per computer.  With these faster speeds, we can use one cellular data card to be the head of the site’s network.

My team has already successfully setup a network in a disaster with one 4G aircard providing connectivity for 30 computers.  Granted it was rare that there were users on all 30 computers simultaneously surfing the net and streaming large files.  But, that’s the point during disasters — and really even day to day.  It isn’t about providing maximum bandwidth to each user all the time.  Instead, focus on load balancing to provide enough bandwidth to meet the combined average need ~90% of the time.  It is ok for the system to be a little slower during peak demand times.  Set the user’s expectations correctly, and your team will get through it.

A cellular connection could be used to back up a wireline circuit.  Advanced routers can handle multiple uplink connections with prioritization and failover settings.  This will provide redundancy.  It is better than two wireline circuits backing each other up when the backhoe cuts through the utility lines outside the building.  Redundancy is nice.  Diverse redundancy is better.

Your users in a disaster response will be on the computer only part of the time, with the rest of their time filled with other activities.  If a disaster responder travels to a location and spends the entire time behind a computer, then the question should be asked: could that person just stay in the office or at home to complete the same work?

If this interests you, take a look at this post.

Red Cross comm team ready for disasters

An interview that I did and is posted originally at http://www.networkworld.com/news/2011/082911-red-cross-comm-team-ready-250197.html?page=1
and http://www.cio.com/article/688730/Red_Cross_Comm_Team_Ready_for_Disasters

Red Cross comm team ready for disasters

IT system’s design is based on experience from years of disaster experience, says Red Cross IT exec

By Matt Hamblen, Computerworld
August 29, 2011 01:34 PM ET
While most businesses back up data and records as potential disasters approach, the American Red Cross has a communications and information systems infrastructure built to bring key data into areas ravaged by storms like Hurricane Irene . Continue reading Red Cross comm team ready for disasters

Flood Humor

A local faith group held a disaster preparedness fair to encourage everyone to be ready in case the nearby river flooded the town.  One of the church going ladies said “No thanks. I have faith that God will keep me safe.”

Inevitably, the river flooded in the spring.  The mayor called for an evacuation of the area.  A truck drove by the chuch lady’s home to assist her.  She called to them, “No thanks, God will keep me safe.”

The river waters rose to her home, and she moved up to the 2nd floor of her home.  A boat came by to rescue here.  She turned them away, “No thanks, God will rescue me.”

The water rose even more.  She sat on the roof of her home and prayed.  A helicopter came down to rescue her.  She waived them off, “No thanks, God will save me.”

The water washed her away and she drown.  Standing before God, she said “What happened?  I put my faith in you and you let me down.”  God look at her and said, “I sent people to help get you prepared.  When the flood came, I sent you a truck, boat and helicopter.  You turned them away.”

Fox News Interview, March 18, 2011

I was interviewed by Garrett Tenney of Fox News for a story about cell phone use in disasters.  The story was published on March 18, 2010 at http://www.foxnews.com/world/2011/03/18/memorizing-cell-phone-numbers-save-times-crisis/.  It is reprinted here.

Memorizing Cell Phone Numbers Could Save You in Times of Crisis

By Garrett Tenney
Published March 18, 2011 | FoxNews.com

Many Americans feel naked or lost without their cell phones. 

But in times of crisis those very devices — instead of connecting people — can sometimes lead to collapses in communications.

One reason: who memorizes cell phones numbers anymore? 

A week after Japan’s 8.9 magnitude earthquake, there are still more than 10,000 people unaccounted for. 

Philippe Stoll, a spokesman for the International Red Cross, told the BBC earlier this week that people are still alive, but can’t tell anyone because cell phones that were not swept away by flooding waters quickly ran out of power. 

“I don’t know how many of the phone numbers saved on your mobile phone you know by heart,” Stoll said. “How do you reach someone whose number you have in the mobile you lost?”

In tech-savvy Japan, cell phones are widely used by young and old, as opposed to the U.S., where they are predominantly utilized by just the younger generation, said Ken Wisnefski, founder and CEO of Webimax.com. 

“In Japan, even the older generation was reliant on technology, for some time, so the impact of this crisis is more far reaching because a large part of the population relied so heavily on that technology,” said Wisnefski. 

A study released earlier this month by Research and Markets, the world’s largest market research firm, revealed that of Japan’s population of roughly 127 million, 117 million are mobile subscribers and 90 percent of those users have access to a high speed 3G network. 

Communication in and out of Japan has begun to improve, and some wireless carriers, such as Verizon Wireless, Sprint and Comcast have been offering free calling to Japan from the U.S. 

But, in some of the hardest hit areas, communication with the outside world and emergency responders is still difficult. 

Keith Robertory, manager of disaster services technology with the American Red Cross, said this is a reminder to everyone to be prepared in the event of a disaster. 

He said people can take these few, simple steps to help improve communication and get you on the path of personal recovery:

–Save all your contacts on your home computer, update them every few months, and print a hardcopy of your contacts to keep in your car in case of an emergency. 

–Write down the toll-free numbers for your banks and utility companies. In the event of a disaster, this will allow you to turn off your utilities, reprint credit cards, and temporarily change your address. 

–Designate a friend or family member who lives outside your area to be your family’s emergency contact. In emergency situations, long-distance calls have a better chance of getting through jammed phone lines because they only require one connection to get through, while local calls require two connections. 

–If you are in a disaster area, and aren’t able to get a hold of family or friends on your phone, change your voicemail to say the current time, your location, and that you are safe. This will allow anyone trying to reach you to know you’re alive and where rescuers can find you. 

Robertory said communication in crisis situations is vital for families and communities. Although preparation is a personal decision, families should make plans to handle disasters, he said.

Satellite 2011 Conference Interview

This interview is reposted from the Satellite 2011 Conference page.

An Inside Look with Keith Robertory, American Red Cross

Conference Chairman Scott Chase sat down with Keith Robertory, Disaster Services Technology Manager, American Red Cross, to discuss the relationship between government agencies and the satellite industry when a disaster strikes. You can hear more from Keith Robertory and other experts at Satellites to the Rescue on Tuesday, March 15

Scott Chase: In the event of an emergency situation of any type, how effective is coordination of government and industry satellite resources, and how does that all work?

Keith Robertory: Coordination of shared resources is going to be very important. Many organizations have satellite technology positioned as the emergency solution when terrestrial services do not work. If all these organizations pull out their satellite equipment during a disaster and try to use it, the limitations of shared bandwidth abruptly smack these organizations with reality. The satellite industry needs to work with its clients to better educate them on potential limitations. 

Offers of donated satellite systems and air time are welcome at any time. That said, the worst time to engage a response organization with an offer of new equipment, new technology, and limitations unknown to them is right after a disaster occurs. The priority of key decision makers will be the response effort. Effective government and industry coordination occurs long before the emergency situation ever arises.

SC: What applications do satellites bring to support the communications requirements of users in remote locations during and after the disaster?

KR: The American Red Cross satellite infrastructure has the IP packet as a common foundation. We are not trying to push different modes and protocols through the equipment. IP allows the core network to handle the information at a very basic level, reducing the number of conversions between the source and the destination. It can be data packets, voice or video, but it is all based on the basic IP packet. The trick to be successful in disasters is to make technology transparent. 

SC: What are the biggest changes you have seen in the use of satellite technology and equipment over the course of your own quarter-century in the high-tech arena?

KR: Satellite technology is becoming more and more commonplace. Imagine a couple decades ago telling someone that we’re going to send a radio signal to their car from tens of thousands of miles away. Small transceivers used to only exist in the realm of science fiction. It doesn’t seem that long ago that connectivity between computers was about as fast as you could read plain text. Now we are streaming HD video, video teleconferences, and entire site connectivity through a single satellite connection using equipment that is (relatively) easily shipped from site to site.

On the flip side, technology advances and shifts in philosophies are bringing previous “obsolete” concepts back to the mainstream. Many people consider cloud computing to be a new technology, but it isn’t. We used to call it mainframes and terminals. The current events in Egypt show that no matter how advanced the technology, technologists need to be ready to fall back to older methods to establish connectivity. Egypt is an example of how to communicate should a nationwide network be disrupted. Disaster technologists should be versed in many different tools.

SC: What would you say has been the biggest advance in satellite capability since you joined the American Red Cross nearly 15 years ago?

KR: Honestly, we have not made many substantial changes to the American Red Cross satellite system since it went live in 2000. Standing up a system of the size we have is a costly endeavor and major changes also cost more money. We’re in the maintenance mode of the IT life-cycle. We are going to keep the system running as long as possible because a poor economy is not the time to request a multi-million dollar upgrade that may not have a measureable direct impact on the mission to deliver disaster relief services to disaster survivors. 

Our system has grown to have two downlink stations and nearly 80 remotes in the field, including 12 satellite trucks. It is a completely internal system behind and protected by our corporate IT systems. The only thing we don’t own is the satellite itself. What has changed is how we use satellite capability and the philosophy behind technology selection. 

Consider that all the technology we have is a tool in a tool box. We are first and foremost a service delivery organization.  My unit’s objective is to establish connectivity in a disaster zone. We need to leverage everything in the most mission-sensible way to balance cost with service delivery. Technology that doesn’t enhance service delivery isn’t used.  The situation drives technology needs. As there is less and less local infrastructure, the selected tools shift to satellite-based technologies. 
Terrestrial technology, like cellular, is giving satellite a good run for its money. Cellular is getting faster, cheaper, and more resilient to disasters then it has been in the past. Satellite is also getting faster and cheaper. The decision point between where we can expect cellular to work and when to shift to satellite is in constant motion. Both are getting better but one will never replace the other for disaster work. Use the right tool for the right job. There is no single magic bullet idea.

SC: What can the global satellite industry do better to facilitate emergency response and humanitarian efforts at the scene of major disasters of any type?

KR: The satellite industry must reach out to humanitarian and other response organizations long before disaster occurs. It is challenging to fit a new connectivity solution into an existing network that is activity being used to respond to a catastrophe somewhere. And I say “catastrophe” because it seems to take the huge disaster to get lots of companies off the bench and in the game. Taking satellite technology to an organization responding to disaster is similar to telling a freighter that you’re going to change its propellers while it is navigating a horrendous storm in the North Atlantic.

Haiti was a time when this was successfully done, and that is an exception. Haiti was the largest response of the International Federation of the Red Cross. The technology that is normally sufficient for a disaster was quickly out-scaled and couldn’t keep up with demand. Luckily, the American Red Cross domestic response team’s experience with satellite was able to screen and facilitate the offers of satellite service on behalf of the international response team who could not shift attention off the response.

SC: In your role as supervisor of literally hundreds of volunteers, many of whom may have never seen, for example, a satellite phone, what is the one thing the satellite industry could do now to simplify emergency response?

KR: A larger diagram on the satellite phone to tell them to use it outside would be a good start. Simplify, simplify, simplify. Any device that is stored “for emergency use only” will not be successful in an emergency without a lot of training. In my experience, even that can be questionable. The best “in case of emergency” device is one that a user uses every day and is resilient to disasters. The American Red Cross actually uses fewer satellite phones then you probably think. Amateur radio plays a vital role in the first couple days of a disaster, and cellular is coming back online after that. Satellite phones are used in pocket areas where we have few other working options.

It is fair to say that all the technology deployed to an American Red Cross disaster response is received, set up, managed, troubleshot, packed up and shipped back by volunteers. I’m blessed with a cadre of high caliber volunteers who can use technology and speak human. We’re the high-tech in a human-touch organization. My technical volunteers in the field are the support system for the volunteers in the field that directly touch the clients. A key to our success is a step-by-step job aid for every action that needs to be done. These range from wiring a laptop to setting up a full VSAT. As long as a new volunteer is willing to be flexible and follow directions, we can put them to use in American Red Cross Disaster Services Technology with minimal upfront training.  Our more experienced volunteers can get more deployment and training opportunities.

The short answer comes back to simplifying the technology to be more reliable, set up quickly with less user intervention, and require less hands-on to keep it running.

Join Keith for Satellites to the Rescue: Industry and Government Partnership in Disaster Relief, 4:30 p.m. to 5:45 p.m. on Tuesday, March 15, Room 207A. Access to the session is included with your Full Conference registration.