Resilience during virtual disasters

Virtual worlds have been offered up in recent weeks as alternatives to physical events because they are disaster-proof. No volcano or tsunami can take down the whole Internet. And virtual events are particularly budget-friendly during that other kind of recent disaster — a financial crisis.

But virtual worlds are prone to their own kind of disasters, as yesterday’s shutdown of Second Life demonstrated. OpenSim isn’t immune either, with both local outages and widespread failures possible. For example, this past weekend, the owner of Aesthetica found that his entire region was gone, and several months of scheduled backups had never taken place at all — a combination of hardware, software and management failures. Last summer, a hacker took out servers running over a hundred thousand websites — and around a hundred OpenSim regions were destroyed in the attack as well.

Image by Clearly Ambigous (via Flickr)
Image by Clearly Ambigous (via Flickr)

If that’s not enough, an entire virtual world can disappear overnight, leaving corporate customers hanging in the wind. For example, Coca-Cola lost its CCMetro center in There.com when that world closed in March. Not only did the physical infrastructure vanish — the buildings and objects, the designs, the interactive elements, and all the labor that went into them — but also the social relationships that Coca-Cola developed with local residents.

Finally, a virtual world hosting company can unilaterally decide to sever a relationship with an existing customers, as recently happened with Woodbury University’s island in Second Life.

Fortunately, many of the lessons learned by enterprise users during the last 15 years of the Internet can also be applied to virtual worlds. After all, websites were notoriously unstable during the early years of the Web. It wasn’t until 1999 that Akamai launched its services, allowing websites to be duplicated and distributed around the Internet. Prior to that, websites were typically run on individual servers, and sensitive to hardware and software failures as well as network outages.

BACK UP CONTENT

One of the most critical elements of a virtual world disaster recovery plan is to have back up copies of all content in a separate location, preferably one controlled by you, or at least by a different vendor that the one running your virtual world. Most enterprise-grade virtual worlds allow back ups, but many social worlds do not. In the latter case, a copy of the virtual build can be created manually, by downloading and duplicating individual objects. If the main world goes down, or goes out of business, the duplicate world can be transferred to a new location.

For example, a Second Life build can be copied or duplicated in a private OpenSim build. Worlds that support mesh imports can be duplicated in a virtual platform that supports 3D mesh standards, such as Open Wonderland (formerly Sun’s Project Wonderland before Oracle bought the company and shut it down) or the realXtend branch of Opensim.

But content isn’t limited to builds alone. Second Life scripts can be copied to OpenSim, and will usually run with no or minor modifications. Scripting languages of other virtual worlds, however, are more difficult to export.

OpenSim content can be backed up in seconds and saved, as an OAR file, in multiple locations. OAR files include region terrains, and all the content located on those regions — objects, scripts, and textures.

In addition, the social aspects of virtual worlds are often very important for companies, especially those who participate in a virtual world as part of their marketing efforts. To maintain contact with their communities in case of an outage, or a shutdown of the entire world, companies should ensure that they have other ways to contact their users, such as mailing lists. For example, instead of encouraging their users to join in-world groups, companies can promote membership on their websites instead.

BACK UP SERVERS

With a presence in a social world like Second Life, a company is dependent on that company’s hardware.

Running an enterprise-class virtual world, like Second Life Enterprise or OpenSim, however, is more like running a website — you can choose to run it on servers located at your firm, or at a third-party co-location facility, or even with a cloud-based service. Or you can use all of the above, switching between providers as traffic warrants.

There are a number of OpenSim hosting providers, for example, that can be used to run backups of a company’s private virtual world.

Unfortunately, there is as yet no service similar to Akamai that will take a virtual world and duplicate it around the Internet. It took almost a decade for Akamai to appear — the first website went up in 1991. Hopefully, it will take less than a decade to get a similar level of resilience for enterprise virtual worlds.

ALTERNATE DESTINATIONS

If you’re planning to hold a mission-critical meeting in a virtual world, consider a pseudo-3D platform like On24 or Unisfair instead, which are basically hyped-up, interactive websites. These platforms offer many of the advantages of virtual worlds but can handle many more simultaneous visitors. Since they are traditional websites, they benefit from all the advancements in Internet stability and resilience that have taken place over the last 15 years.

If you’re holding a meeting in an immersive virtual world, expect that some of your visitors won’t be able to make it because of hardware or software problems on the client side. Offer alternative means of participating, such as streaming video or a live chat channel.

Finally, have an alternate venue lined up in case your first choice is not available. This can create some logistical problems, however. For example, Blue Mars is not a practical alternative venue to Second Life, since it will take considerable time and effort for your users to download the new software, register themselves as users, and learn to use and navigate the Blue Mars virtual environment. It’s not reasonable to tell your guests a few minutes before the event that the Second Life venue is not available due to grid-wide maintenance or outage, and you’re all heading over to Blue Mars.

Similarly, an OpenSim grid — while much closer to functionality to Second Life — requires re-configuration of the Second Life client, or a download of an OpenSim-friendly viewer like Hippo, Imprudence or Meerkat that has built-in multi-grid support. This can be tricky for first-time users to do.

If the primary meeting venue is on an OpenSim grid, however, there are a number of public and private grids offering event hosting. OpenSim grids are run by different companies, on different servers, in different countries — if one is down, it’s unlikely that others will go down simultaneously. Even last summer’s outage only took out 100 regions, out of thousands of OpenSim regions then in use. List of the top 40 largest OpenSim grids is here.

However, switching to another OpenSim grid will sometimes require that users create new avatar accounts, which can be time-consuming and disconcerting. In addition, users will lose access to their inventories by doing so. One alternative is to pick a destination that allows hypergrid teleports. Hypergrid teleports allow users to teleport in from their home grids while retaining access to their avatar shape, appearance — and to the PowerPoint slides in their inventories. Directory of hypergrid-enabled meeting venues is here.

Hypergrid teleports are a new technology, however, that many users aren’t familiar with, and many grids don’t support. In addition, teleports will often fail if the destination grid is running a much newer version of OpenSim. In addition, OpenSim venues often lack security features. Most social worlds, Second Life and OpenSim included, allow administrators to have access to inventory items and chat transcripts, so before deciding to hold a sensitive meeting on a virtual world, find out who runs it and what security policies they have in place. Your conference center might be operated by a nosy teenager in his parents’ garage — or a potential business competitor.

Maria Korolov