OSgrid halfway through third data recovery attempt

OSgrid has successfully recovered 1.9 terabytes, out of 3.5 terabytes total, grid president James Stallings II said during a conversation on OSgrid’s Internet Relay Chat channel yesterday. The conversation was reposted on OSgrid’s forums by grid administrator and OpenSim developer Allen Kerensky.

Stallings — who was posting under the user name “leTwitch” — did not say how much longer the recovery would take, or in what form OSgrid would be once it came back up.

Unlike smaller grids, which use mySQL databases for their assets, OSgrid uses a structured file system, he said. There are a total of 21 million files in OSgrid’s system that have to be recovered.

This is the third and last recovery attempt for OSgrid.

OSgrid's LBSA Plaza is the cross-roads of the hypergrid.
OSgrid’s LBSA Plaza is the cross-roads of the hypergrid.

The first two recovery attempts involved “partition cloning,” Stallings said.

“These completed successfully and tested out at recovery lab,” he said. “But when hooked up to our server still showed unrecoverable issues.”

Stallings added that the grid will be making some changes to its backup system, to avoid similar problems in the future.

“We’re discussing, for instance, a pair of asset servers in ‘hot and ready standby’ configuration — one writes everything written to it to the other,” he said.

“This would be hella simple to do with SRAs,” he said, referring to a storage technology known as Storage Replication Adapters. “I think we’ve reached that point. SRAs will be a part of what ever we do moving forward, though it might see some tweaking for effect.”

Lack of communication

In yesterday’s discussion, Stallings also addressed the grid’s communication issues.

“I wish people would stop saying there is no new information,” he said. “That’s just a lot of FUD. That is whats getting old, as I mentioned less than a week ago.”

In addition to the Internet Relay Chat channel, he also suggested that OSgrid residents can look to Twitter and Facebook for information.

However, OSgrid’s Twitter page hasn’t had a relevant update since mid-October. The grid’s home page just has that same Twitter feed.

OSgrid’s Facebook page has had two relevant updates — “Grid is currently offline, no ETA at this time. Sorry folks!” on August 18 and then a slightly longer note on November 4, saying: “Concerning the recent and longstanding problems with OSgrid, we’re basically continuing in a holding pattern until the first of the coming week, at which time if we don’t have our files recovered, we’ll be moving forward without them.”

A total of five updates have been posted on the OSgrid news page, the last one a copy-and-paste from an October 15 Internet Relay Chat channel conversation promising that “if nothing else, the end of the waiting will soon come to pass.”

When contacted by email today, Stallings declined to provide any additional information about the status of the latest recovery attempt.

There are updates less than one day old on the forums,” he told Hypergrid Business. “There is nothing more to know than what is to be found there.”

Maria Korolov