Fire at Podex data center temporarily halts payments

OVH Strausbourg data center on fire March 10. (Image courtesy Feuerwehr Kehl via Wikipedia Commons.)

Podex, the second-largest OpenSim payments provider, has suffered a fire at its data center and services are temporarily down, but no user money has been lost.

Jacek Shuftan

“The Podex system is on OVH servers based in Strasbourg and fire destroyed some of their data center,” Podex CEO Jacek Shuftan told Hypergrid Business. “We don’t know if our data was destroyed as no details have been published so far.”

However, users’ currency is safe, he said.

“Nobody can lose any money because Podex does not keep any money of the users,” he said.

Instead, Podex only facilitates the purchase and sale of the virtual currency.

The part of the data center where Podex was located is scheduled to be up and running again on Monday, he said.

“If all is okay, Podex will be working as before,” he said.

If Podex’ own server was destroyed in the fire as well, there is a recovery plan, he said. “We will simply set up the Podex system from backups on another server, but it will take time.”

If Podex has to restore from backups on a new server, then the system might not be fully functional again until the end of next week, he said.

For now, the payment system is down and OpenSim users cannot use their Podex currency.

“We simply have no possibility to process payments and the ATMs are not working at all,” said Shuftan. “However, we will soon set up a temporary system of accepting payments and will send coins manually so that users will not suffer more than its necessary. I do hope it will be done tomorrow.”

Users will also be able to buy currency on the Podex website, he said. “We will send them manually.”

In-world transactions uninterrupted on DigiWorldz, Discovery

For OpenSim shoppers who use Podex-based currencies in-world, transactions continue uninterrupted at some grids because that part of the processing happens on the grids’ own servers.

One of the grid using Podex currency is DigiWorldz, where region owners have a choice of using local D$ currency, which is uses Podex, or the multi-grid Gloebit currency. But the grid only uses Podex to buy and sell D$, not for in-world transactions.

Terry Ford

“As long as our users have Podex-based money in the grid, they can continue to use it and everything functions as it should,” said DigiWorldz owner Terry Ford.

Cashouts, however, are a different story, he told Hypergrid Business. “We don’t cash out, so our users will need for Podex to come back online for cashouts.”

Those regions owners who currently have Podex currency enabled on their regions can ask DigiWorldz to switch them over to Gloebit, he added.

DigiWorldz has also been loaning Podex currency to those who need additional D$, he said. “Once the users are able to buy Podex currency, they will repay the loans provided.”

Another grid using the Podex currency is Discovery Grid., with the local DC$ virtual currency.

“Regions using DC$ virtual transaction are not affected since that all happens on Discovery Grid local servers,” said grid owner Rene Vega, also known as “Balpien Hammerer” in-world. “It is only cashouts or new currency purchases that are not available at this time. Balances are also kept on grid servers.”

He said that he has not received any support tickets from residents related to the Podex problems.

“Cashouts tend to be done in bulk, so they happen infrequently,” he told Hypergrid Business. “So, it is just a matter of time for Podex to execute its disaster recovery plan.”

Discovery Grid also supports Gloebit, currency, and those regions are not affected at all, he added.

Fire details emerging

Hosting provider OVH said in its statement that nobody was hurt in the fire and that services will begin to be restored next week. If that is not soon enough, customers can also move their systems to one of their other data centers.

Podex provides the infrastructure for OpenSim grids to have their own virtual currencies. Each grid has its own currency, but Podex allows users to trade one grid’s currency for another.

This is in contrast with OpenSim’s biggest payments provider, Gloebit, which offers a single currency that can be used on many OpenSim grids, via a single wallet accessible in the viewer.

Podex currently lists 27 grids as customers on its website, but some of those grids have since switched away, and others have been closed for quite a while.

According to Data Center Knowledge, OVH is one of the world’s most popular hosting company, after the big three of Amazon, Google, and Microsoft.

Millions of websites are reportedly down as a result of the fire. It’s not yet clear whether OVH has had its own off-site backups of the servers.