Paper: hypergrid allows virtual world to grow without limit

The invention of the hypergrid will allow virtual worlds to grow to the size of the World Wide Web, or bigger, according to a paper made public today by OpenSim core developer Justin Clark-Casey.

“The World Wide Web is much larger than any existing virtual environment system,” he wrote in the dissertation, submitted for his Masters in Software Engineering at the University of Oxford. “The World Wide Web has 1.6 billion estimated users while even the most popular online role playing game has only 11.5 million monthly subscribers.”

One problem is that virtual environments are normally highly centralized — an environment’s servers, databases, even browsers are typically all controlled by a single company.

Justin Clark-Casey

According to Clark-Casey, an “Internet scale” virtual environment should be resilient — if something happens in one part of it, the rest of it should stay up. In addition, such an environment should allow for independent deployment — new pieces added, running on new computers, without affecting how the rest of the system functions.

Put together, this is often referred to as a resilient,”loosely coupled” network — parts of the network connect to other parts, but if pieces disappear, are added, or are replaced, the network as a whole continues to function.

Such a network would allow unlimited numbers of concurrent users, allow different companies to host different parts of the network, and avoid having a single point of failure.

A scalable virtual environment will also allow users to take their single avatar identity and travel with it across the entire environment — and also access their inventories from any location on the network.

Because of their reliance on centralized databases, centralized servers, and centralized controls, platforms like Second Life, World of Warcraft, and many other popular virtual environments aren’t likely to develop in a scalable direction, he said.

“I don’t believe that expanding the current classic architecture is a viable solution for producing an Internet-scale network,” he said.

One partial solution to create multiple grids, each separate from one another, he said.

Many video game world do this, in a process known as “sharding.”

Another partial solution is to allow third-party hosting of parts of a grid, while having a central administration database to coordinate travel between the regions, and to store inventories and avatar identities. This is the approach taken by OSGrid, the largest grid running on the OpenSim platform. OpenSim allows third-party vendors to connect their own servers to the central grid. Many regions are even hosted on home computers.

However, even with off-loading region hosting, there is a limit to how big the central databases get before it starts affecting performance.

Enter the hypergrid

Crista Lopes

A solution that addresses all of these concerns while offering resilience and unlimited scalability is the hypergrid, invented by Crista Lopes, a professor at the University of California.

In the hypergrid, each grid runs independently from each other grid, allowing companies, organizations, hosting providers and hobbyists to run as many different grids as they want.

However, each grid can also communicate with any other grid, using the hypergrid protocol, allowing avatars and objects to travel between them.

In his paper, Clark-Casey discusses the first version of the hypergrid protocol, HG 1.0, which lacked protection for user inventories. Today’s HG 1.5 version locks down the inventories so that rogue grids can’t access the assets of visiting avatars — while still allowing avatars to transfer their possessions from one grid to another.

A planned HG 2.0 will add even more layers of security to hypergrid travel, by moving some functionality out of the server running the virtual world into the viewer software used to access the world. For example, today, to rename an inventory item,  the request is passed from the viewer to the region server, which passes it along to the grid where the avatar originated from. In Hypergrid 1.0, the region could eavesdrop on that request — or change it to something more damaging.This is a problem when avatars are traveling around to strange and distant grids, some of which may be operated by unscrupulous — yet technically-adept — people.

With Hypergrid 2.0, that request will be passed along from the client straight to the home grid, entirely by-passing the region servers on the foreign grid.

“The direct client services architecture appears the most promising for an Internet scale virtual environment system,” Clark-Casey said. “It allows many independently operated simulators and services to exist in the same network. It also resolves the security problems inherent in proxying user and inventory service access through the simulator.”

In addition, the assets don’t necessarily have to live on grids — they can be located anywhere on the Internet, and fetched by URL addresses.

“Under this scheme, user, asset and inventory services can now be operated completely independently of the simulators,” he said.

An avatar looks at Crista Lopes' model of hypergrid teleport connections between three separate grids.

Both Hypergrid 2.0 and Clark-Casey’s independent asset storage plan require significant changes in the viewers used to access the OpenSim virtual worlds.

Today, however, such access is accomplished by the use of Second Life viewers and compatible third-party viewers. As a result, there may be a significant wait before Clark-Casey’s full vision is realized.

Maria Korolov