Kitely moves assets to Amazon cloud

By:
On:
Tagged:
With: 12 Comments

On-demand hosting provider Kitely Ltd. has eliminated a major OpenSim scalability obstacle by moving its asset database to the Amazon storage cloud.

“This is the most significant change we’ve made to our system since our beta started,” Kitely CEO Ilan Tochner told Hypergrid Business.

Because of the upgrade, Kitely was down for eight hours on Monday while data was migrated from the old database to the Amazon cloud, Amazon S3.

Previously, Kitely was using Amazon’s computing cloud for the virtual servers running individual OpenSim regions — but all the assets, such as avatar inventories and in-world content, was stored in the standard OpenSim database. Since this one database had to handle all the content for all the avatars and regions, it was a significant bottleneck.

“The standard OpenSim asset server just doesn’t scale well,” Tochner.

Now, the database is mostly eliminated, and individual assets are stored in the Amazon storage cloud — which can handle an almost unlimited number of simultaneous access requests.

According to Oren Hurvitz, Kitely’s vice president for research and development, the new system, the Comprehensive Assets System (Cassy) will be used to store all the textures, sounds, animations and other content used in OpenSim.

“Cassy enables Kitely to have faster world startup times, less server lag and provide faster viewer rezz times than grids based on standard OpenSim,” Hurvitz said in a blog post.

Cassy also detects duplicate assets, he said. Kitely users frequently upload content from popular OpenSim content sites such as OpenSim Creations and LindaKellie.com — Cassy checks to see whether the content has already been uploaded before, and stores just one copy.

The Linda Kellie region is a popular upload for Kitely.

“We also improved how viewers download textures,” said Hurvitz. “Textures are the largest and most common type of asset, and viewers spend a lot of time downloading them from OpenSim… To lighten the load, we now serve textures from web servers running Apache. When a viewer requests a texture OpenSim redirects it to Apache, where it can download the texture. Apache is much more efficient than OpenSim at serving files so this makes the server run more smoothly, and viewers can display worlds faster.”

Cassy is also used to speed up script loading, he said.

“This means that in most cases script-heavy worlds that would take minutes for all the scripts they include to start running in standard OpenSim will take seconds to start running in Kitely,” he said. “Scripts still need to be compiled when a newly imported OAR file is first started, which may result in slower startup times, but after the scripts have been compiled worlds should now become fully functional much faster than they did previously.”

Kitely regions are stored away when not in use, and loaded up only when needed. A major goal for Kitely is to reduce region load times to the point where users can teleport in and not notice any delays.

Other grids don’t have the same problem since regions are always up and running. But if Kitely is able to reduce region startup times far enough, other grids may also begin looking at on-demand regions.

The old OpenSim asset database isn’t completely gone, however, said Tochner.

“Assets themselves are no longer stored in the database but other things are, such as some metadata about assets,” he said. “We could have moved that to S3 as well but it wasn’t worth the current effort to do so. Metadata is tiny compared to the assets themselves so even a single server can handle orders of magnitude more load when it just has to deal with metadata.”

Kitely also used the upgrade as an opportunity to roll out a couple of other features, including a fix for terrains showing up striped, enabling physics for meshes to eliminate invisible boundaries around mesh objects, and increasing the size limit for uploaded OAR region files from 250 MB to 1 GB.

Maria Korolov
Latest posts by Maria Korolov (see all)