In light of several griefing incidents on OpenSim grids over the past few days, grid owners are starting to consider taking proactive steps to protect themselves against future attacks.
Multi-grid nuisance
On FleepGrid, a griefer dropped colored spheres all over the grid, which brought down the entire world.
“It’s taken a little time to figure out how to clean everything up since the griefer objects cause the sims to crash after a few minutes,” said Chris Collins, a project manager in Instructional and Research Computing at the University of Cincinnati. Collins, also known as “Fleep Tuque” in-world, is the owner and founder of FleepGrid.
“I use phpMyAdmin to poke around in the Opensim database, so it was easy to go to the ‘useraccounts’ table and look up the UUID for that user and then delete the items from the prims table,” Collins explained in a detailed blog post about cleaning up after the attack.
According to Kai Ludwig, owner and manager of the Open Neuland grid andCEO of OpenSim hosting company TalentRaspel virtual worlds Ltd., there is a five-step process grid owners — or their hosting companies — should follow if they find themselves in the same situation:
- stop all regions
- select UUID of the griefer from the database
- delete all his prims and primshapes from the database
- lock his account
- start all regions
“The above is a fast and easy procedure to fix the problem and completely works around slowed down or crashed regions or having to use autoreturn — which may not fix the problem when regions have high load,” he said. “We had an attack of the griefer in Neuland on April 10 and fixed the problem with the above procedure within minutes after we got aware of it.”
However, Ludwig warned against giving griefers press, since these children are often just seeking attention. “A grid owners-only mailing list would be much better,” he said.
The same griefer hit the Hyperica grid, but fortunately deleting the objects in-world was sufficient.
In both cases, the griefer teleported in from the CyberWrld grid.
“We have been under attack for weeks lately,” CyberWrld grid founder and CEO Timothy Rogers told Hypergrid Business. The grid has since instituted IP bans, eliminated public building everywhere except on sandboxes, turned on email confirmation for new account registrations, and reached out to residents to help them use the estate access lists.
At the DaseinWorld grid, the attack brought it down for ten hours.
“I made DaseinWorld very accessible in the spirit of friendliness and cooperation,” Â DaseinWorld founder Alexander Duncan told Hypergrid Business. “So a person opened an account, calling themselves Samantha Stick, with a false email address. They entered the world and proceed to create a fairly large number of self-replicating physical prims, which populated the region and the regions immediately adjacent with tens of thousands of physical prims until the world crashed on my PC. My PC itself was not affected. I had to go in and delete them all using parcel return, which took several hours to do. I also had to disable scripts since they were running hundreds of scripts that also slowed DaseinWorld to a crawl. I was barely able to login, even running it on my computer.”
Since then, Duncan has turned on autoreturn and turned off scripting rights, and will be working on setting up group-based authorization.  Meanwhile, the griefer has returned.
“I don’t understand how he can do those things,” Duncan said. “It allowed him to rez 1,448 physical objects, where the limit per sim is set to ten. And then when I try to use ‘Region/Estate’ or ‘Show’ in ‘About Land’ to remove them, they don’t work. Last time, resetting the autoreturn to one minute worked. This time, it didn’t. This guy knows exactly what vulnerabilities to exploit.”
It’s not just the small startup grids that are affected. Griefers have hit even the largest grids, like OSGrid, as well as mid-sized grids.
The 115-region French-language Logicamp grid has been hit by the same griefer who attacked FleepGrid, CyberWrld, DaseinWorld and Hyperica, said grid founder Didier Preud’homme.
“His attack is not very severe and may be evicted by configuring correctly autoreturn objects on the land properties,” Preud’homme told Hypergrid Business. “But I know that it is not always easy to correctly configure each region, especially when you have a lot of regions. Maybe it would be interesting to ask Justin [Clark-Casey, OpenSim core developer] to adjust the automatic return by default to 5 or 10 minutes for new region creation.”
The same griefer attacked Virtual Worlds Grid, said grid founder Myron Curtis, but in a more insidious way.
“The attack was deployed in regions where building and scripting were available only to administrators,” Curtis told Hypergrid Business. “Another attack seems to have destroyed the links between inventory items and their asset entries in the database, and that has been a nightmare. After almost two months, I am finally getting stable performance, but I have not been able to repair the database without losing a significant amount of work my residents have done. I will get it done, but it is going to be the hard way.”
The commercial social world 3rd Rock Grid has seen three major attacks since 2010. The first was where a group of griefers used open land permissions and an OpenSim exploit to destroy and rearrange objects, and create self-replicating prims, The solution was a combination of IP banning, and restoring regions from backup. The second attack used an unpatched web server vulnerability which brought down a region server. Those regions were moved to another server, everything was reinstalled, and the grid saw no downtime. The third attack was simple, using just self-replicating prims, moves and deletes. The solution was IP bans. In response, 3rd Rock Grid also added the ability to instantly find, kick, ban, or IP ban any user right from the grid dashboard.
Self-defense league
Gudule Lapointe, owner of the Speculoos grid, has proposed creating a multi-grid blacklist service — similar to an anti-virus subscription — that would help grids keep out griefers.
“This is getting really annoying,” Lapointe told Hypergrid Business. He suggested a system under which grid owners could voluntarily get together to create and maintain a list of griefers and their identifying information — and institute a process for getting people taken off the list if they are added inadvertently.
“I just hope we can discuss this rapidly,” he said.
CyberWrld’s Rogers is in favor of a multi-grid effort, as well.
“I think something of this nature is a must at this time,” he said. “ It is really hard for us up and coming grids to deal with this with no knowledge of these people’s origins, or how to protect ourselves from their monstrous attacks on out communities. If we could just have a blacklist or warning list viewable only to grid or standalone owners who want access to it, it would be the best buyable solution we could have till someone creates a better module server side to help.”
Rogers is also working on a project to bring grid owners together, called HG Connection.
“I like the idea of an IP blacklist your could subscribe to, created by a trusted source,” said John Lester, chief learning officer at ReactionGrid, Inc., which runs the ReactionGrid OpenSim grid, and also provides hosting for private-label grids. Lester, who is known as Pathfinder in-world, also recommended turning off open building rights.
A simple IP blacklist might not be enough to defend against some extremely determined attackers, however, said Tho Millgrove, co-owner of 3rd Rock Grid.
“We have used IP blacklisting, but it is of limited value against a determined and savvy attacker,” Millgrove told Hypergrid Business. “A shared blacklist might help a bit, but I don’t see huge value in it, as IP addresses are almost throwaways for some.”
Another approach would be to look for suspicious behaviors, she suggested.
“I do like the concept of intrusion detection, perhaps using some sort of pattern detection heuristic,” she said. “For example, we’ve found that attackers tend to follow similar patterns, such as flying from region to region, selecting everything that’s movable, and tossing it several meters up. So, a tool that could detect edit operations in multiple regions over a short period of time might be useful.”
“ Having an intrusion-detection service sounds like a good idea,” added Anthony Gill, founder of the YourSimSpot grid and hosting company. “But it will take some effort to come up with a good one,” , as for IP blacklisting this would just be a minor annoyance for most and can be easily gotten around.
Instead of a blacklist, another option would be to create a whitelist, said Klaus Klingner, founder of The World of Begabungs educational grid.
“An IP blacklist is a problem,” he told Hypergrid Business. “Quite often the attacking servers are hacked themselves and will only be used for a short moment. Once you blacklist a server the attacker will just jump to another system and resume the attack from there. A white list might work better but requires more effort since systems have to be authorized separately.”
Virtual Worlds Grid’s Curtis said that he’s interested in both vendor-driven and community-driven solutions.
“It would help if there were vendors who were dedicated to developing security systems tailored for virtual worlds,” he said. “But intrusion detection, as it is usually defined, and blacklists would be relatively useless unless they were managed by an AI system that was robust enough to recognize and react to attacks. I do believe that can be built. Most grid owners do share security ideas with each other to a limited extent. Expanding on that would be an excellent idea.”
Until then, grid owners are doing their best to battle the issues on their own.
“For the grid, we do have some self-made protection,” said Taisjan Quintin, technical lead at the Dutch OpenSim hosting and development company Tharidos International. The company also runs the public grid Your Alternative Life. “Our grid also has an IP blacklist, and some other options.”
Quintin added that she was interested in finding out more about the attacks, and in offering help where it is wanted and needed.
At Littlefield Grid, new users are required to register and be approved before they can enter the grid, said founder Walter Balazic. The grid also has IP blocking implemented at the firewall level, he told Hypergrid Business. “It is the most logical alternative for us, and if it enhances the safety and well being of the residents I don’t find why that’s an issue.”
Security checklist
Here’s some advice from 3rd Rock Grid about keeping a virtual world safe:
Protect your web server. That means a firewall and intrusion detection. “No new technology needs to be invented here, but its doubtful many grids use such things.  At 3rd Rock Grid, we do some simple filtering, as well as penetration testing of our web servers.  We see script kiddie penetration attacks multiple times a day. In addition, firewalls that protect web servers can also protect both full grids using ROBUST and mini-grids using WIFI against distributed denial of service attacks.
Protect region servers. Regions communicate in two ways — using standard responses on the main ports, and UDP traffic to the viewers. A web server firewall can protect the first kind of traffic. Protecting the UDP traffic would require a dynamic rate-limiting filter.
Protect content. No parcels should allow public building, except for individual sandboxes, and those sandboxes should be located in separate instances of OpenSim, on isolated regions. Â That way, if an attacker hits a sandbox with self-replicated prims, only that one region would be affected.
Set up automated throttling. In the future, OpenSim should add configurable throttling limits — spammers who create large numbers of prims or instant messages using automated tools would see their scripts slow down substantially after they hit a certain point, to keep the region from crashing. Other functions which should be throttled include email, HTTP requests, and similar script commands.
“I think throttling and other control on the individual simulator or grid level does need to improve and patches for that are very welcome,” said OpenSim’s Clark-Casey. “This will probably remain as fairly straightforward stuff. I should think anybody with sophisticated security requirements should look to third party addons or other parts of the network –throttling or blocking on firewalls, etc.”
- International singers gather on Alternate Metaverse Grid for first annual International Day - April 15, 2024
- OpenSim hits new land, user highs - April 15, 2024
- Wolf Territories rolls out speech-to-text to help the hearing impaired - April 15, 2024