Free database for OpenSim scripts

Late last summer, I had an idea for a hypergate script that would automatically calculate the distance to your destination’s address and, if it was too far away, send you to a mid-way transit gate on Hyperica, and then that gate would send you on to your destination.

However, this required a way for gates to communicate with one another, even if they were located on different grids. Now, there is a way for objects to send messages to one another via LSL, but it requires knowing the object’s HTTP address — and this HTTP address can change.  So I needed a way for the hypergates to look up the HTTP addresses of the mid-point transit gates, in order to find out whether they were available, and to tell them the final destination the avatar wants to go to.

I needed a permanent way to save data.

Many scripts in OpenSim need to store variables, someplace where they won’t disappear.

Say, for example, you have a house that changes colors. And you want it to remember the color it was set to last, so you don’t have to keep setting it each time you rezz the house. You can add the color — say, “blue,” to the description line, but if you change the description, you lose the data.

If you had some permanent way to store the variable, then you could, say, change the color of that same house everywhere — even on other grids — when you change any one instance of it.

Okay, maybe that’s not so useful, but you could also use the same method to, say, send messages across grids, or update the location of hypergates, like what I needed — or set high scores for a particular game.

Earlier, I used a Google Spreadsheet to store data. Here’s my sample script for logging visitors to a Google Spreadsheet, and another one for displaying the data from a Google Spreadsheet.

The downside to using Google Spreadsheets is that you can only add rows of data — you can’t edit existing rows. Unless you can figure out the Google API and authentication mechanism, which I couldn’t. If anyone can, and can write a sample script to edit existing rows of data, please please let me know! And you also can’t search through data — you can only load the entire sheet and then search through it inside the script. And there’s only so much text you can load at once. I assume — I don’t know what the limits are here.

So I was looking for a way to save data into some database somewhere — ideally, a free database. A way to update that data. And a way to find that data.

I turned to Latif Khalifa’s Grid URL Persister for inspiration. It uses a Python script running on the Google App Engine, a free, cloud-based place to deploy applications and databases. Well, free for small projects — if you’re running something big and commercial on it, you’d have to pay some dough. But the limits on the free account are pretty good — you get ten free applications, 1 gigabytes for data storage, and 3,000 free calls a minute — more than half a million a day.

So I’ve been tinkering around with the Google App Engine ever since then, going through their tutorials, setting up the launcher on my desktop, deploying their sample Guestbook application, trying to figure out what the Go language was.

Finally, early this week, I decided to buckle down and just get it done. Based on advice from InWorldz CTO David Daeschler and InWorldz software developer Jim Tarber, I switched from Go to Python, the language Khalifa’s application was written in.

And … ta da! … here it is: the Hypergrid Business Easy Free Database.

Here’s the way it works.

First, you need to get a unique identifier for your LSL script. You can use a different one for each script, or, if you want to share data across several scripts, you can use the same identifier for all of them. You can get an identifier here. Don’t lose this — if you do, you will lose all your data!

Second, load the following URL to save data:<service id>&keyfield=<name for the data>&datafield=<data to be stored>

The bold items are the ones you replace your own own data. Say, for example, you want to save high scores for two games, Bingo, and Blackjack. You get a new service ID, and invent key names for the data — BingoHighScore and BlackjackHighScore. Store the data by going to the following URLs:

Later on, you want to get these high scores back. You can look them up again by pulling up the following URLs:

Go ahead, give it a try.

If you also want to save the avatars who had the high scores, you can create two new key names, such as BingoHighScoreAvatar, and BlackjackHighScoreAvatar.

There might be a limit on how long you can make the key names, and how long the data strings are, but I don’t know what it is. If you have key names or data that has spaces or other characters in it, that’s fine, but you will need to encode them for the URL.

So, for example, if “John Smith” got the high score for Bingo, you would save it with the following URL:

Note that the space between “John” and “Smith” is replaced with %20.

You don’t have to just call these URLs from in-world scripts, of course. You can also call them from a website.

Say, for example, you want to be able to control the color of your house from a webpage.

You can create a key name HouseColor and use clickable links to set the house different colors, like this:

Hint: if doing this from a website, use the “target=” link property so that you’re not sending your users to a blank page that just says OK. For example, you can load the “OK” into a small frame on the page.

I understand that many of my readers are really good Javascript programmers, or know MySQL, or whatever, and will ask, “Why not just write a little code to do the same thing, set up a MySQL database on your server?” And the answer is: There are people who want to add functionality to their scripts without having to learn JavaScript or Python or MySQL. Is that too much to ask?

Creating your scripts

To demonstrate the use of this tool — and to find out whether it works! — I created two LSL scripts. One turns a box a random color when it’s touched, and then saves that color. The second script, when touched, looks up the color and sets the box that color.

The box on my right — the pink one — turns a new random color when touched, then saves that color to the database. The box on my left — the orange one — is waiting for me to touch it, When I do, it will go to the database to find out what color it’s supposed to be, and then turn that color. That is, pink.

The LSL script to save the color is here in the form of a text file — copy and paste in-world.

The key lines are these:

string URL = “” + serviceid + “&keyfield=colors&datafield=” + colors;
key httpkey=llHTTPRequest(URL, [] ,””);

 In the first line, serviceid is a string variable, defined at top of the script, as the unique identifier. Change this for your own script. The name of the key field is “colors” and the datafield is the string variable colors, defined just above to hold a string composed of three floating point numbers, separated by commas. Since the string has commas in it, I used the llEscapeURL function so that it would work in a URL.

Once you define all the parts of the URL, you just use the llHTTPRequest function and that’s it. The data is stored into the database. Click on this link to see what’s stored there now:

That’s it. Just two lines to save any data you want into the database. Again, remember — don’t forget the service ID, and don’t forget your key field names!

So how do you get the data back out of the database?

It’s a little trickier — it takes an extra subroutine (or whatever they’re called in LSL).

You can get the script, as a text file, here.

The first part is the same as before — an llHTTPRequest:

string URL = “” + serviceid + “&keyfield=colors”;
key httpkey=llHTTPRequest(URL, [] ,””);

Again, the serviceid variable is defined as a string at the top of the script. Make sure that it matches the serviceid in your other script. Note that the datafield is missing — you’re not storing data here, you’re trying to get it. The other change is that instead of, you’ve got at the start of the URL.

But it’s not enough just to send the command out there — you have to actually get data back. That’s the function of the http_response subroutine:

http_response(key id, integer status, list meta, string body)
list colors = llParseString2List(body,[“,”],[” “]);
llSetColor(<llList2Float(colors,0), llList2Float(colors,1), llList2Float(colors,2)>,ALL_SIDES);

The data comes back into the body variable, in the form of a string. I had to unpack it a little bit to turn it into something I could use — first, parsing the string into a list, then, when I set the box color, turning each of the three elements of the list into a floating-point number.

If the command fails for some reason — the service ID or the key field name are wrong, for example, the body string will hold the error message.

If you want to add error checking to the first script, you can add an http_response section as well. The body string will be “OK” if the data was stored successfully, and will hold an error message if it wasn’t.

Saving more than one variable

In the example above, I save a three-element list. That’s one way of saving multiple data points.

Another option is to use multiple variables — like with the high scores example. But what if you want to save a long list of high scores? A list where you might not know the length of the list ahead of time?

One idea is to save one variable as ListLength, which is the total list of data points you’re saving. As the list gets longer, you can just increment this field. Then have the other variables with names such as ListField1, ListField2, ListField3 … just keep adding numbers to the word “ListField” until you have as many as you need.

Similarly, you can save tables. Have one variable called TableSize, with the number of rows and columns of your table. Then the table data can be saved as TableField1-1, TableField1-2, TableField1-3, TableField2-1, and so on.

Saving list data or table data into individual fields means you can store or load one data point at a time. However, if you need to load the entire list or table at once, you’ll have to send in a new llHTTPRequest for each one. That might take a lot of time. That’s why I saved all three colors as a single list variable in my script, instead of fetching them one at a time. I recommend using lists whenever possible, but there’s a limit to how long a URL can be — I just ran a quick test, and for a keyfield name of “longstring”, the longest data field string I could store was 500 characters even.

Run your own database

Someday, my Easy Free Database will run out of storage. If you’re creating business-critical in-world scripts, you don’t want your stuff to go down.

Sure, you can email me and let me know that the database is out of room, and, if I’m feeling generous, I could pay to add more storage.

Or you can create your own database.

Download my source code from BitBucket and unzip it. Create a free account with Google App Engine. Download and install the Google App Engine Launcher. Open the Launcher, go to File-Add Existing Application, point it to the folder where you unzipped my source code. Then hit the Deploy button and give it the email address and password you set up your Google App Engine account with.

It took me several tries to get this to work, but once it does, you can fiddle around with the application to customize it for your own needs. You can email me if you have questions at [email protected], but keep in mind that, as of this writing, I have exactly three days worth of experience programming in Python.

Again, here’s the code:'

Maria Korolov

Maria Korolov is editor and publisher of Hypergrid Business. She has been a journalist for more than twenty years and has worked for the Chicago Tribune, Reuters, and Computerworld and has reported from over a dozen countries, including Russia and China.

  • WhiteStar

    Great idea for offsite settings data storage. This could certainly have many other applications as well… progress tracking, task tracking etc for simulators which have tasks to complete (think educational).

    One item though… not everyone could or can use an external system such as this, luckily in OpenSimulator we have which allows up to store material on the fly in notecards. Of course this depends on it being enabled & available to use.

    • Maria Korolov

      I was considering using Notecards — except they don’t solve the issue of cross-grid communication — you still have to do something else for that. Plus, I wanted to track how often individual destinations were up. (A lot of the smaller grids are down most of the time.)

  • casias falta

    Whoo, hoo, we’re having fun now!! Very nice Maria

    • Maria Korolov


  • Maria Korolov

    Another thing I wanted to add is a comment about flow control. For people who are expert LSL programmers, this is probably a no-brainer, but I’m shocked each time I’m reminded that LSL programs don’t go step-by-step the way normal programs do (or the way they did 20 years ago — God knows what programs do these days).

    LSL scripts are state-based. That means that if you want something to happen after some bit of information comes back from the database, you’ll need to put in the command to do it in the http_response subroutine, or tell the script to switch to another state at that point.

    This is the main reason I went with saving the three color values as one list, instead of as three separate variables — I’d either need to load in one color value at a time, or send out for all three at once, then keep track of which ones came back. Pain in the ol’ patootie.

    • Serendipity Seraph

      Well, there are methods available to parse json and if you work in a C# script it is pretty straightforward to get a key value map of arbitrary complexity back from some web services that returns json.

  • Serendipity Seraph

    Nice. I would tend to use a NoSQL solution in the cloud with a Rest http interface to store and retrieve the information. But nice idea and kudos on getting it working.

  • A. T.

    wonder why you have archived source code into single RAR instead of leaving it in repo and letting others to clone/fork and play… you don’t play much with GIT otherwise or something else?

    • Maria Korolov

      A.T. — I understand the individual words you used,.. you’re asking something about my choice of code repository? I used Google App Engine because it was free, and because I modeled my app on Latif Khalifa’s, which also uses the Google App Engine. (By “modeled” I mean “ripped off wholesale.” Latif, you’re the greatest!)

      I understand that some people use Github to manage their code development process. I don’t really have a code development process, and I’ve written about Github before, but I’ve never used it.

      • A. T.

        Not exactly about repo server choice, rather you steps you had done vs what you could do.

        _Normally_ people launch project, even tiny one, for weekend or few hours. And while they code from scratch or hack what they had forked from someone else, they push code-change diffs back to own repo. Gradually project growths up not only for them but for public. When they have finished, they do _not_ zip/rar everything and put up there — they left it as is in repo, in source code form. This way it makes easier to follow up past things. It also let others to build more and more in tree-like way, sometimes eventually building significant things from small weekend projects. I hope you’ve got by now what I suggest – even if you hack a bit someone else, please-please-please do it as above :) Archive tarballs are so 80s :) And you’re journalist, you address crowds so it is even more important! Did I say it also raises publicity for you too? And you don’t have to become coder after single time project in this manner :)

        • Maria Korolov

          Sorry, don’t actually know how to do any of that.

          • A. T.

            well, if I read you between lines correct, you’re not exactly interested. apologies.

          • Maria Korolov

            AT — If someone else wants to do that, I’d be happy to link to it.

            Like I said, I’m not a developer, and while I understand — in theory — the value of what you’re suggesting, and it may be quick and easy for a developer to do, I wouldn’t even know where to start.

          • A. T.

            yes, I re-read original post few times and found where had been my mistake in assumptions. if you ever need free help with that – I sent you Linkedin request. otherwise — nevermind.

          • Maria Korolov

            I NEVER turn down free coding help!

            For example, I’d like to turn this database into one that can be used to manage the back end of a navigations system. You would put down a sign in-world, change its object name, and click on it to update it and it will automatically register itself and update all the other destination signs in the same group.

            And then each sign will have its own location at the top, and the other destinations hanging down under it. And if you move a sign, just click on it to update it and all the other signs linked to it (via a common group UUID). And if anyone else clicks on the top sign, nothing happens. And if anyone clicks on any of the other, attached signs, they get teleported to that destination.

            It seems like it would be a very useful script to have — not just for OpenSim, but Second Life users, as well.

          • Maria Korolov

            Except, of course, in OpenSim, you would use the text-on-a-prim OSSL commands to write the names of the locations. I don’t know what you’d do in SL — some kind of text cludge, probably. Maybe store 26 letter textures in the sign, and pull them out as needed.

          • A. T.

            Frankly I lost most of interest in SL after several months helping colleague back at N. I still have some interest in OpenSim mainly because of open source side.

            I gather there is some API for SL and I know where is API for OpenSim but… why one would need offline db for signposts? what’s value?

          • Maria Korolov

            So say you have a corporate campus, or an educational campus, or a non-profit campus in OpenSim, one that stretches across and entire region or group of regions, and may include skyboxes, etc…

            You have multiple offices, event venues, classrooms, meeting spaces, etc…

            You want to have a navigation board at the welcome point that takes people to all these space, and navigation boards at each of those spaces to take people to the other spaces.

            Meanwhile, things are always moving around and being rearranged. You add new facilities, and remove old ones. You hold new events. etc… Right now, you have to manually keep track of where all the navigation boards are and each time you make a change, you have to go around to all the boards and update them. As your grid gets bigger, this gets harder and harder to do.

            As a parallel, remember the old days when webpages were hand-coded HTML? If you had a menu at the top of each page, each time you made a change to the one page, you had to update all the other ones. (Then Dreamweaver came out, with their templates, and now we have content management systems that keep track of it for us.)

            Actually, my dream is to eventually create a WordPress-style CMS, but for OpenSim, one that would allow third-party themes but handle all the standard layout and navigation tasks in a uniform way on the back end.

            But — as you might have noticed! — my programming skills date from the 80s, and also I have a day job (for CSO) that takes up all my time, and Hypergrid Business which takes up all the rest of my time. :-) So my dream will have to wait!

          • A. T.

            well, what you describe is more “want” than “need” on surface, but I know there is a need because I remember laborious and mundane management of LSL assets back then for N company, I’ll spare you off painful details. I wanted to see your take on it – I got it, thanks.

            for opensim, there had been attempt to establish CMS – two items mentioned here at bottom but it hadn’t survived: coincidentally I had been looking into its history in parallel with starting this thread (so that was my buzz about “save the source, Luke”). I found author despite thorough cleanup. My current perception that he abandoned it due to lack of interest in outer opensim world – he is happily doing daily job now. This is why I asked about need/interest in CMS: we might perceive it as needed yet opensim/lsl-ers do not create necessary demand to justify it as large project.

            As for your comment above, I do suggest to put it into “product description / request” on opensim wiki -. who knows, maybe someone would perceive it as starting point for weekend project or two 😉 If you need help or second opinion with that, feel free to ping – you have my contacts now. I’m currently in short streak project but intense one till end of August at least. So I can’t promise anything significant. Plus I believe web interface for opensim would be too much 2D to tolerate. Maybe WebGL interface would be great, but afaik it is still far from perfect. Or maybe API/hooks into LSL/opensource VW browser. Personally I cease to invest my time into LSL, only into opensim time to time. That’s it for now.