Blog Entries
THE MYTH OF PRIM LAG Tags: prim prims lag

    One of claims I hear from time to time... both from users and server companies... is that prims cause "lag".   The more prims you have, the more lag they create.  There is also the concept of prims taking more server memory.

    While prims do take up a bit of RAM*, the claim that prim count causes lag is a myth.    We know this because we tested this claim years ago, on Second Life and put an end to that myth forever-- or so we had hoped.  Some monsters just won't die, even when you cut off their heads.

 

CORRELATION is not CAUSATION

    Now to be fair, there is a correlation between prim count and lag.   It would appear that sims with low prim count have little lag, whereas sims with high prim count have a lot of lag. 

    But as statisticians and math majors know, correlation does not equal causation.   Quite often correlation is just a symptom, whereas the disease is something else entirely.  This is the case with prim count.

   

THE ULTIMATE TEST

    About a decade ago when the "Lag Monster" hit SL seemingly out of nowhere, we techs started running lag tests.  What was the source of the lag?   Was it "user content" as Linden Lab claimed?  Was it "too many prims"?   Was it "too many textures"?  Too many scripts?  If it was any of those things, then why had sims with identical setup worked just fine a week before and now they weren't?     Users wanted answers.  (The answer was that LL had made some major server changes without informing their customers... but that's another story entirely.)

    A friend had just bought a new region and before building anything on it, invited me to spend a day running whatever lag tests I wanted to run.   I accepted his invitation.

    The tests were simple:  we would rez prims in various quantities and configurations and run metered "lag tests" checked both by scripts and avatar experience (mainly, turning in place, walking around and flying and seeing if we noticed any difference at all).   We had half a dozen people present serving as observers to verify experience.  

    First we walked and flew around an empty region.   We checked the sim stats, recorded sim stats, got a baseline to check against.

    Second, we rezzed 1,000 cubes and placed them all around us.   Ran the same tests.   No change in baselines.

    So we duplicated that 1,000 cubes until they numbered 5,000.   Ran the tests again.   No change in baselines.

    Linden Lab allows 15k prims on a sim so we couldn't go past that.  To be totally fair and not push the server to max, we rezzed 12,000 cubes and spread them around.   Ran the tests again.   All observers reported the same thing:   Absolutely no change in baseline stats or avatar experience. 

    Twelve thousand prims-- zero server or client impact.

    Okay, there was the "prims cause lag" myth blown out the window.

    But to be fair, let's start from scratch and do it differently.   So we rezzed cubes, spheres, cones and cylinders to see if prim shape made any difference.   Ran the tests.   No change in baselines.

    Several years later an Opensim team decided to test prim lag on their system.  This was a good test to run since they were trying to stabilize their platform by eliminating as many causes of server lag as possible.   They rezzed 140,000 prims at ground level.   Result:   Zero discernible lag.   This validated all the tests we'd run years earlier, but on a factor almost 12x greater than what we'd run.

SCRIPTS.  IT MUST BE SCRIPTS

    Okay so if it's not prims, what would cause lag?  Scripts, right?  Linden Lab told us:   the more scripts, the greater the lag.   That made sense.   But there are two kinds of scripts:  active and inactive.   Active scripts are constantly changing things, doing something.   Inactive scripts just sit, waiting to do something (sit / touch scripts) or have already done something (texture animation scripts).   Some inactive scripts can be run once and then removed-- with their effect still on the prim.   For example, you can animate a texture and then remove the script.  The texture will continue to animate.  A non-animated sit-scripted prim, once the script is run and removed, will still continue to seat avatars properly.

    But Linden Lab claimed that all scripts cause lag, because the sim server had to check every script every cycle to see what was going on.   Okay, let's test that out.   We removed 5,000 of the prims and created a prim that contained a touch-based sit script (inactive).   Then we replicated that prim to a count of 5,000.    Now we had 5,000 prims with trigger-happy touch-scripts in them, just waiting for someone to click them... an amount the company claimed was "far too many scripts on a sim".

    You have probably already guessed the outcome:   zero change in sim baselines.  The entire region still ran as if it were totally empty.   How about that.  Inactive scripts have zero perceptible server or viewer impact.  No lag.

 

OKAY, SO WHAT WAS CAUSING THE LAG?

    If prim count, prim type and simple script count wasn't causing lag, what was?    It turned out the primary lag issue was due to internal SL server issues-- which we discovered by a very extensive Elf Clan experiment that revealed LL was stacking full regions on single cores "by accident".    We asked a dozen people to check all 800 regions on SL (the sim count at the time) and found out that "accident" happened a LOT... and was the primary source of region lag at the time.  It has been so ever since.

    However in case of modern day lag issues, with companies that do not pull such shenanigans (at least, not without properly informing the customer, such as with Inworldz legit 2x2 regions)... lag is caused by several identifiable issues, depending on the grid involved.

    Active scripts can indeed cause lag to an extreme amount.  In fact on SL it was possible for one badly-designed script to completely crash a sim.   Inworldz took care of this by allowing scripts only so much "server room".  While script response might lag if the scripts are poorly written-- they would be unlikely to  impact the region server itself.  That was a smart move for Inworldz.

    On all grids a major problem is texture handling.   The more textures there are, the more textures the client has to load when entering a region.   Add to that poor texture processing where many grids load the same textures over and over again.   This is the result of a badly-designed cache system (the software that stores textures on your hard drive and in RAM for instant access) and buggy loaded-texture tracking.   That is a major issue, because this is in part server but mostly viewer based-- and grids tend to use the same viewers.  (I am not sure at this time how the official Inworldz viewer handles textures; I haven't tested it.  I have tested Firestorm and the problem is still present in that viewer.)

    Avatars cause incredible lag.  Take every factor in the book (textures, prims, active scripts in AO devices) and put them all on a moving, multi-jointed avatar, and you are going to have server and client impact.   The more avatars there are, the more lag.  Pretty much everyone is aware of this. No huge surprise there.

    Arguably (depending on the surroundings), Avatars are the single most laggy factor on any grid.  You can take a nice, peaceful, content-rich, fast-performing region and drag it down significantly by adding 20 avatars.  Add 50 to 60 avatars and the region can go to borderline crash-status.  Yup, avatars lag.  We all know this.

    Server issues.    From time to time there are server-related issues that cause lag; the best thing to do about that is to reset your region daily, or at least once a week.  Server software can get confused, start dragging its heels and needs refreshed from time to time.   This is simply nature of the beast.

    But some server issues could be fixed-- teleporting for example.   On OpenSim grids, an avatar teleporting into a region can bring the entire region to a standstill until the avatar has finished porting.  You can imagine the impact this has on a busy nexus.

    Flexys cause considerable lag.   A classic test was when we examined three types of avatars:

    1. A huge armored avatar consisting of a solid 1,500 prims.

    2. A standard female avatar with flexi hair and dress.

    3. A dwagon, 401 prims.

    Checks were done to see the impact each of these avatars had on a standard sim.  Surprisingly the dwagon caused least impact, with an avatar "lag factor" of 1.0 (by our scale, starting on a scale of 0, a plain avatar with no attachments).   Second place was the armored avatar, which caused a lag factor of 2.5.   (No surprise there... direct prims-on-a-moving-avatar correlation).    But the big surprise was the standard human avatar with flexi hair and dress.  Nothing special, just standard attire.   A whopping lag factor of 6.5.  That one avatar took double the system resources of the two other avatars combined. 

    Try telling a paying customer they can't have flexi hair or clothing and see how far you get.   ; )

    In another test we went to a region that had several "content tests" already set up (very interesting region, that).   One of the most interesting tests was their "flexi ring".   They had twelve flexi "blankets" all hung out in a ring.  When any blanket was clicked they all either turned not-flexi (standard prims) or flexi (standard prims with flex), blowing in the wind.  The result was astounding.  When the flexi on those twelve items was turned off, the region ran fine.  When the flexi was turned on the region lagged significantly.  Walking became difficult.  Turning in spot became more difficult.  Turn off the flexi blankets, stability restored, no lag.

    So if you have a region that seems to lag and there are lots of flexi tree branches and flora and flags and other items... turn off the flexis.  They are seldom worth the incredible impact they have on a region.

 

LIMITATION OF PRIMS ON A REGION

    If you ever set up an Opensim server you will discover something interesting:   in the instructions they recommend setting maximum prim count to 200,000 per region.  What?  Are they insane?  

    Well no.  As we've seen simple prim count has little if any impact on the server or client.   You can set up 45,000 prims on a region (as on Inworldz) and still have plenty of server RAM left for sim performance.  You can set the prim count to 200,000 and so long as you keep scripting and textures to a minimum, no problem.

    Bottom line:  the number of prims allowed on a server is pretty much irrelevant.  What is relevant is what you do with them.

    The reason smart grids like Inworldz limit prim count is because they know there is a correlation between prim count and server performance.   Why?  Because the more prims people use, the more active scripts they use, they more textures they use, the more flexis they use.   So limiting prim count is a sensible way to keep things within server stress limits.   45K seems a "happy medium"... allowing far more prims than the sim owner is likely to need while at the same time keeping away from the edge of the cliff.

 

BUILD SMART, BUILD WISE

    If you visit ElvenSong region on Inworldz, you will find a prim-rich environment.   You will also notice (once the sim loads), there is very little "lag".   This is especially the case if you go into high sky and visit Replicant City.   This prim-heavy, script-heavy, texture-heavy area manifests hardly any  lag at all, if any (once you give it time to load).  Why is this?   How is it such a creation can have almost no lag?

    1.  Smart scripting.   The scripts are written to be inactive where possible.   The fewer active scripts you have, the lower the sim impact.   Replicant City is highly interactive and almost everything is scripted, but the majority of scripts have to be triggered either by touch or contact.   Sensor scripts are very limited.   Active scripts are very limited.  There are no scripted bots.  (Scripted bots can be a major source of lag if not used sensibly.)

    2. Smart texturing.   There are thousands of textures at Replicant City... but they're all hidden from simultaneous view.   They're down corridors and around corners and inside closed buildings.  To "see" them you have to enter the area where the textures are.   So for the most part you're never loading or seeing more than about 200 textures at any one time.   This greatly limits texture impact on visitor experience.

    3. Almost no flexis.  Flexis were limited to an absolute minimum.  Almost no flexis = almost no flexi lag.  Simple fix.

 

    That's the skinny on lag and prim count.   Prim count does not cause region lag.  Zero prims or 140,000 prims...  they're not a source of "lag".

     Watch your scripting (use active scripts as little as possible-- especially scripted bots), how you display textures, and above all watch how many flexis you use.   Keep these down and you'll significantly reduce or even eliminate lag on your region.

--o--

* SERVER RAM.  While prim count does require server memory, most server systems are set up so that they have more than enough RAM to run a server, regardless of prim count.   A 2-gig server (standard size) can handle an entire region full of prims and still have lots of room left over.   In fact it can handle multiple regions-- to greater or less effect depending on the number of regions.    So any grid company claiming that they can only offer 15K prims because of server limitations is full of baloney-- or cutting their server costs so low they're really not giving their customers what they're paying for.  

    Inworldz offers 45k prims-- and could easily provide more except for the prim-count / lag correlation (along with prim count tends to come scripts and textures and flexis etc-- items that can cause severe lag).   45K prims is a reasonable and customer-friendly prim allowance.

RSS

This website is powered by Spruz