This month we get deep into the tech jungle to talk about scripting and content development updates.
First of all, thank you to everyone who came out for our new US time slot for our monthly Q&A! Now let's dive right into what we talked about this month.
The Blue viewer is going out now (Tuesday night / Wednesday morning depending on your time zone). By the time you're reading this it should already be available for download at: https://sine.space/download
As for what is in the Blue viewer, our strategy is to try to hinge every release on a single large or important feature and then fill it in with any bug fixes or quality-of-life improvements that are ready to ship at the same time. The 'important feature' in Blue is supporting the ability for users to be in multiple voice channels simultaneously. This feature is unfortunately not going to be available to Worlds grids initially, although we will be reviewing whether it makes sense to enable it. If you have thoughts on how this feature might be useful to you on the public grid or a Worlds grid, please feel free to let us know via F5 in your viewer (can be used for feedback, not just bug reports!) or Discord.
The rest of what is in Blue is helpfully covered by today's questions, so let's go down the list:
⏹ When will cross-grid teleporting be available for Worlds grids without having to log out and back in?
⏺ Today! This is one of the new features of the Blue release. Attempting to teleport to a region in another grid via Events, scripting (such as portals), or other means should now work. The same access rules apply to teleporting as they do to logging in. That means if you are blocked from a World grid, you won't be able to teleport either! If you're teleporting to a region on a private World, you will need to have been granted access permission by the administrators of that grid.
⏹ Will the next release fix the radio bug?
⏺ Yes! The Blue release today should fix the overlapping streaming bug for the SDK component used by radios.
⏹ Is any work being done on Mac stability? The Mac viewer seems to crash more than the Windows viewer does.
⏺ We are absolutely looking into this. Some work has been done on Blue, but we know we haven't found all of the issues. Unfortunately, the answer to platform-specific issues often lies in third-party libraries that we have little or no control over. We in turn must pursue the third-party vendors who supply those libraries for fixes from them. Third-party plugins include features like embedded web browsers, voice support, and other features we support but did not create. We believe one of the most common Mac crashes we are seeing falls under this third-party umbrella.
If you're wondering why we call it the "Blue" viewer, that is just our internal code name for the release. We give all of our builds color codes to keep track of them internally, the next one is Indigo! We're going to talk to you today about a variety of work-in-progress features, some of which may end up in Indigo, but many of which will need longer timelines. In the land of software development, writing the code and proving it works on one developer's computer is just the tip of the iceberg in what it takes to get a feature into your hands. The testing process can often be longer than the development, and that was true even before AI!
We're currently working out what features we can move into different releases for the rest of the year, and testing resources will likely be the bottleneck on how quickly this happens. We have to be careful and methodical in how we deploy changes for two reasons: The first is that we want the software to work reliably when you want to use it. The second is that deploying a new update effects both Worlds and Breakroom grids at the same time. We're considering doing a hard split between these two services to give us the ability to deploy different versions of the viewer, backend, and website to Worlds at a faster pace, while delivering them to Breakroom on a longer timeline.
So with that out of the way, what are we working on?
Adam has been focused on maximizing what we can get out of AI-assisted development practices and exploring how they can help us revive old feature projects that were never completed or jump on new ones more quickly. This has involved a lot of work in training AI systems to work on our software specifically, including embedding MCPs directly into the viewer code, and creating a way for coding agents to spin up a complete but entirely separate copy of the Sinespace platform + viewer on their own inside a secure container where they can test their own work and iterate on it. This is already yielding some very promising results, but as mentioned above... testing... testing... testing! Agents can help us move faster on the development side, but we never trust what they give us without thorough QA testing just like we would any other code that we're considering putting into production.
⏹ Will we get recurring events on the Worlds Events window so that we don't have to keep re-entering them every week?
⏺ We're actively looking at implementation on this, and this is one of the things AI can likely help us deliver quickly. It is more than a viewer change, however, it is also a backend and database change. On top of that, it is also a policy and UX decision. We want to prevent the calendar from filling up with events that aren't actually happening. It's no good for users to open events and see dozens or hundreds of events and then show up and find out no-one is there, because someone once checked "recurring" and never deleted the event when they stopped running them. Some solutions we're looking at include a maximum time window, such as "will only automatically recur for the next 6 weeks." The exact time window is under discussion. We're also looking at finally adding a "Duplicate Event" button to clone a previous event for a new entry. This will save a lot of time with configuring images, landmarks, descriptions, etc. as you would only need to set the new time slot and save it.
Other things being actively developed in test branches include:
➤ Differential uploads: Right now entire assets get uploaded every single time, including all dependencies (sound effects, materials, videos, images, prefabs, scripts, etc). Not so bad for clothes or furniture, but really tough on whole regions where changing even one tiny thing requires uploading a multi-gigabyte region bundle again. A new feature as of about 3 weeks ago means that the editor pack can now negotiate with the backend as to what files have been changed since last upload. So content being uploaded only needs to upload much smaller pieces after the first upload, only what was actually changed. Allen has implemented this branch on our QA servers and it is being tested now. Also includes upload resuming, so if there's an outage or disconnect then it doesn't have to start over from scratch. This will require a new Editor Pack to use, but it is backwards-compatible with older editor packs that use the monolithic-uploader as well.
➤ Scripting performance: This is a broad topic and encompasses a lot of changes under the same performance umbrella. Lua is an interpreted language, and while the (modified) open-source MoonSharp engine we use to run Lua is quite fast for an interpreted language...it is still interpreted, and that means it will never reach the performance of C# or other compiled languages.
Adam started a project several years ago to use WASM instead. WASM wasn't very mature at the time, so it was not pursued all the way to finish in its initial R&D phase. Other priorities made revisiting it a challenge over the years, but WASM is much better today, and we have scripting running on WASM now on a test branch. We're targeting a compiler chain for C# being up front, but other languages could be done also including C or RUST. It can already compile C# code using the Sinespace API directly, but we're also working on wrapping the entire UnityEngine API to bridge native Unity code to the Space.* APIs automatically. This means you could bring in native Unity scripts and they just work without modification. Some stuff won't be allowed still, either because it's a security risk or a performance bottleneck (such as anything that tries to manipulate the render pipeline through WASM).
This process has also meant expanding the Space.* APIs to cover a broader swathe of the UnityEngine native calls (which also benefits Lua, as it will share the same APIs still). This has increased our API from roughly 1500 functions to over 1900 so far on this test branch.
We're also looking at some smaller performance improvements, including:
⏹ What is the real performance difference between Lua and C# here for user scripts?
⏺ A lot! Most of which is just the natural difference between interpreted and compiled code. C# will probably be 2-3x faster as a general rule, but it's not a uniform performance gain. Some functions will gain more benefit than others, like loops. Loop control code could be potentially hundreds or thousands of times faster.
➤ Asynchronous scripting: A more extensive structural change is moving all user scripts (Lua or WASM) to background threads. Right now Lua scripts run on the main thread and can affect viewer performance (framerate). This is why we have 'tick' limits enforced on scripts, so that if they run too slowly or try to do too much processing on a single frame then we terminate them. Otherwise the scripts can appear to freeze or lock up the viewer, where rendering appears to stop and the viewer appears to be unresponsive, when in reality it is doing something a user script asked it to do that is taking way too long.
Already running on a test branch is a new overhaul that uses preemptive multitasking and cooperative multitasking with synchronization points on each render frame. These background / async threads have a shadow copy of the scene and changes that happen from a script will sync back to the real scene at set points. This synchronization happens so fast you shouldn't ever notice, because it checks for changes every frame. If there are no pending changes for a given frame, then nothing happens and rendering moves on separately from scripts.
This is a potentially breaking architectural change, so it will need a lot of testing! But the upshot is that slow scripts won't impact the feel of the viewer's performance to the user, we can be more lenient about long executions, and scripts that don't need to touch the scene regularly (or at all) can run at much higher speeds completely independently.
⏹ How does the async script know how many frames it has crossed and how does that impact using DeltaTime?
⏺ It is being considered, delta time may reflect elapsed time since the last sync rather than the last frame, or a "script delta time" and a "world delta time" will offer two separate options for managing this.
⏹ Will users have the option to decide whether their scripts run synchronously (like they do now) or asynchronously, so we can benchmark it ourselves?
⏺ Our current plan is "yes", it will be a switch on the runtime. Async will likely become the default, but we expect the option to switch back to a legacy mode to be available.
Some other thing to consider...
➤ Developer workflow: C# workflow will be somewhat different from Lua. There will be a backend compiler that takes C# and compiles it for you as part of the processing architecture already in place. However, we also want to support a local option to compile on your computer and convert it to WASM to run in the editor. We want to emphasize towards using real tools as an option for developers who are already comfortable with a native Unity workflow, such as Visual Studio, VSCode, or other IDE you already like. We don't want to invent our own custom IDE when so many good ones already exist.
➤ Future language support: We've already started playing with some other language ideas, like C. C is safe in this environment, and better than C# in terms of size and efficency, but C# can be much easier to work with. As an example, the same script written in C only uses 460 bytes of memory, while using the nano framework for C# it is about 44kb.
Okay, so that was a lot! If you're still with us... congrats! Also, there's a few more things...
➤ Revising asset loading: One of the things that has always driven us nuts as much as it does you is having to wait an hour and a half to resolve a new upload for a simple change. We hate processing times too! This is another project that was started years ago and is now being revisited. We currently rely on Unity's built-in asset loading system, but it has drawbacks. It's opaque and complicated to migrate between Unity versions to maintain content (which is one of the reasons it takes us a long time to upgrade engine versions). The second problem is that it is out of our control in many circumstances, such as during viewer and region loading, which can create stalls when new content appears. We have no control over how Unity loads this content, we just say 'here it is' and then it is all up to Unity, and the engine doesn't handle realtime loading like this very gracefully.
Several years ago we started building our own asset loader and content system, and we've been able to make great strides on this recently. It helps us with migrating assets because once processed the first time, we don't need to reprocess it ever again. With the format under our control, we only need to upgrade the loader in the future, not the content. We have an alternative loader now working, meaning we can now detach from Unity's builtin loading systems. Shaders are complicated and not currently covered by this, but we have a solution coming for that too. We have a new shader pipeline that is compatible with Unity's own pipeline, but again gives us more control. Part of this is our own shader trimming service so we no longer need to load hundreds of megabytes of standard shader variants at once like Unity does just to get the one variant we actually need, we can be more selective. This is very experimental, just entered first stage of testing last night in fact!
➤ In-world editing: As much as we want to improve the editor pack experience, we also want to make it entirely optional, at least as much as is possible. As important as it is to support the full capability of the Unity editor and a standard Unity workflow as much as we can, we also know that many people aren't comfortable using the Unity editor and it is often overkill for making small changes even for experienced developers. In-world editing is now working on a side branch, including uploading meshes, editing materials, adding new components, etc. While this works in a local dev build right now, it is one of the things that will take considerable testing resources before it is ready to go to production.
This means that once deployed, there is a way mapped out to fully author new content and get it into the store from scratch without ever installing the Unity editor. Some very complex or advanced types of content may still require the full editor experience. One example is cloth physics, which will be more difficult to support in the viewer directly (because of all the painting tools mainly) and may come to in-world editing later.
As part of looking at in-world creation tools, other aspects of the room editor are being upgraded as well. The Inspector and Hierarchy windows are already in that pipeline for improvement, including UX updates and removing clutter where not needed (such as components that have no editable values/parameters) to streamline the editing experience.
➤ Terraformable terrain: Years ago, Adam started a project to create in-world terraformable voxel terrain/water as a feature. This was never completed for various reasons, including shifting priorities for development time and some rather cryptic math bugs in the terrain calculation. This project has been revived though, and the previous problems have been fixed! So this feature joins our testing queue for a future update.
➤ Some other smaller features being looked at include:
⏹ Is this seemingly increased productivity due to AI or just more time to focus?
⏺ Both! AI agents are force multipliers if used cautiously and deliberately. They can accelerate work 10-100x faster than before, but only with all of the correct tooling and guidance in place (discussed above). We still have to solve the bottleneck of testing features before they can go to production, as AI can only be trusted to an extent with self-testing.
⏹ Any news on avatars for NextGen or the upcoming NextGen alpha preview access?
⏺ NextGen is still in progress of course! Everything we talked about above is actually bridging some of the work between the two of them, including the new asset system. A public version (alpha, invite only) is still planned to go up sometime this month. Much of the work being mentioned above helps to improve and prove our content compatbility between CurrentGen and NextGen. Even as we get further along on the tech, it's important to remember that there will be lots of heavy lifting still to productize NextGen above and beyond merely the code existing for it.
⏹ Is there any news on that grant you mentioned before?
⏺ Nope, but that is entirely expected! It's just the nature of the process and we knew that going in. We are unlikely to hear any updates about this until at least September, and closer to December is more likely. And that's if everything stays on schedule on their end! We are optimistic though, it just takes time.
And that's where we're going to leave it for this month! Thank you to everyone who asked thoughtful questions and stayed to listen to this incredibly jargon-dense episode of "Oops, We Gave Adam The Microphone Again!"
Next month we will see you again at our alternating EU-friendly time slot on August 11th at 9am Eastern / 1pm UTC.