Mod Support #2 – Embedding Lua into C++

This is part of a series of posts revolving around mod support, the introduction and links to the other posts can be found here.

Our current game in development, Solitude, is a cooperative multiplayer action survival game. Since its a coop game we use a game server that is bundled with the client to allow players to host their own servers and it’s also used for the single player experience. From the very beginning we wanted to give players the maximum mod support we were able to so we decided both the game client and server would be moddable. As our game server is written in C++ that meant embedding and integrating Lua into C++. I’ll write about integrating Lua into Unity as a separate post.

 

A Faster Lua – LuaJIT

Lua is the scripting language we decided to integrate into our game’s scripting layer for code logic control. The aim is for the scripting layer to control all gameplay logic while the core, C++, layer will control generic non-gameplay game logic and time critical algorithms. To gain the speed increases discussed in the last post we decided to go with LuaJIT, which is a Just-In-Time compiler for Lua.

LuaJIT is distributed as source code so to gain a binary version you need to compile it against your compiler of choice. Our server is developed using the C++ compiler MinGW x64 and since we want it to be as cross platform as possible we use the POSIX threads version. What this means is that the threading library used is cross platform instead of using the default Windows threading library when on Windows.

LuaJIT can have some trouble compiling if you’re using the wrong variation of MinGW. For LuaJIT to compile correctly you need to use the SEH and not the SJLJ variation. SEH and SJLJ are two different types of exception handling systems and if you try using the SJLJ version it just won’t compile at all. After that’s compiled you need to link your C++ project against LuaJIT and run some examples to test everything is working. If you are already using core Lua then it’s a simple drop-in replacement.

 

Bringing Object-Orientation to Lua

Lua is a powerful, fast, lightweight, embeddable scripting language but Lua is not an object-oriented language.

The above is a pretty powerful statement, especially since today so many languages are object-oriented. Ideally we would like to use all the advantages of Lua but still have object-orientation. So… is that possible? Yes, by using Lua’s flexible table data structure and some clever libraries. To help us save time we use a library called classes.lua. It allows you to construct classes in Lua that can be passed around and used in a way that feels natural to anyone used to object-orientation.

For an example, at the top of your intended lua class file you’d add:

Door = classes.class();

Provide a constructor:

function Door:init() 
	-- code
end

and then when you want to create an object from the class you’d call:

local door = classes.class(Door);

and it’s as easy as that. You can then call methods on the object like you would in any object-oriented language but with the syntax of:

door:open();

Without classes.lua you wouldn’t be able to do this in Lua (unless you coded it yourself). By bringing in object-orientation it allows us to better bind our core and scripting layers together and make an easy to use and flexible modding framework. I’d highly recommend you look at classes.lua, or the many other OO libraries, for Lua.

 

Binding Lua and C++: The API

Once you have LuaJIT and object-orientation working in your project you need to consider how you want modders to interact with your game. Generally it’s a very good idea to create a level of abstraction so to make things easier for modders. Make C++ do the heavy lifting where possible and allow it to present the results to the scripting layer when requested or required.

When you want to start coding your API you quickly find yourself asking questions like “How on Earth do I get Lua to understand C++ objects? Can my Lua objects be sent back into C++ for processing?”. Welcome to the world of Lua / C++ binding.

Lua does not understand your C++ classes unless you code it to understand it. Binding, in this example, is the process of exposing the C++ classes to Lua. It allows Lua to understand your C++ code and there are two ways to bind C++ and Lua together.

Manual Binding

Manual binding means you write the entire binding yourself. There is a pretty infamous and painfully sharp post about why you should take this approach instead of automatic binding. You can read it here but to summarise they paint automatic binding in a bad light. I happen to disagree with the post but it does raise some valid points. At the end of the day, manual binding is harder and takes much more time to complete. If you’re a small team there will always be the cost vs. benefit argument. For us, manual bindings didn’t bring any real advantages.

Automatic Binding / Glue

Automatic binding / glue means the binding is automatically created for you. Most binding libraries simplify the process massively and provide various advantages depending on the library used. I won’t go into all the libraries as there are too many out there but each take a slightly different approach to binding. There will be a performance hit for using a library but a lot of libraries take performance very seriously and do their best to keep things as fast as possible.

For Solitude we used an automatic binding library called LuaBridge. After a lot of research and evaluation I found that this worked best for us and was the right balance of sensible syntax, functionality and performance. When using LuaBridge, you link it to your C++ project then register the classes you wish to expose to Lua. For example,

void ScriptAPI::registerEventManager(lua_State* state) {
	getGlobalNamespace(state)
			.beginNamespace("event")
				.beginClass<event::EventManager>("EventManager")
					.addFunction("registerEventHandler", &event::EventManager::lua_registerEventHandler)
					.addFunction("registerEvent", &event::EventManager::lua_registerEvent)
					.addFunction("triggerClientEvent", &event::EventManager::lua_triggerClientEvent)
					.addFunction("triggerEvent", &event::EventManager::lua_triggerEvent)
				.endClass()
			.endNamespace();
}

The above code registers the Solitude C++ EventManager so it’s accessible to Lua. You can pick and choose what methods you expose on the class so it’s flexible if you only want to expose select functionality to Lua. You specify the method names and how you’d like to access them from Lua. The method signature is automatically extrapolated by the library. This extrapolation can cause some issues with method overloading but there are workarounds for that. From Lua, you’d access the EventManager as follows:

-- Door was opened so trigger the door opened event
EventManager:triggerEvent("ON_DOOR_OPENED");

Even from this simple example you can already see how powerful and easy it is to bind C++ and Lua together with LuaBridge. Things can become even more powerful when you expose your custom objects and actually create them in Lua instead of C++, then feed them back into C++ for processing. That’ll be for a later blog post though.

After LuaJIT, classes.lua and LuaBridge are in your project you can slowly create your scripting API. Knowing what should be in the core layer and what should be in the scripting layer takes time as there is not always a clear boundary. Hopefully this helps create a clearer understanding of what it takes to embed Lua into C++. Thanks for reading. Feel free to get in contact with me on the comments, email or grab me on Twitter at @CWolf.

 

Links

Lua – Scripting language
LuaJIT – Just-In-Time version of Lua
classes.lua – Object-orientation library for Lua
LuaBridge – C++/Lua binding library
MinGW-w64 C++ Compiler – C++ compiler

Mod Support #1 – Scripting Languages

This is part of a series of posts revolving around mod support, the introduction and links to the other posts can be found here.

For our current game, we’ve decided to provide as much mod support as possible. One large part that will contribute to this is providing a scripting language both on the server and client.

 

Scripting Language

So… what exactly is a scripting language? A scripting language is a programming language used to control another application. In the case of games, and depending on how they are implemented, they can be embedded and used to control the game logic layer. There are advantages and disadvantages of embedding a scripting language in your game, these are:

 

Good: Mod Support

By pushing as much as possible of the game logic into the scripting layer you allow for a mod community to grow around your game. Over the last few years mod support in games has been very lacking and if you provide mod support you’ll be greatly increasing your game’s shelf life and differentiate yourself from the crowd. Modders can become a very loyal and technically helpful addition to your community. Also, who wouldn’t like to see what other people can do with the world you’ve provided them? You make a game because you love the idea and world (hopefully!). Seeing what others can craft with the tools you give them is a great feeling and can even rekindle your original feelings if you’ve unluckily burnt yourself out.

Good: Fast Development Iterations

When developing, if you can do something faster without sacrificing quality – you’ll want to do that. By pushing the game logic out to the scripting layer you’ll be able to reload the scripts without restarting, or recompiling, the game. This is especially useful for developing artificial intelligence and if you’re working with C++. Recompiling your project for every minor edit is exhausting and, after the thousandth time, can send a developer to the edge of insanity and then past the gates of Oblivion.

Bad: Initial Integration

The initial research, selection, embedding and binding of the scripting language to your game core can take a while and is not the easiest thing to do. It may seem like too much effort for developers who are on a very tight schedule and are not experienced enough to get through the process.

Bad: Slower Processing

Any embedded scripting language will almost certainly run slower than the core language you’re running it from. This can eventually have an effect on the end game system requirements but there are steps that can be taken to mitigate the worst cases. Any heavy duty, performance critical code should be run in the core language and then this can be provided to the scripting language as an API call. By doing this, you’re correctly breaking the core logic and the game logic apart and keeping them in their respective places too.

Bad: Exposure of Source

Whilst some scripting languages allow you to pre-compile your scripts to shield your code from prying eyes – you may actually want to share the uncompiled scripts with your community and modders. Your scripts can be the best teaching tool for how to use your scripting API and provide a great start to mods. This matter really depends on how you view your game logic source code and if you’re willing to provide it for the greater good of your game and the modding community, or not.

Bad: Security

If you expose a way for the community to mod your game you will always get someone who will try and do malicious things with it. You will need to sandbox your scripting language runtime and / or whitelist libraries to prevent this. It’s very hard to maintain a complete sandbox but in the end you have to try your best. No one likes a rogue mod deleting save files… or worse.

 

Prerequisites

After deciding that we want to use a scripting language for the above reasons, and feel the above disadvantages aren’t too bad, we need to decide on what language to use. There are lots of choices out there for game scripting languages. For a good game scripting language a few prerequisites need to be met.

 

Performance

Since we aim to provide extensive mod support we are developing as much of the game logic in the scripting layer whilst keeping any processing intensive code and generic client / server logic in the core layer. Due to this the scripting layer needs to be as fast as possible. This is a very important point as stacked inefficiencies will eventually bite you.

Ease of Use

We want modders to be able to get up and running creating mods as quickly as possible. For this we need a scripting language that won’t take new users too long to get into so the lowest possible barrier to entry is required. My early days of development was modding newly released Fallout 1 and 2 so I’d like to give people an easier ride than what I had to do with installing compilers and editing huge files of identifiers.

Security

Since we plan for mods to be client and server side then we need to ensure nothing terribly bad can happen when running mods. Crashing games from bad mods can’t usually be avoided, however, preventing unrestricted access to the end user’s computer is very important. Either a sandbox or whitelisting of functionality is a must here.

Power / Extensibility

Game logic can become very complex so we want to use a language that is either powerful enough to do what we want or can be extended to do what we want. There is no reason to handicap ourselves.

Easy to Embed

At the end of the day, Rogue Vector is a two person company and adding mod support is a huge undertaking that can equally provide huge benefits. We need a language that will be fast for us to embed without spending months trying to integrate and bind it.

 

Language Choice

With the prereqs all agreed on, we need to select which language will be a good fit for us. There are too many scripting languages for me to go into them all, however, they fall into two main categories: dynamically or statically typed. Here are a few languages and the considerations to take into account.

 

Statically Typed

As a rough generalisation, statically typed languages are high level languages that, at compile time, execute extensive checks to ensure type safety and are bound to a data type and an object (or null). Compiler optimisations are also applied in statically typed languages.

mono

Mono is a .NET framework runtime that supports embedding and running of many other languages, even those not from the .NET family such as Java. While Mono is rather large to include in your game it’s surprisingly nimble when it comes to the benchmarks and code execution. In many of the benchmarks it comes out as the fastest scripting system of them all. The main issue is that to use Mono embedded in a commercial product you have to pay a licence fee. This was a deal breaker for us so we stopped investigating it but it has been used amazingly well by Unity and Second Life as their scripting base.

angelscript

AngelScript is a mature language that is very close to C++ in syntax. It hides and handles pointers and memory automatically so that reduces the complexity, however, for a scripting language to remain so similar to C++ may raise the barrier to entry for modders. On the other side, it allows for developers to swap between the scripting and core language without any worries on having to mentally swap development modes. If you want to have C++ style scripting and your core game is C++ then AngelScript may be your winner here. It has a very clean integration and so that reduces the need for a binding library, which is a big plus. It’s used successfully in games such as SOMA, Amnesia and Overgrowth.

Thanks to Ian Thomas (@wildwinter) from Frictional Games and GameDevWales for feedback on AngelScript.

 

Dynamically Typed

Again as a rough generalisation, dynamically typed languages are high level languages that, at runtime, execute many common programming behaviors that statically typed languages perform during compilation. They are not bound to a data type but only to the object (or null).

lua

Lua is generally seen as the games industry scripting language of choice. It’s very lightweight, fast and very portable. Lua becomes even faster when using the JIT compiler called LuaJIT. While it’s only a procedural language it does have a very flexible data structure that allow for object oriented programming techniques to be used. Extension libraries can bring that support even closer to full object orientation. The language itself has a few things to get used to, such as indexes starting at 1 and not 0 and missing expected keywords like ‘continue’, but it’s not too bad to get used to. All in all, it’s easy to see why Lua is used frequently for embedded game scripting and has been used in games such as World of Warcraft, Freelancer, Garry’s Mod and Natural Selection 2.

python

Python is another popular procedural language similar to Lua. It’s a lot heavier due to it including all the nuts and bolts that Lua does not. It also runs slower than Lua so this is something to keep in mind. I have an issue with the white space nature of Python but that’s more a personal dislike more than anything else. In the end, Python’s focus is wider than Lua for better or worse. Python has been successfully used for game scripting, like in Civilisation 4 and Battlefield 2, so that speaks for itself.

v8

Google’s V8 Javascript-based scripting language seems to be a faster choice than core Lua, however, it’s slower than LuaJIT according to the sources I’ve found. I haven’t been able to confirm this with hard benchmarks as I don’t have time so I’d just say it’s pretty fast. Binding support seems to be lacking and it’s an uphill struggle to get it embedded and working the way you want, especially if you want to pass complex objects between the core language and V8. I couldn’t find any games that currently use V8 as their scripting language but, on the brightside, a lot of people know Javascript syntax.

 

Our Conclusion

We decided to back Lua (LuaJIT) for our scripting language of choice. The main reasons are already mentioned above but also we needed a scripting language that is consistent between the server and client. While Unity has no native scripting support, there are assets that help integrate LuaJIT in nicely. Other scripting languages would be harder to integrate into our game client. At the end of the day most of the scripting languages highlighted have been used very successfully so personal preference is also a key deciding factor.

For a more indepth look at some of these languages and a set of benchmarks see the blog post at upvoid.com. A wider set of benchmarks are found here and here. My next post will go into more details of LuaJIT embedding and binding into a C/C++ game core.

Thanks for reading. Feel free to get in contact with me on the comments, email or grab me on Twitter at @CWolf.

Mod Support #0 – Introduction

Mod support is a loved, much asked for and difficult to implement side of game development. It can range from small configuration file changes to full blown total conversions. In the more extensible form it takes enough experience with games to extract game logic from core logic, embed scripting languages, load models and animations at runtime and merge it all together as seamlessly as possible.

Coming from a modding background before entering software and games development it’s an area very close to my heart. I feel it’s an amazing tool to help increase your game’s longevity and, more importantly, allow for a community to form around your game. In the best situations this community can create amazing additions or changes to your game that can add new life to a game that, even as the developer of, may rekindle your interests after you’ve long exhausted yourself.

As we’ll be adding mod support to our current game we’ll write about what we use and how we go about using it. As there tends to be little information about the more complex side of things we’ll try to expose these under discussed areas to help more developers planning the same thing.

If anyone wants to get in contact with me feel free to reach me at richard@roguevector.com or my Twitter at @CWolf.

Contents

Unity Development: Monitors

In the spare time between our contract work I’ve been developing technology for our future game. One piece of tech that we feel will feature heavily in the game is interactive monitors and consoles. While going through my thoughts I immediately remembered the strong impression that was made on me when I first played Doom 3 and came across their interactive monitors. I loved how they integrated more functionality into an area of games that was traditionally just an ‘on/off’ switch-style game entity.

I wanted to create something simple and effective similar to what Doom 3 achieved.

Doom3 Monitor

So, with the decision to do some tech prototyping I went about wanting the monitors to be as flexible as possible. It’s still early days but I intend to have lots of different monitor types supported. In my eyes the system should support quick access, full focus in-environment and total immersion monitors. So, here are the initial set of types I am dwelling over:

Panels – Quick access monitors like ‘Open’ panels near to doors, lifts or single use machines. How Doom 3 did it.
Environment Monitors – The player’s view is zoomed in and locked onto it. It will still show part of the environment around it.
Fullscreen Monitors – Monitors that go into full screen mode.
Security Cams – Full screen, limited movement angles and possibly screen effects
Avatar Monitors – Full screen, full movement control (for turrets or robots/probes)

So… with those types in mind I went to create a prototype. You can see my efforts below in the Youtube video.

As you can see, it’s very early days but this proves the concept pretty well. Now… how is this implemented? As we’re using Unity Pro this is where a Pro specific feature steps in – RenderTexture. It allows a camera to render, or draw, its view to a texture. This texture then can be used on, for example, a Plane. By setting up a scene where the cubes can be viewed; the Plane can show this. By doing some raycasting on the Plane, then continuing the raycast from the intended camera – you can detect hits on the objects and then trigger the interaction.

In general this approach works well but I still have a few design choices to chew on. Firstly, while the camera and RenderTexture approach works really well for ‘real’ views such as Avatar or Security camera mode, since it’s looking at something in the ‘world’, it gets a little cloudy for the monitors and consoles. Where should the ‘view’ exist in the game scene for a monitor with just buttons / user interface options / typical computer interfaces and feedback? The two main trains of thought on this type of thing are:

Same coordinates as the monitor but on another layer for culling
This keeps the ‘view’ positionally relevant to the monitor and is easy to remember where the view is. The problem is that it feels messy to me. You can enable / disable layers in Unity so it doesn’t need to be messy but it just feels a little wrong to me, however, the ‘view’ in-game will be culled so you won’t ever see the actual ‘view’ – just the result on the monitor.

Same Position Example

Very far away from the active game world
This keeps the ‘views’ away from any active area but our game will make use of a very large in-scene area. While our game will use a very large in-scene area, ultimately, there will always be space for it somewhere far out. We’ll be loading world data in and out as the player(s) move. I’m not too comfortable with this approach though as the ‘views’ will be a little dislocated from the monitor – it feels wrong to me too.

views

The game scene is the building area of the game. The active area is what we would load game objects into. This leaves some ‘no mans land’ which we could add the monitor ‘views’ to.

I’ll have to give this issue a good bit of thought. I’m going to see if there are other options available to me that I’ve not thought up yet. I’m more inclined to go with the first option but we’ll see. One very interesting side effect I encountered has been getting an effect similar to Portal’s portals. By leaving a mouse controller active on the security camera monitor when it should be turned off gives a very similar effect to what it’s like to look through a portal.

We’ll see how the tech progresses – thanks for reading!

Game Released: Mechadroids – Closed Alpha

Robots Fighting

War… war sometimes changes. Tonight we released Mechadroids to the closed alpha testers! We’ve been playing the game during the evening; we’re stealing credits while attacking each other with some insane weaponry.

Alpha testers will receive an email with details on how to play when they have been accepted. We are having some small issues with our emails being sent into spam folders so please check there before taking out your rubbish! If you’re a tester please log feedback at Deadworld Feedback. If you have any problems with your login or our websites please email us at info@deadworldstudios.co.uk.

The next few weeks will be used to balance Mechadroids ready for our big release onto the Android Market. Once there anyone with an Android phone will be able to download and play Mechadroids.

Remember for more information visit the Mechadroids information page and the Mechadroids site! Good hunting alphas!

Game Announcement: Mechadroids

So, finally after all the secretive, nondescript tweets regarding our hard work we’d like to announce a new game – Mechadroids.

Mechadroid

Mechadroids is a fast paced, tongue-in-cheek game that will be released on Android, Facebook and the Mechadroids site. Initially, we are only releasing the Android version but we’ll be branching into the other platforms soon.

Before we hit the Android Market we’re going to have a closed alpha testing period. This will help us iron out any bugs and tweek the game. This period will last a few weeks then we will release Mechadroids onto the market as an open beta.

Help us perfect this game by going to the Mechadroids alpha registration page and registering to join the Android alpha test. Please also visit our forum to share any thoughts you have about the game.

For those who are wondering what our plans are regarding Root Access, you don’t need to worry. Mechadroids is a small side project for us that we really wanted to make before we continued development on Root Access, which we consider a much larger project.

All Work and Some Play

Profit and Loss Sign

I’ve been pretty productive tonight. I’ve clocked in some programming, some design and a small website tweak. Now for the past hour I’ve been looking though the financial side of things. This side of setting up and running a company will be one of the harder things to handle in Deadworld. As I’ve had ever so slightly more experience in this field than James we decided that I’d be the poor soul to tackle it (not without the help of James though… or I think a little of me would break down inside). So tonight I’ve been trying to figure out cashflow and profit/loss record keeping.

While things are confusing now I know I’ll grasp it sooner or later. I know for one thing that it’ll be very important to have our transactions well recorded. We’re helped there with Google Checkout and Paypal to a certain level as they can print out nice reports for us on a monthly basis. What I’m more concerned about is the non-third party system transactions – the ones that interact directly with our bank account. Each one will need to be labelled for identification and recorded as such for later. It’ll take a bit of practice but we’ll get there.

Now for a game of Project Zomboid before bed!

Interfacing with UI #4 – Coherent UI

February 5th, 2016

This is part of a series of posts revolving around user interface design and development, the introduction and links to the other posts can be found here. Last I wrote about user interfaces I discussed the new Unity UI system and I wrote about our process of porting from Daikon Forge to it. That was a year and a half ago and a lot has changed since then. To keep things interesting we decided to move from Unity UI (yet another move?!) to Coherent UI and I’ll explain why we did it. Why Move… Again?!... (read more)

@RogueVec: And so it begins! #RebootDevelop

19/04/2018 @ 8:05am UTC

@RogueVec: We'll be at @RebootDevelop this year. We can't wait! If you want to hang out just give us a shout! #RebootDevelop2018 #GameDev

16/04/2018 @ 12:06pm UTC

@SolitudeGame: Fullscreen terminals allow you to hook into your ship's guns for fine control! Moddable gun modules, terminals and UI! https://t.co/B5N01jrA70 #GameDev

8/12/2017 @ 4:58pm UTC

@CWolf: Woo! And, now we have a cross-compiled (nix --> win64) @SolitudeGame server in our build and deploy pipeline #RV #GameDev

28/11/2017 @ 3:39pm UTC

@CWolf: CI love #GameDev #RV

21/11/2017 @ 4:06pm UTC