Ok so i’m writing a game and its engine in c++, i have decided to use python3 as scripting to give an easily editable way to allow addition and modification events or changing the behavior of different parts of the game. This is done through the Python C api, and bindings i have written.
So now i’m left with deciding how many python modules i should package with the game, to allow for modders to use to build on top of what i leave behind.
So, if anyone has any thoughts on what to or not to include let me know?
I’m a supreme codelet, but if the intention is to make things as open as possible to modders, wouldn’t the obvious answer be, “All of them”?
Rephrasing might clarify the issue (again, I’m a codelet): what are the downsides to packaging modules?
Three downsides, all because there are a LOT of Python libraries out there:
They would take up a lot of space, maybe 100s of GB
Users would have a hard time finding what they want unless the game also comes with a library search feature, which shouldn’t be necessary
For most purposes there are a few libraries ready for prime-time and many which aren’t. Packaging the latter puts burden of library selection on the user.
So, yeah. Every library shipped with the game needs to pull its own weight in usefulness to players.
All is not a good answer, there are ones that reasonably are just not needed, and since my game is auto loading python files and running them i don’t need someone being cheeky and making a mod that is a virus written in python that then my game runs. For instanced im considering removing one of the basic ones os, not sure i need to open the door on forking the process, killing other processes, and or changing sifting through the filesystem.
BTW, your usecase (embedding a scripting language in a C or C++ game) is what Lua was designed for. It should have less friction on your end to update/add engine hooks, run much faster (seriously, compare the benchmarks!), and have a smaller download size.
Disadvantages are that on average Python is more familiar, and Lua comes with a minimal standard library. (But Penlight is a good “standard library part 2”).
Just putting that out there.
And to answer your question, Python already has basically everything I could want for scripting a game. All I can think of is a color library like Colour and maybe an object deboilerplater like Attrs. (But see also Dataclasses).
Gonna have to think on Attrs(and the like), not sure it’s needed don’t really want people creating long term objects in python. Mainly writing logic for how already defined objects in c++ that have bindings interact. The main rub on Attrs is how i’m abusing python. In order to maintain multi-threading (the entire reason for writing the engine in c++ and not python), i have any c++ thread create its own sub interpreter, allowing for multiple python scripts to be running at the same time, without the pesky gil crashing my multi-threading party. The problem this creates is there is no shared python state between threads, and im clearing the interpreters state also after running a script, to set it up for a different script. There is no risk of python only objects being used from two different threads here since they are tied to different interpreters, and the way my bindings are setup, i have the c++ class’ already setup for multi-threaded use.
You could think of a script being a definition to a function, and the engine would load the python file containing it and run it. then clean up the interpreter state. Basically say there is a player function walk() that will be a pyhton script, allowing anyone to modify it. without having to write c++ code and recompile the whole game. The engine is taking a clean interpreter wrapping the player and putting it in the threads python interpreter global variables, along with wrapping other c++ objects of relevance and adding them also to the globals, then executing the python script in this setup environment.
Via bound methods you will be able to add to the c++ state, but the python state can be though of as temporary stack state, that will be discarded at the end of the script. Not sure defining many classes will be necessary.
Setting up a fresh interpreter state for each call to script is a very clean design, but doesn’t it threaten to be very slow? On my machine, python3 -c 'exit()' takes 0.052 seconds, or three frames worth of 60fps*.
I can imagine a workaround (basically start the interpreter normally once per game session, copy its memory, and paste that for every interpreter instance afterwards). I don’t know whether that would be faster or by how much.
*: And - forgive me for harping on this - on the same machine, luajit -e 'return' takes .009 seconds, only half a frame.
Let me start with im not sure im going to allow for scripting of any of the frame time dependent code. My game is not real time or 3d so, most scripted actions appear to happen in the background, and as long as they don’t take exceedingly long will not be noticed.
Yea i know this takes a while which is why i don’t do this.
I use a thread pools, so rather than create and destroy threads, and also have to create a new interpreter and destroy it each time i want to launch something in a thread, i rather hand off a function to a thread in the thread pool. Each of these threads in the thread pool creates its own python interpreter when created. To clear the state of any of these thread pools interpreter is simply clearing the global and local variables(releasing memory is an almost instant operation), which is rather straight forward, and fast. Im not exiting the interpreter and making a new one(the real slowdown allocating memory) each time. Also i’m pre compiling the .py scripts at game startup into .pyc byte code, and then reading them into memory, where i can call the a script as a whole and not have to wait for the python interpreter to first compile it.
Bottom line is i have yet to find a compelling reason to fully stop and restart an interpreter. So, i simply don’t do it.
I gave lua a look. Creating bindings for it is no easier than python, and lua is not a truly object oriented language, making python more natural to use once i have the bindings. Also python is easier in my opinion for a non programmer to read and possibly write or modify. I have written both lua and python and much prefer python as a language. Considering the complexity of the player object alone that I want to bind to lua or python to, makes lua a pain in the ass. To both bind in a sensible way, and use. Where python due to being object orientated feels natural.