Difference between revisions of "Building TPT with Meson"

From The Powder Toy
Jump to: navigation, search
(Page is now protected, remove note at the top; also some gRAmMer)
(More of the same)
 
(36 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{Languages|Building TPT with Meson}}
 
{{Languages|Building TPT with Meson}}
  
This is a guide to get you started on coding for The Powder Toy. Before doing anything, make sure you are checked out to the latest commit on the master branch, which this guide is kept up to date with. If you have any questions, just ask in the Development Assistance section on [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 The Powder Toy forums].
+
This is a guide to get you started on coding for The Powder Toy. Before doing anything, make sure you are checked out to the latest commit on the master branch, which this guide is kept up to date with. If you have any questions, just ask in the Development Assistance section on [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 The Powder Toy forums] or in the #development channel of [https://powdertoy.co.uk/Discussions/Thread/View.html?Thread=25871 our discord server].
  
 
= Compiling for Windows =
 
= Compiling for Windows =
  
Tested on Windows 10. The cross-compiling bit at the end was tested on Ubuntu 18.04.
+
== With MSVC ==
  
== Required environment setup ==
+
Tested on Windows 10.
 +
 
 +
MSVC is Microsoft's C/C++ compiler for Windows, which comes with Visual Studio. Using MSVC is the recommended way to build TPT on Windows if you are new to modding or have any reason whatsoever to prefer MSVC over MinGW (for which a separate guide is provided below).
 +
 
 +
=== Required environment setup ===
  
 
* install Git ([https://git-scm.com/ get it here])
 
* install Git ([https://git-scm.com/ get it here])
Line 13: Line 17:
 
* install Python ([https://www.python.org/ get it here])
 
* install Python ([https://www.python.org/ get it here])
 
** it is recommended to allow the installer to add Python to PATH and to disable the path length limit (the latter is offered near the end of the installation process)
 
** it is recommended to allow the installer to add Python to PATH and to disable the path length limit (the latter is offered near the end of the installation process)
* open an elevated command prompt (search for "cmd" the Start Menu, right-click the most sensible result, click "Run as administrator") and execute the following commands (you can close the prompt afterwards)
+
* open an elevated command prompt (search for "cmd" in the Start Menu, right-click the most sensible result, click "Run as administrator") and execute the following commands (you can close the prompt afterwards)
 
  python -m pip install --upgrade pip
 
  python -m pip install --upgrade pip
  pip install --upgrade meson ninja
+
  python -m pip install --upgrade meson ninja
 
* open a non-elevated command prompt (the same as before, but instead of running as administrator, just run it by clicking its shortcut) and execute the following commands (you can close the prompt afterwards)
 
* open a non-elevated command prompt (the same as before, but instead of running as administrator, just run it by clicking its shortcut) and execute the following commands (you can close the prompt afterwards)
 
** replace <code>[wherever you keep your repositories]</code> with the actual path, taking care to remove the <code>[]</code> characters also; if the path has spaces in it, add <code>""</code> around it
 
** replace <code>[wherever you keep your repositories]</code> with the actual path, taking care to remove the <code>[]</code> characters also; if the path has spaces in it, add <code>""</code> around it
Line 22: Line 26:
 
  git clone https://github.com/The-Powder-Toy/The-Powder-Toy
 
  git clone https://github.com/The-Powder-Toy/The-Powder-Toy
  
== Building for the first time with MSVC ==
+
=== Building for the first time ===
  
MSVC is Microsoft's C/C++ compiler for Windows, which comes with Visual Studio. Using MSVC is the recommended way to build TPT on Windows if you are new to modding or have any reason whatsoever to prefer MSVC over MinGW (for which a separate guide is provided below).
+
* install Visual Studio ([https://visualstudio.microsoft.com/vs/community/ get it here]; no, Visual Studio Code is '''not the same''' as Visual Studio, you need Visual Studio)
 
+
** make sure you install at least the "Desktop development with C++" workload
* install Visual Studio ([https://visualstudio.microsoft.com/vs/community/ get it here])
+
* open a (non-elevated) command prompt and execute the following commands
** select the Desktop Development workload
 
** you really only need "MSVC" and "Windows 10 SDK", so you can uncheck everything else in the list on the right
 
* find "x64 Native Tools Command Prompt for VS" (or similar, hereafter referred to as "VS prompt") in the Start Menu and execute the following commands
 
** it is recommended to pin the resulting window to your taskbar; you will be using this a lot to build TPT
 
 
  cd /d [wherever you keep your repositories]
 
  cd /d [wherever you keep your repositories]
 
  cd The-Powder-Toy
 
  cd The-Powder-Toy
  meson build-debug
+
  meson setup build-debug
 
  cd build-debug
 
  cd build-debug
  ninja
+
  meson compile
 +
* make sure Meson is using MSVC (look for <code>cl</code> in the log the setup step prints)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
** if you are not sure, run Ninja again; if it says "no work to do", everything worked fine
+
** if you are not sure, run <code>meson compile</code> again; if it says "no work to do", everything worked fine
* at this point, running TPT from the VS prompt should be possible
+
* at this point, running TPT from the prompt should be possible
 
  powder.exe
 
  powder.exe
  
== Using the Visual Studio IDE ==
+
=== Using the Visual Studio IDE ===
  
The method above does not let you use the 'Visual' part of Visual Studio, the IDE. Although Meson has limited support for this use case, if for some reason you wish to use the IDE, you can ask Meson to generate a build site that uses Visual Studio instead of Ninja.
+
The method above does not let you use the 'Visual' part of Visual Studio, the IDE. Although Meson has limited support for this use case, if [https://learn.microsoft.com/en-us/visualstudio/windows/?view=vs-2022 you know how to use the IDE] and for some reason you wish to use it, you can ask Meson to generate a build site that uses Visual Studio instead of Ninja.
  
* open a VS prompt (see above) and execute the following commands
+
* open a command prompt (see above) and execute the following commands
 
  cd /d [wherever you keep your repositories]
 
  cd /d [wherever you keep your repositories]
 
  cd The-Powder-Toy
 
  cd The-Powder-Toy
  meson --backend=vs -Dbackend_startup_project=powder build-debug-vs
+
  meson setup --backend=vs -Dbackend_startup_project=powder build-debug-vs
* you will not need the VS prompt beyond this point
+
* you will not need the command prompt beyond this point
  
 
This will generate a build site with a Visual Studio solution inside (<code>the-powder-toy.sln</code>). You can use this as any other solution, except for a few key differences:
 
This will generate a build site with a Visual Studio solution inside (<code>the-powder-toy.sln</code>). You can use this as any other solution, except for a few key differences:
  
* the IDE can only be used for more comfortable editing and debugging and not much else, including changing the project structure; you will have to familiarise yourself with Meson and work with the <code>meson.build</code> files in order to do that
+
* the IDE can only be used for more comfortable editing and debugging and not much else, definitely not things such as changing the project structure or compiler flags; you will have to familiarise yourself with Meson and work with the <code>meson.build</code> files in order to do that
 
* once you actually change <code>meson.build</code> files, Meson will automatically regenerate the solution the next time you try to build it; Visual Studio will give you a scary popup about the solution having been changed, which is perfectly normal, just click 'Reload Solution'
 
* once you actually change <code>meson.build</code> files, Meson will automatically regenerate the solution the next time you try to build it; Visual Studio will give you a scary popup about the solution having been changed, which is perfectly normal, just click 'Reload Solution'
* the configuration you currently see (most likely <code>debug|x64</code>, if you are following this guide exactly) is the only configuration the solution has, and it is not recommended to add other configurations as Meson will just overwrite them the next time it regenerates the solution; instead, you have to generate separate build sites with Meson
+
* the configuration you currently see (most likely <code>debug|x64</code>, if you are following this guide exactly) is the only configuration the solution has, and it is not recommended to add other configurations as Meson will just overwrite them the next time it regenerates the solution; instead, you have to generate separate build sites with Meson.
 +
 
 +
'''Important note''': Using this method to build TPT, you may encounter the following errors at link time:
 +
CVTRES : fatal error CVT1100: duplicate resource.  type:MANIFEST, name:1, language:0x0409
 +
LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt
 +
This is caused by a bug in older versions of Meson triggered by [https://github.com/The-Powder-Toy/The-Powder-Toy/commit/18084d5aa0e5544ffce4de514351734aa5f73aed this commit], which adds a manifest resource to the executable. The problem is that VS also tries to add its own default manifest to the executable because Meson does not instruct it otherwise.
  
== Building for the first time with MinGW ==
+
[https://github.com/mesonbuild/meson/pull/12472 This has been fixed in Meson] and works correctly in version 1.4.0. You can upgrade Meson with
 +
python -m pip install --upgrade meson
 +
If for some reason you do not have access to this version, you can either build TPT differently (see the MSVC guide above) or temporarily remove [https://github.com/The-Powder-Toy/The-Powder-Toy/blob/a594ae996d4ed943891115a28069eb728309db05/resources/powder-res.template.rc#L10 this line of code] (making sure to not commit its removal to Git), which is not critical for testing your builds.
  
MinGW is a package that makes it possible to build TPT with GCC even on Windows. '''If you have no prior experience with MinGW, the recommended way to build is with MSVC, see the relevant guide above.''' If you do have prior experience, you probably have your reasons for preferring MinGW over MSVC; this guide makes no effort to list possible ones.
+
== With MinGW ==
  
NOTE: For MinGW, we only provide 64-bit prebuilt libraries. Furthermore, we are not aware of a way MinGW would be able to provide these libraries on its own (e.g. through its own package manager), so using system libraries and targeting 32-bit hosts is not supported by our Meson setup.
+
Tested on Windows 10. This section applies to [https://github.com/The-Powder-Toy/The-Powder-Toy/commit/bc9d43bb103ed0ed9f9f2d1178e057fbbc5780a0 commit bc9d43bb103e] and onward. See [https://powdertoy.co.uk/Wiki/index.php?title=Building_TPT_with_Meson&oldid=9121#Building_for_the_first_time_with_MinGW_.28outdated.29 the previous version of this section] for a guide on how to build earlier revisions.
  
* install MinGW ([https://sourceforge.net/projects/mingw-w64/files/mingw-w64/ get it here])
+
MinGW is a package that makes it possible to build TPT with GCC even on Windows. '''If you have no prior experience with MinGW, the recommended way to build is with MSVC, see the relevant guide above.''' If you do have prior experience, you probably have your reasons for preferring MinGW over MSVC; this guide makes no effort to list possible ones.
** this comes in multiple "flavours", you will need "x86_64-posix-seh"
 
** this guide has been tested with version 8.1.0
 
** you will be given a zip file to extract MinGW to; decide on a directory and make note of it for later, then extract MinGW there
 
*** '''make sure that there are no spaces in the path to the directory you choose;''' MinGW is known to have issues with spaces in paths
 
*** even if you do not plan on doing anything with MinGW right now other than building TPT, you may need to install other flavours later, so it is recommended to install the required flavour to a location that will not interfere with others, e.g. include the version number and the flavour in the path
 
* open a command prompt (search for "cmd" the Start click the most sensible result)
 
** it is recommended to pin the resulting window to your taskbar; you will be using this a lot to build TPT
 
** replace <code>[your MinGW bin directory]</code> with an actual path, taking care to remove the <code>[]</code> characters also
 
*** the "bin directory" will most likely be the path to whatever directory you installed MinGW to, with \bin appended to it
 
** replace <code>[wherever you keep your repositories]</code> with the actual path, taking care to remove the <code>[]</code> characters also; if the path has spaces in it, add <code>""</code> around it
 
set PATH=[your MinGW bin directory];%PATH%
 
cd /d [wherever you keep your repositories]
 
cd The-Powder-Toy
 
meson build-debug
 
cd build-debug
 
ninja
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
 
** if you are not sure, run Ninja again; if it says "no work to do", everything worked fine
 
* at this point, running TPT from the bash prompt should be possible
 
powder.exe
 
* note that this executable depends on the MinGW runtime, which you will have in PATH due to the first command executed in the prompt
 
* a debugging session can be initiated like so ([https://darkdust.net/files/GDB%20Cheat%20Sheet.pdf GDB cheat sheet]):
 
gdb powder.exe
 
  
== Cross-compiling with MinGW ==
+
NOTE: For MinGW, we only provide 64-bit prebuilt libraries.
  
Cross-compiling amounts to telling Meson about the host machine via [https://mesonbuild.com/Cross-compilation.html cross files]. A very basic example is [https://github.com/The-Powder-Toy/The-Powder-Toy/blob/master/cross-examples/mingw.ini available here], which you may need to tweak for your toolchain, but it will likely work out of the box. Consult your toolchain's manual for the exact location of the binaries required.
+
NOTE: We use, recommend, and provide prebuilt libraries built with the flavour of MinGW that comes with the UCRT64 environment of Msys2. Using other flavours, including mixing our prebuilt libraries with such flavours, is something we have no resources to support.
  
NOTE: For MinGW, we only provide 64-bit prebuilt libraries. Furthermore, we are not aware of a way MinGW would be able to provide these libraries on its own (e.g. through the system's package manager), so using system libraries and targeting 32-bit hosts is not supported by our Meson setup.
+
=== Required environment setup ===
  
The workflow is the same as when you build for Linux, with the exception that the cross file to use is specified at configure time:
+
* install Msys2 ([https://www.msys2.org/ get it here])
meson --cross-file=cross-examples/mingw.ini build-debug
+
* open a command prompt (search for "ucrt64" in the Start Menu, click the most sensible result)
 +
** it is recommended to pin the resulting window to your taskbar; you will be using this a lot to build TPT
 +
* make sure your package lists are up to date (e.g. <code>pacman -Sy</code>)
 +
** this step may recommend you to close and re-open the command prompt; do it
 +
* install a C++ compiler (e.g. <code>pacman -S --needed mingw-w64-ucrt-x86_64-gcc</code>)
 +
* install git (e.g. <code>pacman -S --needed git</code>)
 +
* install [https://mesonbuild.com/ Meson] (should automatically install [https://ninja-build.org/ Ninja]) (e.g. <code>pacman -S --needed mingw-w64-ucrt-x86_64-meson</code>)
 +
* optionally, install ccache (e.g. <code>pacman -S --needed mingw-w64-ucrt-x86_64-ccache</code>)
 +
* install a library utility (e.g. <code>pacman -S --needed mingw-w64-ucrt-x86_64-pkgconf</code>)
 +
* install required libraries (e.g. <code>pacman -S --needed mingw-w64-ucrt-x86_64-{bzip2,luajit,jsoncpp,curl,SDL2,libpng,fftw}</code>)
 +
** if for some reason you cannot or do not want to use luajit, install your lua package of choice instead and set the <code>lua</code> build option accordingly; see [[Building TPT with Meson#Build options|Build options]]
 +
* download the source code (e.g. <code>git clone https://github.com/The-Powder-Toy/The-Powder-Toy</code>)
  
There is currently no supported way to use system libraries discovered via the toolchain's pkg-config or similar; the current setup downloads our prebuilt libraries the same way it does when using MinGW on Windows. Feel free to experiment though.
+
=== Building for the first time ===
  
NOTE: It is possible to not only compile but also run the resulting binary on your host machine, but while our Meson setup copies the DLLs for the prebuilt libraries into the build site, in dynamic mode (e.g. <code>-Dstatic=none</code>, which is the default), there may be additional DLLs that need to be copied or symlinked in for the binary to work, depending on your toolchain and the way you run the binary. Example, which may or may not translate well to your setup:
+
* <code>cd</code> to the repository root and execute the following commands
ln -s /usr/x86_64-w64-mingw32/bin/libwinpthread-1.dll
+
meson setup build-debug
ln -s /usr/x86_64-w64-mingw32/bin/libstdc++-6.dll
+
cd build-debug
  ln -s /usr/x86_64-w64-mingw32/bin/libgcc_s_seh-1.dll
+
meson compile
 +
* make sure Meson is using MinGW (look for <code>mingw-w64-ucrt-x86_64-c++</code> in the log the setup step prints)
 +
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
 +
** if you are not sure, run <code>meson compile</code> again; if it says "no work to do", everything worked fine
 +
* '''if you get something along the lines of <code>meson.build:110:2: ERROR: Include dir /usr/include does not exist</code>, or if Meson otherwise complains about bzip2, see [[Building TPT with Meson#Elusive bzip2|Elusive bzip2]]'''
 +
* at this point, running TPT from the prompt should be possible
 +
  ./powder
 +
* note that this executable depends on the UCRT64 runtime and is very likely to only work properly when run the way described above, from the terminal
  
 
= Compiling for MacOS =
 
= Compiling for MacOS =
  
Tested on MacOS 10.15 and 11.5.
+
Tested on MacOS 10.15, 11.5, and 12.5.
  
 
== Required environment setup ==
 
== Required environment setup ==
Line 112: Line 115:
 
* install Homebrew ([https://brew.sh/ get it here])
 
* install Homebrew ([https://brew.sh/ get it here])
 
* in a terminal, execute the following commands
 
* in a terminal, execute the following commands
** if for some reason you cannot or do not want to use luajit (e.g. because homebrew does not have prebuilt binaries for your architecture), install your lua package of choice instead and set the <code>lua</code> build option accordingly; see [[Building TPT with Meson#Build options|Build options]]
+
  brew install pkg-config luajit fftw sdl2 meson ninja bzip2 jsoncpp libpng
  brew install pkg-config luajit fftw sdl2 meson ninja
 
 
  cd [wherever you keep your repositories]
 
  cd [wherever you keep your repositories]
 
  git clone https://github.com/The-Powder-Toy/The-Powder-Toy
 
  git clone https://github.com/The-Powder-Toy/The-Powder-Toy
Line 120: Line 122:
  
 
* <code>cd</code> to the repository root and execute the following commands
 
* <code>cd</code> to the repository root and execute the following commands
  meson build-debug
+
  meson setup build-debug
 
  cd build-debug
 
  cd build-debug
  ninja
+
  meson compile
 +
* make sure Meson is using Homebrew's Clang (look for <code>clang</code> in the log the setup step prints)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
** if you are not sure, run Ninja again; if it says "no work to do", everything worked fine
+
** if you are not sure, run <code>meson compile</code> again; if it says "no work to do", everything worked fine
 +
* if you get something along the lines of <code>meson.build:110:2: ERROR: Include dir /usr/include does not exist</code>, or if Meson otherwise complains about bzip2, see [[Building TPT with Meson#Elusive bzip2|Elusive bzip2]]
 
* at this point, running TPT from the prompt should be possible
 
* at this point, running TPT from the prompt should be possible
 
  ./powder
 
  ./powder
Line 138: Line 142:
 
* install a C++ compiler (e.g. <code>sudo apt install g++</code>)
 
* install a C++ compiler (e.g. <code>sudo apt install g++</code>)
 
* install git (e.g. <code>sudo apt install git</code>)
 
* install git (e.g. <code>sudo apt install git</code>)
* install python (e.g. <code>sudo apt install python3</code>)
+
* install python (e.g. <code>sudo apt install python</code>)
 
* install [https://mesonbuild.com/ Meson] (e.g. <code>sudo apt install meson</code>)
 
* install [https://mesonbuild.com/ Meson] (e.g. <code>sudo apt install meson</code>)
* install [https://ninja-build.org/ Ninja] (e.g. <code>sudo apt install ninja-build</code>)
+
* install [https://ninja-build.org/ Ninja] (e.g. <code>sudo apt install ninja</code>)
 
* optionally, install ccache (e.g. <code>sudo apt install ccache</code>)
 
* optionally, install ccache (e.g. <code>sudo apt install ccache</code>)
 
* install a library utility (e.g. <code>sudo apt install pkg-config</code>)
 
* install a library utility (e.g. <code>sudo apt install pkg-config</code>)
* install required libraries (e.g. <code>sudo apt install libluajit-5.1-dev libcurl4-openssl-dev libfftw3-dev zlib1g-dev libsdl2-dev</code>)
+
* install required libraries (e.g. <code>sudo apt install libluajit-5.1-dev libcurl4-openssl-dev libssl-dev libfftw3-dev zlib1g-dev libsdl2-dev libbz2-dev libjsoncpp-dev libpng-dev</code>)
 
** if for some reason you cannot or do not want to use luajit, install your lua package of choice instead and set the <code>lua</code> build option accordingly; see [[Building TPT with Meson#Build options|Build options]]
 
** if for some reason you cannot or do not want to use luajit, install your lua package of choice instead and set the <code>lua</code> build option accordingly; see [[Building TPT with Meson#Build options|Build options]]
 
* download the source code (e.g. <code>git clone https://github.com/The-Powder-Toy/The-Powder-Toy</code>)
 
* download the source code (e.g. <code>git clone https://github.com/The-Powder-Toy/The-Powder-Toy</code>)
Line 150: Line 154:
  
 
* <code>cd</code> to the repository root and execute the following commands
 
* <code>cd</code> to the repository root and execute the following commands
  meson build-debug
+
  meson setup build-debug
 
  cd build-debug
 
  cd build-debug
  ninja
+
  meson compile
 +
* make sure Meson is using the toolchain you expect it to (read log the setup step prints)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
 
* you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum] instead)
** if you are not sure, run Ninja again; if it says "no work to do", everything worked fine
+
** if you are not sure, run <code>meson compile</code> again; if it says "no work to do", everything worked fine
 +
* if you get something along the lines of <code>meson.build:110:2: ERROR: Include dir /usr/include does not exist</code>, or if Meson otherwise complains about bzip2, see [[Building TPT with Meson#Elusive bzip2|Elusive bzip2]]
 
* at this point, running TPT from the prompt should be possible
 
* at this point, running TPT from the prompt should be possible
 
  ./powder
 
  ./powder
Line 162: Line 168:
 
== Updating the binary ==
 
== Updating the binary ==
  
Now that you have successfully built TPT for the first time, you will probably want to change things in the code. To update the binary, build it again by executing
+
Now that you have successfully built TPT for the first time, you will probably want to change things in the code. To update the binary so that it reflects these changes, build it again by executing
  ninja
+
  meson compile
 
in the same prompt you executed it the last time. You will have to do this every time you change something and want the binary to reflect the change.
 
in the same prompt you executed it the last time. You will have to do this every time you change something and want the binary to reflect the change.
  
== Release builds ==
+
== Optimized builds ==
  
So far, you have been building "debug" binaries. These make debugging significantly easier than "release" builds, but they also offer less performance. You can instead build a "release" binary, which is much more performant, but very difficult to debug. Only use these builds when you are certain that you have no more bugs to take care of. If a bug does appear later on, go back to building debug binaries.
+
So far, you have been building "debug" binaries. These make debugging significantly easier than "optimized" builds, but they also offer less performance. You can instead build an "optimized" binary, which is much more performant, but very difficult to debug. Only use these builds when you are certain that you have no more bugs to take care of. If a bug does appear later on, go back to building debug binaries.
  
To build a release binary, navigate back to the root of the repository in your prompt and create a new build site with Meson, then build with Ninja again
+
To build an optimized binary, navigate back to the root of the repository in your prompt and create a new build site with Meson, then build again
 
  cd [whatever parameters that take you to your repositories, see above]
 
  cd [whatever parameters that take you to your repositories, see above]
 
  cd The-Powder-Toy
 
  cd The-Powder-Toy
  meson -Dbuildtype=release build-release
+
  meson setup -Dbuildtype=debugoptimized build-optimized
  cd build-release
+
  cd build-optimized
  ninja
+
  meson compile
  
Note that build-debug and build-release directories are independent "build sites", each of which you can update with Ninja when you set them as your current directory (by using <code>cd</code>, see the lists of commands above).
+
Note that build-debug and build-optimized directories are independent "build sites", each of which you can update with <code>meson compile</code> when you set them as your current directory (by using <code>cd</code>, see the lists of commands above).
  
== Static builds ==
+
== Release builds ==
  
Both the debug and release configurations above produce "dynamic" binaries, which means they depend on certain components of your system (libraries you have installed with apt on linux or with brew on macos) or files other than powder.exe (DLLs in the same directory as powder.exe on windows). '''If you are a mod author, please make sure you are not shipping such binaries,''' as your users are used to TPT not depending on anything and thus are unlikely to have experience with installing these libraries on their own.
+
Both the debug and optimized configurations above produce "dynamic" binaries, which means they depend on certain components of your system (libraries you have installed with apt on linux or with brew on macos) or files other than powder.exe (DLLs in the same directory as powder.exe on windows). '''If you are a mod author, please make sure you are not shipping such binaries,''' as your users are used to TPT not depending on anything and thus are unlikely to have experience with installing these libraries on their own.
  
 
You can get rid of most of these dependencies by building a "static" binary. This way, the binary will depend on very basic components of your system that are very likely to be present. It will also gain a few megabytes and linking it will take longer, which is why you would only do this when you are about to release it to the public.
 
You can get rid of most of these dependencies by building a "static" binary. This way, the binary will depend on very basic components of your system that are very likely to be present. It will also gain a few megabytes and linking it will take longer, which is why you would only do this when you are about to release it to the public.
  
To build a static binary, navigate back to the root of the repository in your prompt and create a new build site with Meson (see NOTE below!), then build with Ninja again
+
To build a static release binary, navigate back to the root of the repository in your prompt and create a new build site with Meson (see NOTE below!), then build again
  meson -Dbuildtype=release -Dstatic=prebuilt build-release-static
+
  meson setup -Dbuildtype=release -Dstatic=prebuilt build-release
  cd build-release-static
+
  cd build-release
  ninja
+
  meson compile
 +
 
 +
The "release" buildtype is identical to the "debugoptimized" buildtype, but it produces binaries without debug info. Debug info in static builds can take hundreds of megabytes, because it includes all the debug info from statically linked libraries. Unless you want to separate debug info later for some reason, it is best to get rid of it at compile time. Note that while debugoptimized builds are "only" very difficult to debug, release builds are virtually impossible.
  
 
NOTE: On Windows, if using MSVC, you will also need to add the <code>-Db_vscrt=static_from_buildtype</code> option to the Meson call above, otherwise the resulting binaries will be dynamically linked against the MSVC runtime (vcruntime.dll and similar).
 
NOTE: On Windows, if using MSVC, you will also need to add the <code>-Db_vscrt=static_from_buildtype</code> option to the Meson call above, otherwise the resulting binaries will be dynamically linked against the MSVC runtime (vcruntime.dll and similar).
  meson -Dbuildtype=release -Dstatic=prebuilt -Db_vscrt=static_from_buildtype build-release-static
+
  meson setup -Dbuildtype=release -Dstatic=prebuilt -Db_vscrt=static_from_buildtype build-release
  cd build-release-static
+
  cd build-release
  ninja
+
  meson compile
  
NOTE: On Windows, if using MinGW, <code>-Db_lto=true</code> is known to break static builds; use this setting at your own risk. The Meson configuration will throw an error upon detecting this setting; you will have to fix this on your own.
+
NOTE: On Windows, if using MinGW, you will also need to add the <code>-Dcpp_link_args="['-static','-static-libgcc','-static-libstdc++']"</code> option to the Meson call above, otherwise the resulting binaries will be dynamically linked against the GCC runtime (libgcc_s_seh-1.dll and similar).
 +
meson setup -Dbuildtype=release -Dstatic=prebuilt -Dcpp_link_args="['-static','-static-libgcc','-static-libstdc++']" build-release
 +
cd build-release
 +
meson compile
  
NOTE: On Windows, if using MinGW, even static builds will not be "fully static", as they will still import functions from msvcrt.dll. This is fine though, as this library is present on every modern install of Windows.
+
Note though that even such static builds will not be "fully static", as they will still import functions from msvcrt.dll. This is fine though, as this library is present on every modern install of Windows.
 +
 
 +
NOTE: On Windows, if using MinGW, <code>-Db_lto=true</code> is known to break static builds; use this setting at your own risk. The Meson configuration will throw a warning upon detecting this setting.
  
 
== It used to work, but it broke ==
 
== It used to work, but it broke ==
Line 203: Line 216:
 
Each platform has its own set of quirks that need to be worked around (mostly looking at you, microsoft and apple). Fortunately, our build system takes care of most of these. Unfortunately, the workarounds come in the form of upgrades to the build system, which are not installed automatically, and even if they are, affected build sites may need to be reconfigured manually. If you had at some point gotten TPT to build successfully, but since then your build sites somehow broke, you can try one of the following methods to fix them.
 
Each platform has its own set of quirks that need to be worked around (mostly looking at you, microsoft and apple). Fortunately, our build system takes care of most of these. Unfortunately, the workarounds come in the form of upgrades to the build system, which are not installed automatically, and even if they are, affected build sites may need to be reconfigured manually. If you had at some point gotten TPT to build successfully, but since then your build sites somehow broke, you can try one of the following methods to fix them.
  
The most straightforward way to fix a single build site is to reconfigure it. To check if this worked, try running Ninja.
+
The most straightforward way to fix a single build site is to reconfigure it. To check if this worked, try running <code>meson compile</code>.
 
  cd [whatever parameters that take you to your repositories, see above]
 
  cd [whatever parameters that take you to your repositories, see above]
 
  cd The-Powder-Toy
 
  cd The-Powder-Toy
  meson --reconfigure --wipe build-debug # or whatever you named your build site
+
  meson setup --reconfigure --wipe build-debug # or whatever you named your build site
 
  cd build-debug
 
  cd build-debug
  ninja
+
  meson compile
 
You will have to do this with every broken build site.
 
You will have to do this with every broken build site.
  
If this does not help, you can upgrade pip, Meson and Ninja. '''Like when setting them up for the first time, add <code>sudo -H</code> on Linux and MacOS, and run this in an elevated command prompt on Windows.'''
+
If this does not help, you can upgrade pip, Meson and Ninja. On Linux and MacOS, this is done through the package manager. On Windows, run this in an elevated command prompt:
 
  python -m pip install --upgrade pip # if you get "No module named pip", try python3
 
  python -m pip install --upgrade pip # if you get "No module named pip", try python3
  pip install --upgrade meson ninja
+
  python -m pip install --upgrade meson ninja
 
Once you are done with this, reconfigure your build sites (see above).
 
Once you are done with this, reconfigure your build sites (see above).
  
Line 219: Line 232:
  
 
As a last resort, you can ask us [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum].
 
As a last resort, you can ask us [https://powdertoy.co.uk/Discussions/Categories/Topics.html?Category=5 on the forum].
 +
 +
= Troubleshooting =
 +
 +
== Meson not recognized as a command on Windows ==
 +
 +
If you have installed Python and Meson but when you try to run <code>meson</code>, Windows tells you that it is not a recognized command, you probably did not allow Python to add itself to your PATH environment variable. [https://www.makeuseof.com/python-windows-path/ This guide] will help you fix that. Once you are done with the steps there, make sure to restart every instance of the command prompt before trying to run <code>meson</code> again.
 +
 +
== Elusive bzip2 ==
 +
 +
If you use a package ecosystem whose bzip2 package does not provide a pkg-config file (sadly, this is the norm; this includes the package ecosystems of most Linux distributions, Homebrew for all platforms, and Msys2), you may need to add <code>-Dworkaround_elusive_bzip2=true</code> to Meson calls above. With this setting enabled, our Meson configuration will try to look for bzip2 at a few hardcoded locations, which should be enough for build site configuration to succeed.
 +
 +
If you still get errors, the hardcoded paths were probably wrong, and you will have to specify the correct include directory (<code>workaround_elusive_bzip2_include_dir</code> option), library path (<code>workaround_elusive_bzip2_lib_dir</code> option), and static property (whether you want to link against dynamic or static bzip2, <code>workaround_elusive_bzip2_static</code>); see [[Building TPT with Meson#Build options|Build options]].
 +
 +
Example elusive bzip2 settings for Homebrew on M1 MacOS: <code>-Dworkaround_elusive_bzip2=true -Dworkaround_elusive_bzip2_lib_dir=/opt/homebrew/Cellar/bzip2/1.0.8/lib -Dworkaround_elusive_bzip2_include_dir=/opt/homebrew/Cellar/bzip2/1.0.8/include -Dworkaround_elusive_bzip2_static=true</code>
 +
 +
Example elusive bzip2 settings for Msys2 UCRT64: <code>-Dworkaround_elusive_bzip2=true -Dworkaround_elusive_bzip2_include_dir=/ucrt64/include -Dworkaround_elusive_bzip2_lib_dir=/ucrt64/lib</code>
  
 
= Build options =
 
= Build options =
  
 
Build options are passed to Meson at when you set up a build site (see examples above) or at any later time, when the build site is already in use. In both cases, they are passed as command line arguments of the form <code>-D[name]=[value]</code>. For example, to set up a build site that compiles TPT with Lua 5.1 instead of LuaJIT, you would issue the following command
 
Build options are passed to Meson at when you set up a build site (see examples above) or at any later time, when the build site is already in use. In both cases, they are passed as command line arguments of the form <code>-D[name]=[value]</code>. For example, to set up a build site that compiles TPT with Lua 5.1 instead of LuaJIT, you would issue the following command
  meson -Dlua=lua5.1 build-debug-lua5.1
+
  meson setup -Dlua=lua5.1 build-debug-lua5.1
 
But you could also take an existing build site and change the relevant build option to do the same
 
But you could also take an existing build site and change the relevant build option to do the same
 
  cd build-debug
 
  cd build-debug
 
  meson configure -Dlua=lua5.1
 
  meson configure -Dlua=lua5.1
The next time you run Ninja, everything that needs to be rebuilt as a result of this change will be rebuilt. See [https://mesonbuild.com/Quick-guide.html the Meson quick guide] for more on configuring build sites.
+
The next time you run <code>meson compile</code>, everything that needs to be rebuilt as a result of this change will be rebuilt. See [https://mesonbuild.com/Quick-guide.html the Meson quick guide] for more on configuring build sites.
  
 
Below is a list of build options our Meson setup supports
 
Below is a list of build options our Meson setup supports
  
{| class="wikitable" style="text-align: center; width: 100%;"
+
{| class="wikitable" style="width: 100%;"
 
  ! Name !! Type !! Description
 
  ! Name !! Type !! Description
 
  |-
 
  |-
  | static || one of 'none', 'system', 'prebuilt' || Build statically using libraries present on the system ('system') or using prebuilt libraries official builds use ('prebuilt')
+
  | <code>static</code> || one of <code>none</code>, <code>system</code>, <code>prebuilt</code> || Build statically using libraries present on the system (<code>system</code>) or using prebuilt libraries official builds use (<code>prebuilt</code>)
 +
|-
 +
| <code>beta</code> || boolean || Beta build
 +
|-
 +
| <code>ignore_updates</code> || boolean || Do not show notifications about available updates
 +
|-
 +
| <code>install_check</code> || boolean || Do install check on startup
 +
|-
 +
| <code>http</code> || boolean || Enable HTTP via libcurl
 +
|-
 +
| <code>gravfft</code> || boolean || Enable FFT gravity via libfftw3
 +
|-
 +
| <code>snapshot</code> || boolean || Snapshot build
 +
|-
 +
| <code>snapshot_id</code> || integer || Snapshot ID, only relevant if <code>snapshot</code> is true
 +
|-
 +
| <code>mod_id</code> || integer || Mod ID, used on the [https://powdertoy.co.uk/Discussions/Thread/View.html?Thread=21072 Starcatcher build server]; the build server will compile for all platforms for you and send updates in-game, see jacob1 to get a mod ID
 +
|-
 +
| <code>lua</code> || one of <code>none</code>, <code>lua5.1</code>, <code>lua5.2</code>, <code>luajit</code>, <code>auto</code> || Lua library to use
 +
|-
 +
| <code>x86_sse</code> || one of <code>none</code>, <code>sse</code>, <code>sse2</code>, <code>sse3</code>, <code>auto</code> || Enable SSE (available only on x86)
 +
|-
 +
| <code>build_powder</code> || boolean || Build the game
 +
|-
 +
| <code>build_render</code> || boolean || Build the thumbnail renderer
 +
|-
 +
| <code>build_font</code> || boolean || Build the font editor
 +
|-
 +
| <code>server</code> || string || Simulation server
 
  |-
 
  |-
  | beta || boolean || Beta build
+
  | <code>static_server</code> || string || Static simulation server
 
  |-
 
  |-
  | ignore_updates || boolean || Don't show notifications about available updates
+
  | <code>update_server</code> || string || Update server, only used by snapshots and mods, see <code>snapshot_id</code> and <code>mod_id</code>
 
  |-
 
  |-
  | install_check || boolean || Do install check on startup
+
  | <code>workaround_noncpp_lua</code> || boolean || Allow linking against a non-C++ system Lua
 
  |-
 
  |-
  | http || boolean || Enable HTTP via libcurl
+
  | <code>workaround_elusive_bzip2</code> || boolean || Find bzip2 via the compiler and its built-in library directories, rather than via pkg-config or similar
 
  |-
 
  |-
  | gravfft || boolean || Enable FFT gravity via libfftw3
+
  | <code>workaround_elusive_bzip2_lib_name</code> || boolean || bzip2 library name, see <code>workaround_elusive_bzip2</code>
 
  |-
 
  |-
  | snapshot || boolean || Snapshot build
+
  | <code>workaround_elusive_bzip2_lib_dir</code> || boolean || bzip2 library directory, see <code>workaround_elusive_bzip2</code>
 
  |-
 
  |-
  | snapshot_id || integer || Snapshot ID, only relevant if 'snapshot' is true
+
  | <code>workaround_elusive_bzip2_include_name</code> || boolean || bzip2 header name, see <code>workaround_elusive_bzip2</code>
 
  |-
 
  |-
  | mod_id || integer || Mod ID, used on the [https://starcatcher.us/TPT Starcatcher build server]; the build server will compile for all platforms for you and send updates in-game, see jacob1 to get a mod ID
+
  | <code>workaround_elusive_bzip2_include_dir</code> || boolean || bzip2 header directory, see <code>workaround_elusive_bzip2</code>
 
  |-
 
  |-
  | lua || one of 'none', 'lua5.1', 'lua5.2', 'luajit' || Lua library to use
+
  | <code>workaround_elusive_bzip2_static</code> || boolean || bzip2 static setting, see <code>workaround_elusive_bzip2</code>
 
  |-
 
  |-
  | ssl || currently only 'openssl' || SSL library to use
+
  | <code>tpt_libs_vtag</code> || string || tpt-libs vtag override, only used for tpt-libs development
 
  |-
 
  |-
  | x86_sse || one of 'none', 'sse', 'sse2', 'sse3', 'auto' || Enable SSE (available only on x86)
+
  | <code>android_keystore</code> || string || Path to Java keystore for signing an APK, only used for Android development
 
  |-
 
  |-
  | native || boolean || Build with optimizations specific to the local machine, may not run on other machines, overrides 'x86_sse'
+
  | <code>android_keyalias</code> || string || Signing key alias for signing an APK, only used for Android development
 
  |-
 
  |-
  | ogli || boolean || Enable OpenGL interface rendering (currently defunct)
+
  | <code>app_name</code> || string || App name, used for desktop integration and the window title, change if you work on a mod
 
  |-
 
  |-
  | oglr || boolean || Enable OpenGL particle rendering (currently defunct)
+
  | <code>app_comment</code> || string || App comment, used for desktop integration, change if you work on a mod
 
  |-
 
  |-
  | build_powder || boolean || Build the game
+
  | <code>app_exe</code> || string || App executable name, used for desktop integration, change if you work on a mod
 
  |-
 
  |-
  | build_render || boolean || Build the thumbnail renderer
+
  | <code>app_id</code> || string || App ID, a D-Bus well-known name, used for desktop integration, change if you work on a mod
 
  |-
 
  |-
  | build_font || boolean || Build the font editor
+
  | <code>app_data</code> || string || App data directory name, do not change even if you work on a mod, only if you know what you are doing
 
  |-
 
  |-
  | server || string || Simulation server
+
  | <code>app_vendor</code> || string || App vendor prefix, used for desktop integration, do not change even if you work on a mod, only if you know what you are doing
 
  |-
 
  |-
  | static_server || string || Static simulation server
+
  | <code>enforce_https</code> || boolean || Enforce encrypted HTTP traffic, may be disabled for debugging
 
  |-
 
  |-
  | update_server || string || Update server, only used by snapshots and mods, see 'snapshot_id' and 'mod_id'
+
  | <code>prepare</code> || boolean || Used by ghactions workflows, not useful otherwise
 
  |-
 
  |-
  | workaround_gcc_no_pie || boolean || Pass -no-pie to gcc manually to work around meson's -Db_pie=false doing nothing
+
  | <code>render_icons_with_inkscape</code> || feature || Render icons with Inkscape (inkscape binary needs to be in PATH)
 
  |}
 
  |}

Latest revision as of 21:40, 29 November 2024

Language: English  • 한국어 • 中文

This is a guide to get you started on coding for The Powder Toy. Before doing anything, make sure you are checked out to the latest commit on the master branch, which this guide is kept up to date with. If you have any questions, just ask in the Development Assistance section on The Powder Toy forums or in the #development channel of our discord server.

Compiling for Windows

With MSVC

Tested on Windows 10.

MSVC is Microsoft's C/C++ compiler for Windows, which comes with Visual Studio. Using MSVC is the recommended way to build TPT on Windows if you are new to modding or have any reason whatsoever to prefer MSVC over MinGW (for which a separate guide is provided below).

Required environment setup

  • install Git (get it here)
    • options should be left unchanged
  • install Python (get it here)
    • it is recommended to allow the installer to add Python to PATH and to disable the path length limit (the latter is offered near the end of the installation process)
  • open an elevated command prompt (search for "cmd" in the Start Menu, right-click the most sensible result, click "Run as administrator") and execute the following commands (you can close the prompt afterwards)
python -m pip install --upgrade pip
python -m pip install --upgrade meson ninja
  • open a non-elevated command prompt (the same as before, but instead of running as administrator, just run it by clicking its shortcut) and execute the following commands (you can close the prompt afterwards)
    • replace [wherever you keep your repositories] with the actual path, taking care to remove the [] characters also; if the path has spaces in it, add "" around it
      • example: if your repositories are in C:\Users\Powder Toy Fan\Development, this path will be "C:\Users\Powder Toy Fan\Development"
cd /d [wherever you keep your repositories]
git clone https://github.com/The-Powder-Toy/The-Powder-Toy

Building for the first time

  • install Visual Studio (get it here; no, Visual Studio Code is not the same as Visual Studio, you need Visual Studio)
    • make sure you install at least the "Desktop development with C++" workload
  • open a (non-elevated) command prompt and execute the following commands
cd /d [wherever you keep your repositories]
cd The-Powder-Toy
meson setup build-debug
cd build-debug
meson compile
  • make sure Meson is using MSVC (look for cl in the log the setup step prints)
  • you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it on the forum instead)
    • if you are not sure, run meson compile again; if it says "no work to do", everything worked fine
  • at this point, running TPT from the prompt should be possible
powder.exe

Using the Visual Studio IDE

The method above does not let you use the 'Visual' part of Visual Studio, the IDE. Although Meson has limited support for this use case, if you know how to use the IDE and for some reason you wish to use it, you can ask Meson to generate a build site that uses Visual Studio instead of Ninja.

  • open a command prompt (see above) and execute the following commands
cd /d [wherever you keep your repositories]
cd The-Powder-Toy
meson setup --backend=vs -Dbackend_startup_project=powder build-debug-vs
  • you will not need the command prompt beyond this point

This will generate a build site with a Visual Studio solution inside (the-powder-toy.sln). You can use this as any other solution, except for a few key differences:

  • the IDE can only be used for more comfortable editing and debugging and not much else, definitely not things such as changing the project structure or compiler flags; you will have to familiarise yourself with Meson and work with the meson.build files in order to do that
  • once you actually change meson.build files, Meson will automatically regenerate the solution the next time you try to build it; Visual Studio will give you a scary popup about the solution having been changed, which is perfectly normal, just click 'Reload Solution'
  • the configuration you currently see (most likely debug|x64, if you are following this guide exactly) is the only configuration the solution has, and it is not recommended to add other configurations as Meson will just overwrite them the next time it regenerates the solution; instead, you have to generate separate build sites with Meson.

Important note: Using this method to build TPT, you may encounter the following errors at link time:

CVTRES : fatal error CVT1100: duplicate resource.  type:MANIFEST, name:1, language:0x0409
LINK : fatal error LNK1123: failure during conversion to COFF: file invalid or corrupt

This is caused by a bug in older versions of Meson triggered by this commit, which adds a manifest resource to the executable. The problem is that VS also tries to add its own default manifest to the executable because Meson does not instruct it otherwise.

This has been fixed in Meson and works correctly in version 1.4.0. You can upgrade Meson with

python -m pip install --upgrade meson

If for some reason you do not have access to this version, you can either build TPT differently (see the MSVC guide above) or temporarily remove this line of code (making sure to not commit its removal to Git), which is not critical for testing your builds.

With MinGW

Tested on Windows 10. This section applies to commit bc9d43bb103e and onward. See the previous version of this section for a guide on how to build earlier revisions.

MinGW is a package that makes it possible to build TPT with GCC even on Windows. If you have no prior experience with MinGW, the recommended way to build is with MSVC, see the relevant guide above. If you do have prior experience, you probably have your reasons for preferring MinGW over MSVC; this guide makes no effort to list possible ones.

NOTE: For MinGW, we only provide 64-bit prebuilt libraries.

NOTE: We use, recommend, and provide prebuilt libraries built with the flavour of MinGW that comes with the UCRT64 environment of Msys2. Using other flavours, including mixing our prebuilt libraries with such flavours, is something we have no resources to support.

Required environment setup

  • install Msys2 (get it here)
  • open a command prompt (search for "ucrt64" in the Start Menu, click the most sensible result)
    • it is recommended to pin the resulting window to your taskbar; you will be using this a lot to build TPT
  • make sure your package lists are up to date (e.g. pacman -Sy)
    • this step may recommend you to close and re-open the command prompt; do it
  • install a C++ compiler (e.g. pacman -S --needed mingw-w64-ucrt-x86_64-gcc)
  • install git (e.g. pacman -S --needed git)
  • install Meson (should automatically install Ninja) (e.g. pacman -S --needed mingw-w64-ucrt-x86_64-meson)
  • optionally, install ccache (e.g. pacman -S --needed mingw-w64-ucrt-x86_64-ccache)
  • install a library utility (e.g. pacman -S --needed mingw-w64-ucrt-x86_64-pkgconf)
  • install required libraries (e.g. pacman -S --needed mingw-w64-ucrt-x86_64-{bzip2,luajit,jsoncpp,curl,SDL2,libpng,fftw})
    • if for some reason you cannot or do not want to use luajit, install your lua package of choice instead and set the lua build option accordingly; see Build options
  • download the source code (e.g. git clone https://github.com/The-Powder-Toy/The-Powder-Toy)

Building for the first time

  • cd to the repository root and execute the following commands
meson setup build-debug
cd build-debug
meson compile
  • make sure Meson is using MinGW (look for mingw-w64-ucrt-x86_64-c++ in the log the setup step prints)
  • you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it on the forum instead)
    • if you are not sure, run meson compile again; if it says "no work to do", everything worked fine
  • if you get something along the lines of meson.build:110:2: ERROR: Include dir /usr/include does not exist, or if Meson otherwise complains about bzip2, see Elusive bzip2
  • at this point, running TPT from the prompt should be possible
./powder
  • note that this executable depends on the UCRT64 runtime and is very likely to only work properly when run the way described above, from the terminal

Compiling for MacOS

Tested on MacOS 10.15, 11.5, and 12.5.

Required environment setup

  • install Homebrew (get it here)
  • in a terminal, execute the following commands
brew install pkg-config luajit fftw sdl2 meson ninja bzip2 jsoncpp libpng
cd [wherever you keep your repositories]
git clone https://github.com/The-Powder-Toy/The-Powder-Toy

Building for the first time

  • cd to the repository root and execute the following commands
meson setup build-debug
cd build-debug
meson compile
  • make sure Meson is using Homebrew's Clang (look for clang in the log the setup step prints)
  • you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it on the forum instead)
    • if you are not sure, run meson compile again; if it says "no work to do", everything worked fine
  • if you get something along the lines of meson.build:110:2: ERROR: Include dir /usr/include does not exist, or if Meson otherwise complains about bzip2, see Elusive bzip2
  • at this point, running TPT from the prompt should be possible
./powder

Compiling for Linux

Tested on Ubuntu 20.04 and Raspbian 10.

Required environment setup

  • note that your package ecosystem may have wildly different names for the following packages
  • make sure your package lists are up to date (e.g. sudo apt update)
  • install a C++ compiler (e.g. sudo apt install g++)
  • install git (e.g. sudo apt install git)
  • install python (e.g. sudo apt install python)
  • install Meson (e.g. sudo apt install meson)
  • install Ninja (e.g. sudo apt install ninja)
  • optionally, install ccache (e.g. sudo apt install ccache)
  • install a library utility (e.g. sudo apt install pkg-config)
  • install required libraries (e.g. sudo apt install libluajit-5.1-dev libcurl4-openssl-dev libssl-dev libfftw3-dev zlib1g-dev libsdl2-dev libbz2-dev libjsoncpp-dev libpng-dev)
    • if for some reason you cannot or do not want to use luajit, install your lua package of choice instead and set the lua build option accordingly; see Build options
  • download the source code (e.g. git clone https://github.com/The-Powder-Toy/The-Powder-Toy)

Building for the first time

  • cd to the repository root and execute the following commands
meson setup build-debug
cd build-debug
meson compile
  • make sure Meson is using the toolchain you expect it to (read log the setup step prints)
  • you may see a few warnings throughout all of this, but no errors (if you do at any stage, do not try to skip it; ask us about it on the forum instead)
    • if you are not sure, run meson compile again; if it says "no work to do", everything worked fine
  • if you get something along the lines of meson.build:110:2: ERROR: Include dir /usr/include does not exist, or if Meson otherwise complains about bzip2, see Elusive bzip2
  • at this point, running TPT from the prompt should be possible
./powder

Next steps (all platforms)

Updating the binary

Now that you have successfully built TPT for the first time, you will probably want to change things in the code. To update the binary so that it reflects these changes, build it again by executing

meson compile

in the same prompt you executed it the last time. You will have to do this every time you change something and want the binary to reflect the change.

Optimized builds

So far, you have been building "debug" binaries. These make debugging significantly easier than "optimized" builds, but they also offer less performance. You can instead build an "optimized" binary, which is much more performant, but very difficult to debug. Only use these builds when you are certain that you have no more bugs to take care of. If a bug does appear later on, go back to building debug binaries.

To build an optimized binary, navigate back to the root of the repository in your prompt and create a new build site with Meson, then build again

cd [whatever parameters that take you to your repositories, see above]
cd The-Powder-Toy
meson setup -Dbuildtype=debugoptimized build-optimized
cd build-optimized
meson compile

Note that build-debug and build-optimized directories are independent "build sites", each of which you can update with meson compile when you set them as your current directory (by using cd, see the lists of commands above).

Release builds

Both the debug and optimized configurations above produce "dynamic" binaries, which means they depend on certain components of your system (libraries you have installed with apt on linux or with brew on macos) or files other than powder.exe (DLLs in the same directory as powder.exe on windows). If you are a mod author, please make sure you are not shipping such binaries, as your users are used to TPT not depending on anything and thus are unlikely to have experience with installing these libraries on their own.

You can get rid of most of these dependencies by building a "static" binary. This way, the binary will depend on very basic components of your system that are very likely to be present. It will also gain a few megabytes and linking it will take longer, which is why you would only do this when you are about to release it to the public.

To build a static release binary, navigate back to the root of the repository in your prompt and create a new build site with Meson (see NOTE below!), then build again

meson setup -Dbuildtype=release -Dstatic=prebuilt build-release
cd build-release
meson compile

The "release" buildtype is identical to the "debugoptimized" buildtype, but it produces binaries without debug info. Debug info in static builds can take hundreds of megabytes, because it includes all the debug info from statically linked libraries. Unless you want to separate debug info later for some reason, it is best to get rid of it at compile time. Note that while debugoptimized builds are "only" very difficult to debug, release builds are virtually impossible.

NOTE: On Windows, if using MSVC, you will also need to add the -Db_vscrt=static_from_buildtype option to the Meson call above, otherwise the resulting binaries will be dynamically linked against the MSVC runtime (vcruntime.dll and similar).

meson setup -Dbuildtype=release -Dstatic=prebuilt -Db_vscrt=static_from_buildtype build-release
cd build-release
meson compile

NOTE: On Windows, if using MinGW, you will also need to add the -Dcpp_link_args="['-static','-static-libgcc','-static-libstdc++']" option to the Meson call above, otherwise the resulting binaries will be dynamically linked against the GCC runtime (libgcc_s_seh-1.dll and similar).

meson setup -Dbuildtype=release -Dstatic=prebuilt -Dcpp_link_args="['-static','-static-libgcc','-static-libstdc++']" build-release
cd build-release
meson compile

Note though that even such static builds will not be "fully static", as they will still import functions from msvcrt.dll. This is fine though, as this library is present on every modern install of Windows.

NOTE: On Windows, if using MinGW, -Db_lto=true is known to break static builds; use this setting at your own risk. The Meson configuration will throw a warning upon detecting this setting.

It used to work, but it broke

Each platform has its own set of quirks that need to be worked around (mostly looking at you, microsoft and apple). Fortunately, our build system takes care of most of these. Unfortunately, the workarounds come in the form of upgrades to the build system, which are not installed automatically, and even if they are, affected build sites may need to be reconfigured manually. If you had at some point gotten TPT to build successfully, but since then your build sites somehow broke, you can try one of the following methods to fix them.

The most straightforward way to fix a single build site is to reconfigure it. To check if this worked, try running meson compile.

cd [whatever parameters that take you to your repositories, see above]
cd The-Powder-Toy
meson setup --reconfigure --wipe build-debug # or whatever you named your build site
cd build-debug
meson compile

You will have to do this with every broken build site.

If this does not help, you can upgrade pip, Meson and Ninja. On Linux and MacOS, this is done through the package manager. On Windows, run this in an elevated command prompt:

python -m pip install --upgrade pip # if you get "No module named pip", try python3
python -m pip install --upgrade meson ninja

Once you are done with this, reconfigure your build sites (see above).

If not even this helps, that is your cue to start from scratch, following the guide as if you are building TPT for the first time, except you already have the repository, so you do not need to clone that again. In fact, this is where your changes to TPT are stored, so you should never delete it. Deleting your local clone of the repository never solves any problem, it only results in loss of code. Do not even consider doing it.

As a last resort, you can ask us on the forum.

Troubleshooting

Meson not recognized as a command on Windows

If you have installed Python and Meson but when you try to run meson, Windows tells you that it is not a recognized command, you probably did not allow Python to add itself to your PATH environment variable. This guide will help you fix that. Once you are done with the steps there, make sure to restart every instance of the command prompt before trying to run meson again.

Elusive bzip2

If you use a package ecosystem whose bzip2 package does not provide a pkg-config file (sadly, this is the norm; this includes the package ecosystems of most Linux distributions, Homebrew for all platforms, and Msys2), you may need to add -Dworkaround_elusive_bzip2=true to Meson calls above. With this setting enabled, our Meson configuration will try to look for bzip2 at a few hardcoded locations, which should be enough for build site configuration to succeed.

If you still get errors, the hardcoded paths were probably wrong, and you will have to specify the correct include directory (workaround_elusive_bzip2_include_dir option), library path (workaround_elusive_bzip2_lib_dir option), and static property (whether you want to link against dynamic or static bzip2, workaround_elusive_bzip2_static); see Build options.

Example elusive bzip2 settings for Homebrew on M1 MacOS: -Dworkaround_elusive_bzip2=true -Dworkaround_elusive_bzip2_lib_dir=/opt/homebrew/Cellar/bzip2/1.0.8/lib -Dworkaround_elusive_bzip2_include_dir=/opt/homebrew/Cellar/bzip2/1.0.8/include -Dworkaround_elusive_bzip2_static=true

Example elusive bzip2 settings for Msys2 UCRT64: -Dworkaround_elusive_bzip2=true -Dworkaround_elusive_bzip2_include_dir=/ucrt64/include -Dworkaround_elusive_bzip2_lib_dir=/ucrt64/lib

Build options

Build options are passed to Meson at when you set up a build site (see examples above) or at any later time, when the build site is already in use. In both cases, they are passed as command line arguments of the form -D[name]=[value]. For example, to set up a build site that compiles TPT with Lua 5.1 instead of LuaJIT, you would issue the following command

meson setup -Dlua=lua5.1 build-debug-lua5.1

But you could also take an existing build site and change the relevant build option to do the same

cd build-debug
meson configure -Dlua=lua5.1

The next time you run meson compile, everything that needs to be rebuilt as a result of this change will be rebuilt. See the Meson quick guide for more on configuring build sites.

Below is a list of build options our Meson setup supports

Name Type Description
static one of none, system, prebuilt Build statically using libraries present on the system (system) or using prebuilt libraries official builds use (prebuilt)
beta boolean Beta build
ignore_updates boolean Do not show notifications about available updates
install_check boolean Do install check on startup
http boolean Enable HTTP via libcurl
gravfft boolean Enable FFT gravity via libfftw3
snapshot boolean Snapshot build
snapshot_id integer Snapshot ID, only relevant if snapshot is true
mod_id integer Mod ID, used on the Starcatcher build server; the build server will compile for all platforms for you and send updates in-game, see jacob1 to get a mod ID
lua one of none, lua5.1, lua5.2, luajit, auto Lua library to use
x86_sse one of none, sse, sse2, sse3, auto Enable SSE (available only on x86)
build_powder boolean Build the game
build_render boolean Build the thumbnail renderer
build_font boolean Build the font editor
server string Simulation server
static_server string Static simulation server
update_server string Update server, only used by snapshots and mods, see snapshot_id and mod_id
workaround_noncpp_lua boolean Allow linking against a non-C++ system Lua
workaround_elusive_bzip2 boolean Find bzip2 via the compiler and its built-in library directories, rather than via pkg-config or similar
workaround_elusive_bzip2_lib_name boolean bzip2 library name, see workaround_elusive_bzip2
workaround_elusive_bzip2_lib_dir boolean bzip2 library directory, see workaround_elusive_bzip2
workaround_elusive_bzip2_include_name boolean bzip2 header name, see workaround_elusive_bzip2
workaround_elusive_bzip2_include_dir boolean bzip2 header directory, see workaround_elusive_bzip2
workaround_elusive_bzip2_static boolean bzip2 static setting, see workaround_elusive_bzip2
tpt_libs_vtag string tpt-libs vtag override, only used for tpt-libs development
android_keystore string Path to Java keystore for signing an APK, only used for Android development
android_keyalias string Signing key alias for signing an APK, only used for Android development
app_name string App name, used for desktop integration and the window title, change if you work on a mod
app_comment string App comment, used for desktop integration, change if you work on a mod
app_exe string App executable name, used for desktop integration, change if you work on a mod
app_id string App ID, a D-Bus well-known name, used for desktop integration, change if you work on a mod
app_data string App data directory name, do not change even if you work on a mod, only if you know what you are doing
app_vendor string App vendor prefix, used for desktop integration, do not change even if you work on a mod, only if you know what you are doing
enforce_https boolean Enforce encrypted HTTP traffic, may be disabled for debugging
prepare boolean Used by ghactions workflows, not useful otherwise
render_icons_with_inkscape feature Render icons with Inkscape (inkscape binary needs to be in PATH)