Compiling Storm from Source

The pre-compiled binaries provided on this page are enough for most tasks. However, if you wish to integrate external libraries into Storm, or change things in the compiler itself, it is necessary to build Storm from source. Building from source is also an option in case the pre-compiled binaries do not work on your system for some reason.

The remainder of this page describes how to compile Storm on Windows and Linux. The end of the document also outlines how to use Mymake to work conveniently with Storm, and how to integrate the entire flow into Emacs.

Installing Dependencies

The first step is to ensure that all dependencies are installed on your system. Storm is written to require fairly few dependencies, and the dependencies it does require (apart from Mymake) are in many cases already installed on your system. In short, Storm requires Mymake (the build system), a C++ compiler, and a few libraries for graphics and sound on Linux.

Windows

The following software is needed on Windows. You may already have some of this software installed. If so, you do not need to install it again, as Storm does not require a particular version of these tools to build. Storm does not require any additional libraries, so it is enough to install Visual Studio and Mymake.

Git
Git is available for Windows here. Download the installer and run it. By default, a minimal unix-like shell environment is also installed, but it is not required for building Storm. As long as you can execute git commands in a terminal it is enough for the purposes of building Storm.
Visual Studio
For Windows builds, Storm assumes that it is built with the Visual Studio compiler (Clang may work, but it is not tested). The easiest way to install the compiler is to download and install Visual Studio (it might be possible to install just the toolchain). From the linked page, select "Visual Studio Community" and follow the steps in the installer. Note that Visual Studio Code is not enough for the purposes of building Storm, as it does not include a compiler by default. For the purposes of building Storm, all components except for the C++ toolchain are optional, so you may deselect most options if you wish to preserve disk space.

After the installation is complete, it is necessary to start Visual Studio (the IDE) at least once and open a C++ project. Otherwise, it might not install the C++ toolchain. To do this, open Visual Studio. In the welcome dialog, select "Create a new project". In the next dialog, search for "Empty progject" with the language "C++". Next select a location to save the files. The location does not matter, as we are only interested in Visual Studio installing the C++ compiler for us. After Visual Studio has installed the relevant files, you can close Visual Studio and delete the created project.
Mymake
Storm uses mymake as its build system. Mymake is available here. To install it, first clone the repository into some directory:
```
git clone https://github.com/fstromback/mymake
cd mymake
```
Then, run the file called setup.exe in the root (either from the command-line or from the GUI). This program will attempt to find the Visual Studio compiler automatically and use that to compile Mymake from source. If it finds multiple versions of Visual Studio, it will ask you which one you wish to use. Any version later than Visual Studio 2008 is sufficient. If this fails, ensure that you have a "Visual Studio Command Prompt" in the start menu. If not, the C++ compiler has not been installed, and you need to try to open a C++ project in Visual Studio again.

After Mymake has been compiled, the setup asks you if you wish to add it to your Path. This step is not technically required, but it will make it easier to build programs with Mymake in the future.

Finally, ask mymake to create the default configuration file by running:
```
mm --config
```
In a terminal. This process will once again find a Visual Studio compiler to use and set its environment variables up accordingly. During this process, Mymake will ask how many threads you wish to use when compiling programs. It is a good idea to set this number to the number of cores in your machine. If you are unsure about the number of cores in your machine, you can open Task Manager, select the Performance tab, select CPU in the left column and find the value "Cores" listen to the right. If you are unable to find the number of cores, you can also enter an arbitrary number (for example 4 or 8). The only downside of a too low number is that compilation will take longer than necessary.

Linux

On Linux, Storm requires Mymake (the build system), a C++ compiler, Git to fetch the source code, and a few libraries to interact with the system. As on Windows, Storm is not very sensitive about the exact versions that are used, so the ones available for your system will likely work. Note, however, that Storm has only been tested with GCC. It should be possible to build it using Clang as well, but this has not been tested.

Installing Packages
The following packages are required on Linux. For ease of use, the package names used in Debian-based systems are given below. The names are likely to vary slightly depending on your distribution (e.g. you probably do not need the -dev part of the name). To install the packages, the following command may be used:
```
sudo apt install \
  build-essential git \
  g++ \
  libssl-dev \
  libgtk-3-dev \
  libpng-dev libjpeg-dev \
  libopenal-dev \
  libmariadb-dev \
  libgl1-mesa-dev \
  libgles2-mesa-dev
```
If you wish to connect to PostgreSQL databases you also need the library libpq-dev.
Mymake
If you are running Debian, you can install Mymake from the official package repositories (package mymake). Note, however, that the command is named mymake rather than mm in that case.

If you are not on Debian, or wish to compile Mymake from source regardless, the process is as below. Note, that if you wish to integrate the build proces into Emacs, you need the file mm-compile.el from the Mymake repository regardless. That file is not packaged in the Debian package.

First, clone the Mymake repository:
```
git clone https://github.com/fstromback/mymake
cd mymake
```
Then, run the script install.sh. This script will compile Mymake and optionally add an alias for it. Note, this script should not be run as root:
```
./install.sh
```
If you wish to handle the process manually, the script compile.sh only compiles Mymake. It takes a single parameter, the name of the output file. It can be called as follows:
```
./compile.sh mm
```
If you do not wish to add an alias (as the install script does), you can also copy the compiled program (mm) to somewhere in your path.

Regardless of how you installed Mymake, run mm --config to generate the default configuration file in your home directory. As a part of this process, Mymake will ask how many threads it should use when compiling programs. It is a good idea to set this value to the number of cores in your machine (or slightly lower if you are low on RAM). If you are unsure about how many cores your system has, you can run the following command in a terminal:
```
grep -P 'processor\t*:' /proc/cpuinfo | wc -l
```

Downloading the Source Code

The next step is to download the actual source code for Storm. The repository for Storm contains a number of submodules, so it is easiest to use the following command:

git clone --recursive git://storm-lang.org/storm.git

If, at a later time, the submodules become out of sync (for example, if they were updated from upstream), the command git submodule update makes sure the proper version of each submodule is checked out. If a submodule is missing entirely, run git submodule init followed by git submodule update.

As of version 0.16.19 of Storm, cloning the entire repository from scratch downloads around 810 MiB of data and requires around 1.1 GiB of disk space.

Skipping Submodules

If you wish to reduce the amount of data that is downloaded, or wish to reduce the disk space required, you can ignore some of the submodules. If you are on Windows, you can ignore the submodules in the Linux/ directory. Among others, this directory contains the cairo and skia submodule that is around 500 MiB. On Linux, you can likely ignore the cairo submodule, as it is not used by default. To selectively download submodules, it is necessary to clone the repository in two steps. First, clone the Storm repository without the --recursive flag:

git clone git://storm-lang.org/storm.git

Then initialize all submodules. This also provides a list of the submodules so that it is possible to deactivate them later:

cd storm
git submodule init

Then, for each submodule that you don't wish to download, execute the following command. The example below removes both the cairo and the skia submodules:

git config --remove-section submodule.Linux/cairo
git config --remove-section submodule.Linux/skia

(Note: git submodule deinit should work, but does not seem to work properly before cloning the submodule.)

When you are done, run git submodule update --progress to clone the remaining submodules.

Compiling Storm

Once all dependencies are installed and the source code is downloaded, Storm can be compiled using Mymake. To understand how the process works, we need to know a little bit about how Mymake works.

According to Mymake, Storm is a project that consists of a number of targets. The project is the root of the Storm repository (it contains a file called .myproject). Each target is a subdirectory in the project directory. Each target contains a file named .mymake to indicate this. This means that the directories Test and Main are targets, while scripts, doc and root ar not targets.

To compile a project, Mymake needs to know which target it should produce (this can be specified in the project file, but this is not the case for Storm). As such, we can compile different parts of Storm depending on which target we ask Mymake to compile. The two most important targets for Storm are Main and Test. The target Main produces Storm's main executable (distributed as storm), while the target Test compiles the test suite in Storm.

Finally, it is worth knowing that the default behavior of Mymake is to not only compile the selected target, but to also run it if compilation was successful. As such, it is possible to run and compile Storm with the following command:

mm Main

Similarly, the test suite can be compiled and run with:

mm Test

The commands above both produce binary files in the debug/ directory. If you do not want Mymake to execute the created file, the flag -ne disables this behavior.

Compiling Storm from scratch typically takes a couple of minutes on recent hardware. However, if you are building Storm on Linux, it must first compile the Skia library, which can take upwards of 10 minutes, even on recent hardware. This is, however, only done once as Mymake tracks updates to files and only re-compiles the necessary files.

Compilation Options

The build files for Mymake have a few configuration options that modify the behavior of the build. Note that Mymake tracks the command lines used to produce intermediate build files. This means that Mymake produces a correct build when the build options have changed, even without a clean in between. However, it also means that a large portion of Storm need to be rebuilt when options change.

The following options are available:

mps
Use the MPS garbage collector (the default). The mps flag produces the files Storm_mps and Test_smm in the current output directory.
smm
Use the SMM garbage collector (experimental). Produces Storm_smm and Test_smm files in the output directory.
release
Compile a release version of Storm. This version uses a separate set of build directories to not interfere with the debug version. The resulting binaries are placed in release/ rather than debug/.

The release version utilizes slightly more expensive optimizations than the debug version, and causes asserts to become no-ops among other things. The release option also instructs Storm to build both the Main and the Test projects, and to not execute them.
dist
On Linux: Build Storm to be installed in /usr/bin. This means that the logic for finding the root directory is different than normal.
noskia
On Linux: Do not compile and link Skia to the UI library. This does, however, mean that hardware accelerated graphics will not be available.
localmariadb
On Linux: use the local copy of the MariaDB headers instead of the ones installed in the system. This makes it possible to build Storm without libmariadb-dev, but may cause compatibility issues if your system ships a version of the MariaDB client library that is too different from the one in the source repository.
usecairogl
On Linux: Enable the experimental OpenGL mode in Cairo for the UI library. This will cause Mymake to compile Cairo from source, which can take some time. Since this mode is not officially supported by Cairo, it may be removed in the future.
nostatic
Do not use the libraries in the repository where possible, but rely on system libraries instead. This means that the following libraries will have to be installed in addition to the ones mentioned above: libmpg123-dev, libogg-dev, libflac-dev, and libvorbis-dev.
nouring
On Linux: Don't use the io_uring family of API:s for I/O in Storm, but rely on the traditional read and write APIs instead. This means that doing disk I/O from one thread blocks all other threads on the same OS thread. This option is likely only necessary if you need to use Storm on systems where io_uring is unavailable for some reason (e.g. userspace emulation in QEMU).
branchprotection
On Linux on ARM64: Compile Storm with support for branch proection (PAC). Note that Storm's implementation of PAC (like many other compilers) retain compatibility with hardware without support for PAC by only using the instructions that were previously NOPs. As such, the only downside of enabling branchprotection is that it inserts extra instructions in each function, and that the implementation is currently less well-tested (hardware with PAC support is not too easy to access).

Common Errors

Mymake utilizes precompiled headers to speed up the build process. This is generally positive, but can cause issues in some cases. In particular, the precompiled header files are closely tied to the exact version of the compiler that is being used. This means that the build process might fail whenever the compiler has been updated. Typically this results in a message that says that a "pch file" or "precompiled header" is invalid. This error is resolved by deleting all files with the extension pch (on Windows) and gch (on Linux). In a Unix-like shell, this can be done as follows:

find . -name "*.gch" -exec rm \{\} \;

Using Mymake to Run Storm

When developing Storm with Mymake, it is often convenient to let Mymake handle both compilation and execution of Storm. This means that it is enough to keep track of a single command line for building and testing the program, rather than keeping track of separate ones.

To test a specific piece of code in Storm, it is necessary to pass parameters to the compiled Storm program. Mymake enables this by passing any parameters specified after the special parameter -- directly to the compiled program. As such, anything that appears after a -- on the Mymake command line will be passed directly to Storm. For example, to pass the parameters -f progvis.main to Storm upon successful compilation, specify the following parameters to Mymake:

mm Main -- -f progvis.main

Other parameters, like -i or names of files, can be specified as usual as well. Note, however, that even though it is possible to execute Mymake in any subdirectory in the Storm repository, Mymake will set the current directory to the root of the repository before doing anything. This means that any relative paths passed to Storm will be incorrect, and errors will be reported. As such, to avoid this kind of surprises, it is a good idea to always run Mymake in the root of the Storm repository.

Note: In versions of Mymake earlier than 2.3.3, the flag -- was not available. In these versions, use -a instead of --.