Metaprogramming

The package lang:bs:macro contains a number of convenient utilities for interacting with the name tree, and generating abstract syntax trees for Basic Storm. These are presented below.

Generating Names

When looking for names in the name tree it is necessary to create core:lang:Name instances that represent the desired name. This can be tedious to do by hand, so the lang:bs:macro package contains custom syntax for this purpose:

lang:bs:SrcName name = name{a:b<c>};

The block name creates an instance of the SrcName class (which inherits from Name) that contains the name specified inside the curly brackets. An SrcName additionally keeps a reference to the location in the source code where the name was written, so that error messages can provide an accurate location.

The name syntax extends the allowed syntax for names slightly to allow accessing more "exotic" names more easily. It allows the name of each part to be enclosed in quotes like a string literal. String escapes are interpreted as usual. This makes it possible to access operators as follows:

name{core:Int:"+"<core:Int, core:Int>};

Accessing the Name Tree

Similarly to the name macro, it is often useful to reference entries in the name tree directly. The named{} syntax can be used to achieve this in a type-safe way. The named syntax looks up the name within the curly brackets at compile-time, and inserts a reference to the specified named entity in the generated code. This means that the named{} syntax incurs no overhead at runtime, since it compiles to a single load instruction. The named{} syntax supports the same extensions to names as name{} does.

Since Basic Storm knows the exact type of the named entity at compile time, named{} evaluates to the type of the referred named entity. This often eliminates the need for downcasts, that would be required when looking up names manually.

If no name is specified in the curly braces, the syntax evaluates to the current package.

For example:

use lang:bs:macro;
use core:lang;     // for Package
use core:io;       // for Url

Url findPackageDir() {
    Package current = named{}; // Get reference to current package
    unless (url = current.url)
        throw InternalError("Virtual package.");

    // If the package was not virtual, it has an Url:
    return url;
}

Similarly, one can get a reference to other elements easily (full names are given for clarity, the normal name lookup rules apply, so it is possible to rely on use statements, etc.):

// The integer type:
Type intType = named{core:Int};

// The array type for integers:
Type arrayType = named{core:Array<core:Int>};

// To string function for integers:
Function toS = named{core:toS<core:Int>};

Inspecting the Basic Storm AST

The macro package provides the syntax dump{ <statement or expression> } to inspect the abstract syntax tree in Basic Storm. The syntax simply captures the syntax tree and prints the string representation of it (by calling toS), and fowards it to the surrounding context. As such, the print happens whenever the code is compiled, and nothing is added to the generated code. In most cases, this is fairly uninteresting for constructs built into Basic Storm, as their string representations closely match the syntax. It is, however, very useful for debugging syntax extensions.

For example, it is possible to view how the for loop is transformed as follows:

dump{ for (Int i = 0; i < 10; i++) {} }

Inspecting the Generated IR

Similarly, the macro library contains the dumpAsm pseudo-function that dumps the generated intermediate code at the location. As with the dump syntax, this happens whenever the code is compiled, and does not affect the generated code.

Since Basic Storm stops generating code after a return statement, it is not possible to use dumpAsm to examine all IR of a function. There will always be a piece of the logic for returning the final value that will not be visible using this approach. It is still immensely useful for quickly debugging the output of custom syntax etc.

For example, one can inspect the IR generated for simple assignments as follows:

Int x = 10;
x++;
dumpAsm;
// ...

Patterns

The pattern construct provided by the lang:bs:macro package allows easy generation of Basic Storm syntax trees from source code, without needing to manually creating each node. A pattern saves the parse tree generated by the parser in the source code, and transforms it when the pattern statement is evaluated. As such, evaluating a pattern statement, especially for a large piece of code, is associated with some cost. However, since the code is parsed ahead of time, the cost is not too large.

For example, we can generate a syntax tree that represents creating an array as follows:

Expr x = pattern(block) {
    Array<Int> array;
    array.push(1);
    array.push(2);
};
print(x.toS());

The parameter block (expected to be a subclass of lang.bs.Block) to the pattern syntax dictates in which context identifiers inside the pattern should be resolved. This is the same mechanism that Basic Storm uses internally to keep track of the "current" block when instantiating AST nodes. When using patterns in the context of custom syntax extensions, the parent block is generally available as a parameter to the productions extended for the syntax extension (both SStmt and SExpr has a parameter block that contains the current block).

Patterns also allow inserting expressions generated by the surrounding code using the syntax $name or ${expression}, similarly to the syntax used for string literals. The first version only allows a single identifier to be specified, while the second syntax allow arbitrary Basic Storm expressions to be evaluated and inserted at that location in the syntax tree. Regardless of which version is used, the identifier or expression is expected to evaluate to an instance of lang.bs.Expr or core.lang.Type depending on whether it appears in a location where an expression or a type is expected to appear.

For example, expressions can be inserted as follows:

Expr e = Constant(SrcPos(), 20);
Expr f = pattern(block) {
    10 * $e + ${Constant(SrcPos(), 10)};
};
print(f.toS());

Which produces the expression:

{
    10 * 20 + 10;
}

For example, types can be inserted as follows:

Type t = named{core:Str};
Expr e = pattern(block) {
     Object o = "hello";
     if (o as ${t})
         print(o);
};
print(e.toS());

Which produces the expression:

{
    Object o = "hello";
    if (o as core:Str)
        print(o);
}

It is also possible to insert a variable number of expressions into a parameter list, an array initializer, or a block using @name or @{expression}. In this case, the expression or identifier should evaluate to an array of lang.bs.Expr objects, which are inserted in sequence at the specified location. For example:

Expr[] e = Expr:[Constant(SrcPos(), 1), Constant(SrcPos(), 2)];
Expr f = pattern(block) {
    foo(@e, 8);
};
print(f.toS());

Which produces the expression:

{
    foo(1, 2, 8);
}

It is not necessary to use the pattern construct to create an entire block. It is also possible to create a pattern containing a single statement:

Expr e = Constant(SrcPos(), 20);
Expr f = pattern(block) 10 + $e;

As previously mentioned, patterns are useful when implementing syntax extensions. For example, we could implement a syntax for absolute values as follows:

BNF-file:

SAtom => absExpr(block, expr) : "|", SExpr(block) expr, "|";

BS-file:

use lang:bs;
use lang:bs:macro;

Expr absExpr(Block block, Expr expr) {
    pattern(block) {
        Int tmp = $expr;
        if (tmp < 0) {
            0 - tmp;
        } else {
            tmp;
        }
    };
}

From this example, we can also see that the patterns are hygienic, which means that if evaluating the expression |tmp| works as expected, even if tmp is used as a variable name inside the pattern.