ChunkBake Reference

Introduction

ChunkBake is a line-oriented assembler for Lua 5 virtual machine instructions. It produces a Lua binary chunk from assembly code, which can then be loaded and run on a Lua 5 virtual machine. The project is hosted on LuaForge, at: http://luaforge.net/projects/chunkbake/. Currently, Lua 5.0.2 (instructions and binary chunk format) is supported.

This reference guide assumes you know what Lua 5 virtual machine instructions do and how a binary chunk is put together.  If you want to learn about Lua 5 virtual machine instructions, one document you can read is "A No-Frills Introduction to Lua 5 VM Instructions."

Usage

ChunkBake currently sports a very simple interface:

usage: %s [options] [filenames]

options:
  -h, --help        prints usage information
  --quiet           do not display warnings
  -o <file>         specify file name to write output listing
  --                stop handling arguments

example:
  >%s myscript.asm -o myscript.out

If an output file is not explicitly specified using the -o option, the basename of the source file is appended with .out to form the output filename. The format of the binary chunk file that is generated should conform to that of the platform the script is running on. ChunkBake uses Lua to generate a small binary chunk and takes those parameters as defaults. If this somehow fails (and it should not), the defaults are set for the x86 platform. For other platforms, this is untested but should work. The binary chunk format can also be customized using assembler directives, overriding the platform defaults.

Overview

ChunkBake is a classic line-oriented assembler, meaning that a statement (which can either be an instruction or an assembler directive) must be in a single logical line. Usually each logical line occupies one physical line, but the line continuation character '\' can be used to split a long line into multiple lines. A statement can contain a directive or a mnemonic, plus associated operands or parameters.

ChunkBake's lexer is derived from Lua's lexer. Mnemonics (e.g. MOVE,) directives (e.g. .function) and key names for header properties are not case-sensitive, but other declared symbols are case-sensitive.

Lexical Analyzer

The lexer recognizes and handles the usual line-ending styles automatically: LF, CR, CRLF.

Only Lua 5.0.2 opcodes are supported at this time. The mnemonics follow the symbol names defined in the Lua 5 sources (their syntax will be described in detail later):

Most numbers allowed by the assembler are integers. For constants, a number must be prefixed immediately by a '#' (pound or sharp) symbol, e.g. #12, #0.123 #-123 #1.2e34. When specifying constants, the number may have a fractional part and an exponent part.

Expressions are not supported at the moment, it is planned feature, though. The '#' will still be used to denote a constant, in order to unambiguously differentiate constants from local stack locations.

The lexer recognizes hexadecimal numbers with the '0x' prefix as well. Examples are 0x1234, 0XBEEF.

A '\' at the end of a line is a generic line continuation character, so the following are equivalent:

A comment to the end of the line can use either '--' or ';':

A block comment uses the Lua 5 '--[[...]]' style:

Locals in the current stack frame, which works like registers in a register window, can use several styles. The following are equivalent:

Mnemonics cannot be used as user-declared symbols or labels. Any symbol with a '.' as the first character (e.g. .local) is considered to be an assembler directive. The directive names currently used by the assembler are:

The assembler has three predefined keywords (case-sensitive) that are used as operands in certain instructions or directives:

Labels are explicitly identified when a symbol has a ':' suffix. Labels can also be placed at the beginning of most statements without the ':' suffix. All labels or symbols must be defined before they can be used, except for labels used in jumps, which can be forward-referenced. Examples:

As of version 0.5.0, symbolic names occupies several tables associated with different kinds of objects. Thus it is possile to use identical names for different things. A single symbol table avoids such confusion. I consider the use of multiple symbol tables broken, and it will be fixed in a future release.

Assembler Directives

There are two kinds of directives:

• Header directives:
• .header is used to set global binary chunk header parameters and global characteristics.
• .function is used to set a function header's characteristics, and is paired with .end to contain the assembly code for a single function.
• These two directives have a similar syntax for parameters; parameters are supplied as key-value pairs separated by optional commas like this: version=0x50, size_t=2

  
• Resource directives:
• .param is for declaring named parameters. The assembler will keep track of the number of parameters and update the numparams function header field automatically for you.
  
• .local is for declaring named locals. You may specify the exact stack location for the declared local, instead of letting the assembler allocate the next available location.
  
• .upvalue is for declaring named upvalues. The assembler keeps track of the number of upvalues internally and fills in the correct value in the appropriate function header field.
  
• .const is for declaring named constants. Constant handling is special, as the assembler accepts constants used directly as operands in an instruction. This greatly simplifies assembler code in many cases. The choice whether to declare explicitly or automatically is up to you.
  

As ChunkBake provide sane defaults, there is often little need to use header directives unless you are generating a non-standard binary chunk. Likewise, you can choose not to specify constants explicitly using resource directives and let the assembler handle it for you.

In the following sections, each directive will be described in more detail.

.header

.header [key=<string>|<number>[,]]...

The .header directive is an optional directive; you only need to use it to enforce a value for a particular global header parameter. If you are generating a generic binary chunk targeted towards the platform the assembler is running on, you will not need to use it at all.

Apart from being able to customize the global header elements of a Lua 5 binary chunk, the .header directive is also able to set a number of other parameters that governs the generation of the binary chunk, e.g. the maximum number of allowed upvalues can be changed.

Parameters are specified as key-value pairs, optionally separated by a comma. If a statement is very long, it can be split into multiple .header directives, or the line can be split into several physical lines using the '\' line continuation character. If a key is specified a number of times, the last value is taken as the final valid value. The .header directive can only be used before the definition of the first function prototype.

Keys are case-insensitive names, while values can be numbers (usually integers) or strings. For example:

The directive changes the binary chunk signature from the usual "\27Lua" to "\27Foo". In most cases, illegal values are flagged as errors. The assembler allows binary chunk signatures that are not 4 bytes in length; a warning message is generated instead of an error.

The parameters that can be changed are as follows:

The first 11 keys in the list are almost equivalent to their counterpart fields in a standard Lua 5 binary chunk header. The last 5 keys are constants you probably shouldn't touch unless you are running a Lua binary that has non-standard constraints and you know what to do and what the keys mean.

Note that the Lua binary chunk format may change with different versions of Lua. The upcoming Lua 5.1 has a slightly different binary chunk from that of Lua 5.0.2, so beware when changing version.

Errors may be generated if you run afoul of internal constraints, e.g. the instruction opcode field SIZE_OP must exceed 6 bits. Here is an example of a custom .header directive:

If you don't want to change the detected default parameters, you don't need to use .header at all.

.function and .end

.func|.function [key=<string>|<number>[,]]...
.end

Each function must be started by a single .function directive and ended by a .end directive. .func, an alias of .function can also be used. There can be only one top-level function and functions must have at least one valid instruction. Thus the simplest assembly listing that can be written is:

The key-value pairs for customizing a function are:

Unlike .header parameters, you usually don't need to customize a .function directive. Normally, the assembler will fill in the parameters for you. The source_name parameter is set by the assembler for the top-level function, as is line_defined. numparams is automatically calculated when you use the .param directive, and the assembler keeps track of stack locations used in order to set maxstacksize, so usually it is wiser to leave them be.

For functions that accept a variable number of arguments, is_vararg should be set to 1, and the implicit parameter arg will be in stack position 0. An example of a function that accepts a variable number of arguments is printf.asm in the /test directory.

Here is an example of function header customization:

So, for normal use, you won't need to specify anything for the .function directive, except for is_vararg when you need functions that accept a variable number of arguments.

Functions can be named using labels, so that names can be used instead of numbers when writing a closure instruction. The following are two functions named ONE and TWO:

Next comes directives that are used to declare named resources for a function.

.param

.param <symbol>

.param declares a register location as a function parameter, or argument, and assigns a name to the parameter. Since parameters occupy the lowest register numbers, the assembler assigns the locations for you automatically, starting from location 0. You are not allowed to declare any local variables before parameter declarations. The numparams field in the function header is automatically updated by the assembler.

.function
.param Var
.param Pos
add 2, Var, Pos
; more instructions here
.end


In the example above, Var is assigned register location 0 while Pos is assigned register location 1. The add instruction adds parameters Var and Pos and saves the result in register 2. You can still use 0 and 1 as the parameter locations in an instruction; the usage of symbolic names and numbers is up to the user.

A parameter declaration need not be placed immediately after .function. You only need to ensure that all local variables are declared after parameter declarations, and that symbolic names names are declared before they are used.

.local

.local <symbol> [[,] <number>]

.local declares local variable names. You can either let the assembler assign the next lowest available location that does not have a symbolic name associated with it (either from a .param or .local declaration.) Alternatively, the register number can be explicitly specified.

.function
.param Foo
.local Bar
.local Baz, 4
; rest of function goes here
.end


Since location 0 is occupied by Foo, the first .local directive associates the name Bar to location 1. The next .local directive explicitly assigns the name Baz to location 4. The comma separator is optional. Now, if there is another .local directive in the function without a specified location number, location 2 will be used.

Locals must be declared after all parameters have been declared. Other than that, locals can be declared anywhere inside a function, as long as symbolic names are declared before they are used.

.upvalue

.upvalue <symbol> [[,] <number>]

The syntax of .upvalue is exactly like the syntax of .local. Assignment of upvalue numbers can be automatic or explicit. The assembler sets the number of upvalues (which is a byte-sized field) in the function header for you.

.function
.upvalue Larry
.upvalue Moe 2
; rest of function goes here
.end

Upvalue number 0 is assigned the name Larry, while upvalue number 2 is assigned the name Moe. The operands of the directive can be optionally separated by a comma. When the upvalue list is written out, non-declared values in the upvalue list is assigned the name "(none)" so that loaders don't complain of zero-length names.

.const

.const [<symbol> [,]] <number>|<immediate>|<string>|nil [[,] <number>]

Constants are declared using the .const directive. For constants, both the constant number and the symbolic name of the constant is optional. Leaving out the constant number forces automatic constant number assignment. If no name is associated with a constant, it must be accessed using the constant number.

The constant may be a string, an immediate number (with a leading '#'), a number or nil. A number is interpreted as an immediate, but if you are not comfortable with the ambiguity, you can always declare numbers using the immediate syntax.

Here are some unnamed constant declarations with automatic constant number assignment:

Constants can be declared anywhere within a function as long as symbolic names are declared before use. Here are more constant declarations:

Commas between operands is optional. Constant declarations with identical values are not merged; as far as the assembler is concerned, it is dealing with separate constants.

In order to simplify the writing of Lua assembly code, it is possible to write code without a single .const declaration. The assembler allows constants to be used directly as operands wherever there is an RK(x) operand or a Kst(x) operand. This is also why numbers need to be disambiguated from numerical constants (immediates). When constants are used directly as instruction operands, the assembler will automatically assign constant numbers to the constants and duplicates constants will use the same constant number.

The user is given the ultimate control as to how constants are declared or used in functions in the assembly code. The intent of the handling system for constants is that important variables can be declared as named symbols, while trivial constants can be used directly as operands in instructions.

Labels and Symbols

Labels can be used for .param, .local, .upvalue and .const. Since you can name a function's resources using the declarations only, labels are a kind of aliasing mechanism. For example:

In the example above, both Anderson and Neo points to the same constant number that holds the string "Thomas Anderson". Although mechanism is available to the user, it is probably a good idea to use it only when there are good reasons to do so.

Instruction Operands

• R(x): A register location in the current function's stack frame. E.g. R(A) means that opcode field A specifies a register location. A register can be written in a few ways: For example, register 2 can be written as 2, $2 or R2. It can also be a name that refers to a parameter or a local variable. The name can be defined as a label or in a .param or .local directive.
  
  
• RK(x): The operand may be a register location or it may point to a constant in the constant pool of the function. A register location is specified in the same manner as in an R(x) operand. Constant numbers can be explicitly specified, but note that they must be greater or equal to maxstack, which is normally 250. Note that maxstack can be customized in a .header directive. So, a 250 will normally point to constant number 0. In theory, R250 can also be used, but this is misleading; constant numbers should be written as bare numbers. Alternatively, constants can be referenced using a name or a label. Constants can also be used, e.g. #47, nil or "Foo" are legal operands. Such constants are allocated a number automatically, and identical values or strings share the same constant number. This is to allow trivial constants to be used very easily, without the hassle of declaring each and every constant. The scheme is very flexible, it is up to the user to write clear and readable code.
  
  
• Kst(x): This operand denotes a constant only. Constant numbers, names or labels, or immediate constants in the form of immediate numbers (e.g. #123), strings or the nil keyword can be used. Like RK(x), if constants are used directly, the assembler handles allocation automatically.
  
  
• Disp: Denotes a displacement. A number (e.g. 20) denotes an absolute target location. An immediate number (e.g. #1, #-10) denotes a relative displacement which corresponds directly to the value in the sBx field of the assembled instruction. A label name makes writing jump targets much easier. The target label must be somewhere within the same function body, and forward and backward reference is allowed.
  

The assembler tracks register usage by noting the R(x) and RK(x) operands and sets the maxstacksize field in a function header accordingly. So normally, the user does not need to do anything about the maxstacksize field. You can explicitly set maxstacksize too, but if your custom value is too small, then the assembler will flag an error.

Other operands are described within the descriptions of Lua 5 virtual machine instructions in the next section.

Lua 5 Instructions

Like most classic assemblers, instruction statements are written on a per-line basis. In the case of ChunkBake, 'per-line' refers to a logical line, since a '\' can be used as a line continuation character. Also, strings can occupy more than one line.

Operands are optionally separated by commas. Thus, in the syntax descriptions below, any comma can be omitted. This relaxed rule is to allow for different writing styles. It is recommended that the user stick to a single style in a program. In some cases, '..' can be optionally used to replace a comma to denote a range of registers or values. This helps to make the intent of the instruction clear, e.g. loadnil 0..2

Many illegal conditions are trapped by the assembler, however, writing illegal or incorrect instructions is by no means impossible. The assembler lacks a comprehensive warning system at the moment, so it is largely up to the user to enforce some discipline in coding.

The following describes only how instructions are written in the ChunkBake assembler, it does not describe what each instruction does. For help on the latter, please see the beginning of this document. Most of the examples in the following are copied from the automatic test script, TestRig.lua.

Moves values between register locations. Examples:

Loads a constant into a register. Examples:

Loads a boolean into a register. The boolean value can be specified as a number (0 for false, 1 for true) or the true and false keywords can be used. The flag in field C for skipping the next instruction may be specified as a 0 (don't skip) or 1 (skip the next instruction.) Examples:

Sets a range of registers to nil. In order to make the intent of the instruction clear, '..' can be used instead of a comma to separate the two operands, thus showing that the instruction is operating on the given register range. The second register must be greater than or equal to the first register. Examples:

Loads or saves an upvalue. Examples:

Loads or saves a global variable. Examples:

Gets a value from a table. The table key can be enclosed in square brackets for increased readability. Examples:

Sets the value of a given table key. The table key can be enclosed in square brackets for increased readability. Examples:

Creates a new table object and assigns it to the given register. The assembler accepts either encoded size values (as integer numbers) or raw size values (as immediate numbers). Examples:

Sets up an object call. Part of the operation includes a table lookup, hence the syntax of self is similar to the syntax of gettable. Examples:

Binary arithmetic operations. Examples:

Unary minus and logical not operations. Examples:

Concatenates a range of registers and assigns the result to a register. A '..' can be used instead of a comma to separate the operands that make up the range of registers. Examples:

Relational operations. The A field can be 0 or 1, which signifies the result to be tested for. The next instruction is skipped if the comparison result is not the same as the expected result. Examples:

Text operation. The C field can be 0 or 1, which signifies the result to be tested for. The next instruction is skipped if the comparison result is not the same as the expected result. Examples:

Performs an unconditional jump. Jumps can be to absolute positions, to relative positions or to a label. Absolute jumps must be a positive integer, but bounds checking is not currently performed, so illegal instructions can be coded. Likewise for relative jumps. Also, the assembler does not currently check the case where the jmp instruction jumps to itself, creating an infinite loop. In future versions of ChunkBake, the user will be warned of such cases. Examples:

Calls a closure. Examples:

Returns to calling function or performs a tail call. Examples:

Performs an iteration of a numeric for loop. See the jmp instruction on the use and specification of jumps. Examples:

Performs an iteration of a generic for loop. Examples:

Performs the initialization for a generic for loop. See the jmp instruction on the use and specification of jumps. Examples:

Sets a table with a list of values in a range of registers. You can either encode the range and specify a single number, or specify a range and let the assembler calculate the Bx value. setlisto is used when the final element of the list is a function call, which has an indeterminate number of return values. Examples:

The encoded number and the range specification must be valid, or else the assembler will report an error. Note that FIELDS_PER_FLUSH (or fpf in the .header directive) can be customized, so the index range calculations will be adjusted accordingly.

Closes local variables on the stack from the specified register onwards. Examples:

Creates a closure or instantiation of a function, assigning the result to a register. If the function prototype has a label, the label can be used to refer to it. Examples:

Examples

A number of examples can be found in the test directory of the ChunkBake distribution. The automatic test system, TestRig.lua in the misc directory, has a pretty complete coverage of both legal and illegal forms of all instructions.