SDPL – Language Fundamentals

Contents

Serial Datastream Procedure Language

V2.0

Language Basics and Symbols

I’ve intended for this language to be fairly low level and simple, so that the program can easily compile it into a series of data structures that can be executed quickly.

Comments

The # mark can be used to specify a comment, everything after a # mark is completely ignored until the end of the line.

PRINT "AWESOME" # this is totally ignored.
# this is also ignored.

Hexadecimal Byte Arrays

The default way to specify data is in HEX.  Everything that is not HEX will have a specific symbol prefix.

Hex arrays can be in spaced bytes such as:

AA FF 01 00

Or with no spaces.

AAFF0100

The overall length must be an even number of digits to form 8 bit values, and spaced values must also be in groupings divisible by two.  The following examples, although technically valid hex, would be an error:

AA FF 01 0
AAF 010

Integer Translation

To specify a number as an unsigned base 10 integer, rather than in hex, you can prefix it with a dot.  The maximum size is 64 bits.  This is converted on the fly to a byte array.

.12345

Array Size

One important thing to remember is that arrays have a length.

If you set an array to 11, that’s a one byte array.  00 11 is a two byte array.  They both technically contain a value of 11 (or 17 in decimal terms) but are effectively a different size.  Also, 00 00 00 00 00 00 has a size of 6 bytes, but has the same value as 00.

Arrays generally will resize themselves as data is appended.  To increase the size of an array without increasing its value, prepend zeroes.

String Translation

Strings are important for debug output, and interfaces such as ELM that exist in string territory, so we make it very easy to incorporate a string of text into your data.  Quotation marks define an ASCII string:

"This is a string"

When the program is compiled, the string is converted to an ascii array, and any knowledge that it used to be a string is lost forever.

Variables

The symbols $ and % are used to define a variable.

The dollar sign $ is used to designate a variable that is read from memory, and usually substituted into a command.  For example,

PRINT $VARIABLE

This would simply substitute the current value of $VARABLE into a print command.

The percent sign % is used to define a variable that is being written.  For example,

SET %VARIABLE AABBCC

The two different symbols to define a variable are important for commands such as RX, which allow both read and write variables within the same data.

Result Variable

The system result variable is stored (and can be called as) $*

Many operations return a numerical status code, for example, a chain of RX matches returns the index of the matched reply, or retrieving a value from a map will return its offset.

ARRAY RANGING

Variables called both for input and output can have their target range specified using square braces: []

This can take several forms, and is the basis of a versatile system for array manipulation.

Keep in mind this is one of the only places in the language that, by default, the input is in entered in base 10 integer rather than hex.

Assume a variable contains AABBCC:

SET %VAR AABBCC

If a single term with no dash is specified, we are specifying its length in bytes.

This would output AA:

PRINT $VAR[1]

To output a range of bytes, for example bytes 1 and 2 (this would output BBCC).  Note that when specifying byte positions, we index the first byte as 0, so byte 1 is actually the second byte.

PRINT $VAR[1-2]

To output everything from byte 1 onwards simply leave that term blank:

PRINT $VAR[1-]

Many commands that set a variable’s contents will also allow ranges.  This sets the first two bytes of X to the second and third byte of Y.  The other bytes of the variable would be untouched.

SET %X[0-1] $Y[1-2]

The language also allows dynamic array manipulation by specifying a $VARIABLE as part of a range.  This can yeild very complex array manipulation if necessary.

Great care must be used with this method, as variables can be quite large, so exceeding the available range of a variable is definitely possible.  The ranges have a maximum value of 32 bits (4 bytes).

For example this would result in AA BB being printed:

SET %Y 01
SET %X AA BB CC
PRINT /HEX $X[0-$Y]

Pointers

Code flow generally uses pointers, both for branching and subroutines.  Pointers are specified by the * prefix.  More about that later, but here is an infinite loop:

*X
GOTO *X

Parameters

Options/parameters for a command are always prefixed by a forward slash /

A parameter with a value is specified as such: /PARAMETER=VALUE

Parameters are versatile, and their use varies drastically based on the operation being performed.  Parameter values do not allow spaces, and their format and context varies entirely based on the operation being performed.  Most operations will fail to compile with an invalid parameter.  Some commands care about the order and position of parameters, and some do not.

In the example, the PRINT option can print in text, or as a hexadecimal string.

To designate HEX output:

PRINT /HEX AABBCC

Data Separators

Many commands require two parts (often an input an output) which require separation.  One example would be a compare command, which compares two values.

The primary separator is a | symbol, but we also accept the following words: WITH, TO, and….AND.  These are chosen since they cannot conflict with hexadecimal notation.

The following commands are all equivalent.

COMPARE AA TO BB

COMPARE AA | BB

COMPARE AA WITH BB

COMPARE AA AND BB

Status Register

Each program has a status register which consists of four status bits, and operations may set or clear the bits in this register to convey results, which are then branched on.

This type of thing should be familiar to all programmers that have worked with common processor assembly, and it allows sequences of instructions to react to the result of various operations.

The bits are OK (aka TRUE/FALSE), LESS, MORE, and CARRY.

  • OK is set on success or truth, and cleared on failure or false.
  • LESS and MORE are for math comparisons.
  • If neither LESS or MORE is set, that means the values were EQUAL.
  • If the result of an addition/subtraction overflowed (the value exceeded its maximum and wrapped around), the CARRY bit will be set.  For example, 0-1 would set the carry bit, and 2-1 would clear the carry bit.

Instead of directly branching from a comparison, one would compare two values, then use the status register to determine what to do.

You examine the status register by running conditional IF statements.

For example, this compares a variable X to the value AA and exits if it is equal to AA.

COMPARE $X | AA
IF = EXIT

Not all operations touch the status register, which mean the information will remain until another operation that does manipulate that status bit sets/clears its values.  This can yield some very interesting branching tricks.  For example, because the print command does not touch the status register, you can run multiple conditions against a single comparison:

COMPARE $X | AA
IF = PRINT "EQUAL"
IF > PRINT "GREATER"
IF < PRINT "LESS"

TRUE / FALSE

The commands TRUE and FALSE simply set the status register’s OK bit on or off.  This is very useful when returning from a subroutine where you need to set your own status codes.

Operations – Jumps and Subroutines

LABEL (or *)

This defines a pointer, allowing other operations to call this pointer by name and return to this point.

The LABEL command is a bit ugly, so we also allow just *POINTER (which the compiler will translate to LABEL for you)

Declaring a pointer twice would be considered a fatal error.

*RIGHT_HERE
(or)
LABEL *RIGHT_HERE

GOTO

This goes to the specified label:

GOTO *LABEL

CALL (Subroutine)

This “Calls” a subroutine.

A subroutine starts with a *POINTER , but must call RETURN when finished, at which point execution returns to the call point.

Care must be taken, as no distinction is made by the compiler between a standard LABEL and a subroutine.  Never GOTO a subroutine, and never CALL a pointer without a subroutine RETURN.

In technical terms, subroutine calls are placed in a stack, so subroutines can be nested.  If you GOTO a subroutine, nothing is added to the call stack, so the RETURN at the end of the subroutine command will exit the program.  If you CALL something that does not run a RETURN, then the CALL point will remain in the return stack, and you could end up with some unpredictable code flow.

RETURN

This returns to the call point of the CALL function, effectively returning the subroutine.

If the RETURN operation is used without a subroutine CALL, the program simply exits.  It is a good idea to put subroutines at the end of the program, or jump over them so they are only executed when called.

Here is an example of a subroutine.

CALL *SUBROUTINE
EXIT
*SUBROUTINE
PRINT "Something"
RETURN

Operations – Memory (Variables)

SET

Place a value in a variable.

SET %VAR AA BB CC

The value can contain other variables, so SET can also be used as a copy operation

SET %SOMETHING $SOMETHING_ELSE

DECLARE

Create a variable (or many variables) but do not place any data in them.  This is useful if your program or subroutine is mostly called externally.  It is not used often.

DECLARE %X %Y %Z

CLEAR

Delete a variable’s contents and set its size to zero.  If a size is specified [xx] then the array will be resized and zero filled to that length afterwards.

CLEAR %VAR

SIZE

Place the size (in bytes) of something in a variable.  The result of the following example would set VAR to 03.

SIZE %VAR AA BB CC

APPEND / PREPEND

Append or prepend a value to a variable.

APPEND %VAR AA

CUT

STATUS: No changes.

This removes a range of data from a variable, resizing the array afterwards.

CUT %VAR[1-2]

TAKE

This sets the contents of a variable to a range from another, then removes that data from the source array, in effect ‘taking’ those bytes.

The destination also supports ranges.

TAKE %DESTINATION %SOURCE[1-2]

Operations – Memory (Stack, Global, etc)

PUSH/POP

There is a stack to which you can push/pop arbitrary values.

The POP command also supports ranges (however the entire value is still removed from the stack)

The POP command sets the status register’s CARRY bit depending on whether the POP was successful or not.  Popping a value from an empty stack will SET the carry bit, otherwise it will be cleared, and nothing will be done.

PUSH "SOMETHING"
POP %X
PRINT $X

IMPORT/EXPORT (Global Memory)

A facility is provided to share data with other running (or non-running) programs.

For simplicity, your global variable name and local variable name will be the same.  It is recommended that you choose very unique names as not to conflict with other programs.

To import the value of a variable from global memory, you must explicitly run the IMPORT command each time.

IMPORT %EXTERNAL_THING
PRINT $EXTERNAL_THING

To export data, it must be placed in a variable first.

SET %EXTERNAL_THING "Something"
EXPORT $EXTERNAL_THING

Multiple variables can be imported or exported at once:

IMPORT %X %Y %Z

Operations – Input/Output

PRINT

This prints a value to the log for the user to read.

By default, the value is output as a string, however the mode can be changed on the fly with /TEXT, /HEX, or /INT.

For example:

PRINT /HEX AA BB CC /TEXT "IS A BIT OF HEX"

Will output:

AA BB CC IS A BIT OF HEX

Using a command such as this will likely output garbage, as AA BB CC is probably some unreadable crap in ASCII:

PRINT AA BB CC

To print the value of an array as an unsigned integer (maximum 4 bytes or 32 bits):

PRINT "The result is: " /INT $VALUE

The options /STATUS /DEBUG /COMM or /ERROR specify the log section that the output is directed to.  It does not matter where they are specified, and they affect the entire PRINT statement.

TX

This transmits a value to the currently connected interface.

TX AA BB CC

By default, when in packet mode, transmitting something simultaneously clears the receive buffer, assuring serialization of transmit/receive instructions.  In stream mode, the receive buffer is not cleared when transmitting.  These defaults can be overridden with the /CLEAR and /NOCLEAR parameters.

The TX command sets no statuses and will never fail (failure would be an interface issue and abort the program)

STREAM / PACKET

There are two operating modes for receiving data, “Stream” and “Packet”.  Issuing either one as a command will change modes.  This is usually done at the beginning of a program only.

The default is PACKET mode

In STREAM mode, your RX commands will match against the beginning of a single continuous datastream.   Your program will be responsible for figuring out when one message ends and another begins based on its content.  This mode is generally only used when writing drivers.

In PACKET mode, each received message is interpreted as a separate message.  This is designed for linking with other internal programs, as those are generally guaranteed to deliver complete messages.

RX

This places the program into a state where it is waiting for a specific pattern of data.

The program will block until the intended reply is received, or the timeout is reached.

The /FATAL option will cause the lack of a valid reply to be fatal, in other words, automatically terminate the program.  If this option is not set, a status code can be examined.  This cannot be combined with /OR.

The /TAIL option will ignore the end of the data when matching.  This is useful if you know the start of a reply but not the end.  In stream mode this will match the beginning of the stream, and then clear the RX buffer (not a common thing to want)

The /OR option will allow a list of multiple replies.  To use this, specify many replies with /OR and then the final reply will not have the /OR option.  See examples below.  The system result variable ($*) is set to the matched reply, with the first reply being at 0, so you can easily see which reply has been matched.

The /TIMEOUT=1234 option overrides the default timeout.

Input variables can be specified, which means RX data can be trivially copied to a variable.

For example, the following RX command will accept a reply such as 01 FF AA BB CC AB and place AA BB CC in the VAR variable:

RX 01 FF %VAR[3] AB

If other replies would be valid, the /OR specifier can be used to accept those replies and copy data to variables:

RX /OR 01 FF %VAR_A[3] AB

RX /OR 02 FF %VAR_B[3] AA

RX 03 FF EE

CLEARBUF

Transmitting data (as mentioned in the instructions for the TX operation), by default, clears the receive buffer, although this can be disabled.

When the program is stopped, however, data that is received by the program is buffered.

This instruction allows the contents of the buffer to be manually discarded if required.

CLEARBUF

INTERFACE

The interface command provides a way for programs to issue configuration settings such as baud rate to the low level serial interface.

/BAUD=1234 will set the baud rate of a connected serial interface.

Other options will be available later.

Operations – Comparison

Comparisons are the foundation of a language with conditional branching.  The basic premise of comparison in the SDPL language is to use a comparison to set the status register, then run conditional commands based on the contents of the status register (with the IF prefix).

COMPARE

This compares two values and places the result in the status register.  It modifies the LESS/MORE bits but not the OKAY bit.

It requires a | separator between the terms.

This compares the contents of the variable VAR and AABBCC

COMPARE $VAR | AA BB CC

COMPARESIZE

This is much like the compare function but simply compares the size, in bytes, of two values.

COMPARE AA BB | 00 00 00

In this case, AA BB < 00 00 00 as AA BB is two bytes, and 00 00 00 is three bytes.

MATCH

This is an alternative method of comparison.  It requires two terms separated by a | symbol and determines if they are equal, with the exception of any specified input variables in the left term, which are then copied to memory (if a match is successful).

This works the same as an RX operation except it compares memory rather than serial input.

The /TAIL option is also supported, meaning that if the initial bytes match, the remainder of the right term are ignored.

The result of the match is placed in the TRUE/FALSE bit of the status register.

MATCH AA %VAR[1] CC | AA BB CC
IF TRUE PRINT /HEX $VAR

CONTAINS

This checks if one array contains the contents of another array, at any position, and sets the TRUE/FALSE bit of the status register.

It sets TRUE if the RIGHT hand term contains the LEFT hand term.  The following expression would be true:

CONTAINS AA | AA BB CC

The following would be false:

CONTAINS AA BC | AA BB CC

EMPTY

This simply sets the status register’s true/false bit to true if the array is empty (size is zero), and false if it is not.

EMPTY $VARIABLE
IF TRUE PRINT "Variable is empty."

Operation – IF

This examines the status register (OK,LESS,GREATER,CARRY) and runs the specified operation.

This allows many forms of branching.

The following symbols are accepted for an IF:

  • * – Any (This is like not having an IF statement at all)
  • > – Greater than (MORE=1 AND LESS=0)
  • < – Less than (MORE=0 AND LESS=1)
  • <= Less than or equal to (LESS=1)
  • >= Greater or equal to (MORE=1)
  • = Equal to (MORE=0 AND LESS=0)
  • != Not equal to (MORE=1 OR LESS =1)
  • TRUE (OK=1)
  • FALSE (OK=0)
  • OK (OK=1)
  • FAIL (OK=0)
  • CARRY (CARRY=1) (Overflow occurred from a increment or decrement operation)
  • CCLEAR (CARRY=0) (No overflow occurred from a increment or decrement operation)

Here are some examples:

IF > PRINT "GREATER THAN"
IF TRUE GOTO *SOMEWHERE
IF CARRY CALL *SOME_SUBROUTINE

One useful trick is that many commands do not set the status register at all, so many commands can be chained on the same condition without a subroutine.

COMPARE $VARIABLE TO FF
IF = PRINT "THE VARIABLE IS FF"
IF != PRINT "THE VARIABLE IS NOT FF"
IF = GOTO *SOMEWHERE
IF != GOTO *SOMEWHERE_ELSE

Math

INC

Increments the variable specified.  This results in VAR equaling AC.  The carry register (both LESS and MORE) are set if the result rolled over.

SET %VAR AB

INC %VAR

DEC

Decrements the variable specified.  Syntax is the same as INC.

ADD

Performs 8 bit addition on some memory.

No carry is performed, and the memory size is never altered, however if ANY byte in the array produces a carry, the CARRY status flag is set (otherwise it is cleared).  If the input array is not the correct size, the addition is left aligned.

ADD %VAR 10

File Operations

LOAD

Load a file into a variable.  The filename is relative to the root program directory unless the path is specified.

Failing to load the entire file is a fatal program error, and will result in the program failing automatically and exiting, so there is no need to worry if $DATA contains the entire contents of the test file.

LOAD %DATA "TEST_FILE.TXT"

SAVE

Save data to a file.  This overwrites the contents of the file.  The filename must be separated by a | symbol.

SAVE "TEST_FILE.TXT" | AA BB CC

Operations – Misc

SLEEP

Sleep for the specified milliseconds.  This will sleep for 2 milliseconds.

SLEEP 02

DEBUG

The debug command immediately pauses the program, at which point it can be stepped through manually by a debugger.  It also enables the debug signals for the debugger, which send a snapshot of the status flags and memory after each instruction is executed.

DEBUG

EXIT

Exit the running program with success.

FAIL

Exit the running program with failure.