REBOL 3 Concepts: Extensions: Callback Functions

Callbacks let you invoke REBOL-based functions from the C code of an extension module. This can be useful if you use operating system or other external libraries that require callbacks to process results or signal completion of external operations.

Two callback mechanisms are provided:

Synchronous	Where the callback function in invoked immediately.
Asynchronous	Where the callback function is queued as an event and will be processed later.

Synchronous Callbacks

A synchronous callback passes control back through the REBOL Extension API, which generates the necessary stack frame and re-enters the interpreter's function evaluator. It's like calling a function via apply.

With this kind of callback, you can allocate the required structures on the C stack, and pass them to the callback. But, be aware that the C stack holds data from both the REBOL and C environments. An exception, such as an ERROR in the callback, will throw the C stack back to a prior location, perhaps even beyond where your extension code was running. If your C code had any cleanup or deallocation to do, it won't happen.

Asynchronous Callbacks

An event can be used to trigger a callback in an asynchronous fashion. This method allows C code to use C-based callbacks within an operating system that execute callbacks in a fashion similar to an interrupt.

Your C code calls the extension API which builds an event to hold callback information. The event is queued to the System Port's event list. Later, when events are processed, the event is passed to the Callback Port which will invoke your callback function.

When running asynchronous, the callback function return values is stored in a structure that you allocated earlier, but if you need results on the REBOL side, the callback function must be store it in an object or block for later retrieval.

It should be mentioned that REBOL I/O devices (stdio, files, networking, timers, GUI events) use the same event-driven mechanism for all asynchronous I/O operations. That means that events are queued in the order they happen, including callback events. That's generally considered the best method, otherwise it's possible for a specific callback to be called before its related I/O may have finished.

Example Callbacks

An example callback is included in the host-ext-test.c file within the R3 host-kit distribution. This file includes tests for various extension functions including both callback mechanisms.

To enable the test code add host-ext-test.c to your makefile and within host-main.c uncomment this line:

#define TEST_EXTENSIONS

Build the host, run it, and at the REBOL prompt type:

xtest

to run the test code that is found within the host-ext-test.c file. A few callbacks will be called as part of the test. The wait that is called at the end of the test allows event-driven callbacks to be processed before the test returns to the console (which blocks, and does not let events happen.)

Note: this file does not use the make-host-ext.r script in order to make it clear what the test code is actually doing in the C file.

Details of a Callback

The basic callback mechanism involves setting up to call this specific RXI API function:

n = RL_CALLBACK(cbi);

Here, the cbi argument is a RXICBI structure that includes:

obj	the object context of the function
word	the function's name, a word id
args	the RXIARG argument array to hold input arguments
result	space to hold the result of the callback
flags	for special options (and future controls)

The way this structure is allocated is important, depending on what type of callback you are using. For asynchronous callbacks, this structure must not be local to the stack, because the stack frame is popped before the function is run. It's best to allocate it in memory.

The RL_CALLBACK function returns an unsigned integer:

For a synchronous function, this is the datatype enum of the returned value.
For an asynchronous function, this value is simply TRUE or FALSE. If FALSE, it means that the event could not be queued (most likely because the event queue has reached its maximum.)

Allocating the CBI

As mentioned earlier, the way you allocate the CBI (callback information structure) is important.

The most general way is to allocate it from the heap. You can use it once and free it, or store it for reuse multiple times (as long as not recursively, of course.)

Here's an example that allocates the CBI and an argument frame for NARGS of args:

#define NARGS 4
RXICBI *cbi;

cbi = MAKE_NEW(*cbi);
CLEAR(cbi, sizeof(cbi));

cbi->args = MAKE_MEM(sizeof(RXIARG) * NARGS);
CLEAR(cbi->args, sizeof(RXIARG) * NARGS);

Of course, you could also allocate it on the stack for calling synchronous callbacks:

#define NARGS 4
RXICBI cbi;
RXIARG args[NARGS];

CLEAR(&cbi, sizeof(cbi));

cbi.args = &args[0];
CLEAR(cbi.args, sizeof(args));

Setting the Function

To set the function, your C code must have a pointer to the object where it resides (an object you made or some other system context) and the name of the function, as a word id.

Set those in the CBI (using the allocated form of the CBI structure):

cbi->obj = obj;
cbi->word = word;

Editor note: Add a note about why this technique is used.

Setting the Mode

Next, if your callback must run as a queued event, you will need to set a flag:

SET_FLAG(cbi->flags, RXC_ASYNC);

Setting the Arguments

The argument array has exactly the same format as other extension command! arguments. You will need to set the argument count, the datatypes, and the values.

Here's an example that puts a single integer! argument into the argument frame:

RXI_COUNT(args) = 1;
RXI_TYPE(args, 1) = RXT_INTEGER;
args[1].int64 = 123;

Calling the Function

You're now ready to call the function or queue it (if it's asynchronous):

n = RL_CALLBACK(cbi);

If n is zero, then an error has occurred, and you may need to process it.

Return Value

The return value is stored in the cbi result field. If this value is a series or other non-immediate datatype, then it is volatile and may be garbage collected if you don't store it in a block or object that is referenced from some other part of your code.

Deallocating the CBI

If you no longer need the CBI or argument array, you can free them or add them to your own pool for later reuse.

For synchronous callbacks, you can do this anytime after the callback has returned:

free(cbi->args);
free(cbi);

To help manage the pooling of asynchronous CBI data, a CBI flag is set once the callback has completed and the data is no longer in use. Code like this could be added to a pooling check function:

if (GET_FLAG(cbi->flags, RXC_DONE)) {
    free(cbi->args);
    free(cbi);
}

Don't forget to copy out your result if you need to use it later.

Special Issues

Allocation and GC

REBOL uses automatic memory management, so REBOL and C code cannot be directly merged or referenced.

In addition, if the C code allocates REBOL series such as a string, image, or block, that memory is "volatile" because without proper referencing, it can be targeted for garbage collection.

Struct Passing

C structs and REBOL are not directly compatible. Therefore, it is the responsibility of the C extension code to deal with the conversion of values between the two domains.

For example, a callback may need to return multiple values to REBOL. This can be done with a REBOL block or with an object. The allocation of the block or object can be passed in from REBOL, done in advance by the C command, or done dynamically within the callback, assuming that it has access to the R3 Extension Lib API.

Editor note: Add some notes about event handling... when it happens, etc.

Clips pending

Some things to keep in mind:

Garbage collection can occur when evaluating the callback. If you created

invalidating any REBOL series that may have been created exclusively in the C domain and not otherwise referenced from REBOL.

It should be noted that good coding practices can prevent most of these problems.

The arguments to the function are passed in a standard REBOL extension format, and the result is returned in that same format. Simple immediate values and series values with offsets can be passed and returned. More complex datatypes cannot be passed, and when required, must be handled with context references.

The disadvantage of this method is that a programmer must be careful not to evaluate code that may conflict with the callback function.