REBOL External Library Interface

SDK Documentation

Contents

Overview

This chapter describes the purpose of external libraries and how to include the functionality of external libraries in REBOL programs.

What are External Libraries?

External libraries are groups of executable C or C++ functions that can be dynamically accessed by applications. Most operating systems provide some external libraries that contain critical functions for use by the operating system, as well as by applications. External libraries provided by operating systems typically provide low-level support for operating system objects such as processes, threads, windows, communications, synchronization, and database access. Applications can also provide external libraries that contain functions for accessing data and functionality of the application.

File name extensions and locations of external libraries vary between operating systems. For example, in the Microsoft Windows operating systems, external libraries generally have file name extensions of .dll and are frequently located in the /Windows/System or /WinNT/System32 directories. In most UNIX operating systems, external libraries have file name extensions of .so or .dso and frequently reside in the /lib or /usr/lib directories. In BeOS the file name extension is typically .so and libraries are located in the /system/lib directory. In AmigaOS the file name extension is .library and libraries typically reside in the LIBS: directory. Refer to the operating system documentation for more information about the libraries provided with your specific operating system.

External Libraries and REBOL

The REBOL/Command External Library Interface enables you to access external libraries and integrate their functionality into REBOL applications.

REBOL/Command converts C datatypes to and from REBOL datatypes and provides the functionality to evaluate and manipulate C datatypes.

NOTE: Since external library functions are usually operating-system-specific, use of these functions can limit the portability of REBOL applications.

New REBOL Datatypes

In addition to the standard REBOL/Core datatypes, REBOL/Command provides these new datatypes that are used for accessing external libraries:

Datatype	Description
library!	Refers to an opened external library
struct!	Provides a REBOL equivalent to the C-language struct datatype and is used in the specification of the REBOL routine! datatype
routine!	Defines the interface between the REBOL/Command interpreter and an external library function

Summary of the Library Interface

The following steps provide an overview of using external libraries in REBOL applications. These steps are described in more detail in the remaining sections of this chapter.

1. Load the external library.

Use load with the /library refinement to load the external library; this returns a library! datatype. See Loading External Libraries for more details.

2. Define a routine to access the external library.

Use make routine! to import a function from an external library. This returns a routine! datatype. See Defining Routines and Using Structures for details.

3.Evaluate the routine.

Call the external library function and pass the appropriate arguments. See Defining Routines for details.

4.Free the library resources.

Use free to free the library and associated resources when done. See Freeing Library Resources for details.

Examples of the Library Interface

The following example loads an external library on UNIX and calls one of its functions:

stdclib: %/lib/libc.so.6

getpid: "getpid"

clib: load/library stdclib

pid: make routine! [
    "Get process ID"
    return: [long]
] clib getpid

print pid

In the above example, the external library libc.so.6 is loaded and defined as the REBOL word clib. A REBOL routine is defined using make routine!. When evaluated, this routine calls an external library function that returns the process ID of the currently running process, which, in this case, is the REBOL application.

The REBOL make routine! expression defines the return datatype (long), as well as the library! value that represents the external library ( clib ), and the name of the external library function to be called (getpid). In the last line, the routine is evaluated and printed.

Similarly, the next example loads a Windows external library and uses one of its functions:

stdclib: %msvcrt.dll

getpid: "_getpid"

clib: load/library stdclib

pid: make routine! [
    "Get process ID"
    return: [long]
] clib getpid

print pid

Loading External Libraries

To use an external library function, begin by loading the specific library containing the functions you want to use. You load the target library by using load with the /library refinement, using the following syntax:

load/library library-name

where library-name is the name or the absolute file path of the external library that you want to load.

When the external library is found and successfully loaded, load returns a library! datatype, which contains the name of the loaded library. If you only specify the file name in load/library (instead of the absolute file path), the operating system searches for the library in the current directory, then in the system library directories.

The load native's /header, /next, and /markup refinements are ignored when used with the /library refinement.

You can also use make to load an external library. The syntax for loading a library with make is as follows:

lib-name: make library! library-name

where

• lib-name is the REBOL word defined as the loaded library.
• library-name is the actual name of the library to be loaded.

On UNIX and Windows, external libraries can be loaded without specifying the file name extension.

For UNIX and BeOS to find a library file in the current directory, period forward-slash (./ ) must be prefixed to the file name. The following example shows how an external library can be loaded from the current directory on UNIX:

clib: make library! %./mylibc.so

If period forward-slash (./ ) is omitted from the file name, the library is loaded only if it is in the operating system's library search path. Refer to the operating system documentation for more information about the library search path.

For AmigaOS REBOL attempts to load a library from LIBS: if it cannot load it from the path indicated. In addition to the AmigaOS shared library itself REBOL also needs a "Function Description" .fd file. It has to be located in the same directory as the library or in the FD: directory. .fd files for libraries that are part of AmigaOS are available from Aminet. .fd files for third-party libraries are usually available from the author of the library.

Examples of Loading External Libraries

The following examples show how to use load to load external libraries in a Microsoft Windows and UNIX environment.

Windows Example

This example loads the Microsoft Windows Graphics Display Library from a Windows 98 operating system using the absolute file path as the library-name.

gdi-lib: load/library %/c/windows/system/gdi32.dll

UNIX Example

In this example, the standard UNIX math library is loaded using the absolute file path as the library-name.

mathlib: load/library %/usr/lib/libm.so

Library Datatype Check

To determine whether an object is a library! datatype, use one of the following methods:

1. Use the library? action, which returns true if an object is a library! datatype and false if it is not. In the following example, the object, gdi-lib, is determined to be a library! datatype.

library? gdi-lib

== true

2. Use the type? action, which returns library! if an object is a library! datatype. In the following example, the object, mathlib, is determined to be a library! datatype.

type? mathlib

== library!

NOTE: Even though a library is considered to be a file on most operating systems, the file? action returns false on a library! datatype.

Returning the Path of a Loaded Library

To return the path of a loaded library, use first, as shown in the following example:

first mathlib

== %/usr/lib/libm.so

NOTE: Only the path or file name specified in load/library is returned by first.

Library Load Errors

If load/library generates an error, a message is sent to the console, which is valuable in the debugging process. The message content varies depending on the operating system. Refer to your operating system documentation for information about specific dynamic load error messages.

Freeing Library Resources

When you are done using an external library, you can release it. Loaded external libraries and all resources referenced by them remain in memory until you release the library. Use free to release all memory and any other resources that may have been allocated by the library.

Once an external library is freed, its functions are no longer available to REBOL applications. If a REBOL program attempts to access the functions of a freed external library, an access error ( library not open ) is generated.

NOTE: Some external library functions maintain pointer references to passed in values, such as structures. Attempting to access memory referenced by a function of a freed external library can crash REBOL/Command.

Using Structures

External library functions frequently expect to be passed a C struct as well as natural C datatypes. To accommodate this, REBOL/Command provides the struct! datatype which approximates the use of a C struct . In addition to representing the C struct datatype, a struct! value is used as the argument specification for REBOL routines (see Defining Routines for more details).

The following example shows the syntax for creating a struct! datatype:

struct-name: make struct! spec data

where

• struct-name is the REBOL word defined as the structure.
• spec is the specifications for creating the struct! datatype
• data contains the initial values for the items in the structure. The data argument is specified by declaring values within a REBOL block or using the none value.

The following example shows the complete specification for a struct! datatype:

make struct! [
    "description-string"
    [attributes]
    argument [datatype] "description-string"
] [
    Block of initial values
]

The description-string is an optional argument that describes the structure and its use. Use the optional attribute block to specify the save attribute. (See Garbage Collection Issues for information about the save attribute.) Each argument, which represents an element of the structure, must be followed by a block defining the datatype and can also be followed by a description string. The final block in the structure specification contains the initial values for each element of the structure.

The initialization of the values in a structure can be simplified using the none value, as shown in the following example:

a-struct: make struct! [
    number [integer!]
    character [char!]
] none

In this example, the first argument is an integer! datatype and the second argument is a char! datatype. Both arguments are initialized with none, which assigns a 0 (zero) value to the integer! and a NULL character to the char!. Using the none value enables you to initialize an entire structure with one value, regardless of the datatype of each structure element. To initialize the structure elements with a value other than none, you must supply a block of values, one for each element in the structure, as shown in the following example:

a-struct: make struct! [
    in-string [string!] "Input string"
    in-int [integer!] "Input integer"
] ["This is input" 42]

NOTE: An initial value must be supplied for every element of the struct! datatype. Attempts to initialize just portions of a struct! datatype result in errors.

Accessing Structure Elements

The individual elements of a structure can be evaluated and assigned values as needed. When getting or setting an individual element's value, specify the full path of the element, as shown in the following examples:

a-struct/in-string

== "This is input"

a-struct/in-int: 56

== 56

print a-struct/in-int

== 56

In addition to accessing the individual elements of a structure, separate components of the structure can also be examined. These components are useful for understanding the specification, data, number of elements, and size of the struct!. All of these attributes can be queried by the user. These components all may be obtained using first, second, and third, as shown in the following examples:

first a-struct ;- returns the specification

== [in-string [string!] "Input string" in-int [integer!] "Input integer"]

second a-struct ;- returns the data as a block

== ["This is input" 56]

third a-struct ;- returns the data in binary form

== #{F00E1A0838000000}

Determining the Number of Elements and Size

Knowing the size of a structure is important when passing it to external library functions. External library functions frequently require that the size of the structure be passed as a separate argument. Using second and third in combination with length?, you can determine the number of elements in and the size of a structure, as shown in the following examples:

;- returns the number of elements in the structure.
length? second a-struct

== 2

;- returns the size, in bytes, of the structure.
length? third a-struct

== 8

Using Structures as Templates

Structures can be used as templates for creating other structures, as shown in the following example:

town: make struct! [
    name [string!]
    year [integer!]
] none

town1: make struct! town ["Eureka" 1822]

In the previous example, the specification for town is used as a template for town1.

Creating Nested Structures

Structures can be nested within other structures. To nest a structure, specify the nested structure within a structure's format block, as shown in the following example:

town: make struct! [
    name [string!]
    info [struct! [
        day [integer!]
        month [integer!]
        year [integer!]
    ]
]
] ["Albuquerque" 27 9 1942]

In this example, notice that the initialization block is not nested. Albuquerque is the initial value for name, which is in the main structure. The values 27, 9, and 1942 are the initial values for day, month, and year, which are in the nested structure.

Just as with normal structures, you can access the individual elements of a nested structure using paths, as shown in the following examples:

town/info/year

== 1942

town/info/day: 22

== 22

town/info/day

== 22

In the previous examples, the specified paths access the year and day elements of the info structure, which is nested within the town structure.

In the next example, the path for the nested structure, info, is defined as the word info. The word info is then used as a shortcut for accessing the nested structure's elements.

info: town/info
info/month: 10

== 10

town/info/month

== 10

Obtaining a Structure's Specification

You can obtain the specification of a structure using mold, which shows the structure specification followed by a block of the structure values, as shown in the following example:

mold a-struct

=={make struct! [in-string [string!] "Input string"
in-int [integer!] "Input integer"] ["This is input" 42]}

Obtaining the structure specification with mold enables you to serialize (save to disk or pass around a network) structure values, and load them back into REBOL at a later time while preserving the values of the structure.

Defining Routines

Once an external library is loaded, you can access the library functions by defining a REBOL routine using make routine!. A routine is similar to a function. When evaluated, a routine translates the REBOL arguments into the datatypes expected by the C function, then it calls the C function passing the translated datatypes as arguments. The results of the C function are also translated back into a valid REBOL value.

NOTE: REBOL routines have self-documenting characteristics similar to functions and you can use help to retrieve information about the routine.

When defining a routine, you must

• Name the routine that evaluates the external library function.
• Identify the external library function to call.
• Declare datatypes of the arguments of the library function along with any return value.

The following example shows the syntax for defining a routine:

routine-name: make routine! specs library func-id

where

• routine-name is the name of the routine.
• specs is either a block or a structure that provides a specification similar to those used for REBOL functions.
• library is the value of the library! datatype returned from load/library.
• func-id is a case-sensitive string identifying the function to be called from the external library. To determine the correct function name, consult the documentation for your operating system or external libraries.

A routine! can return a struct! if the external library function returns a pointer to a struct. To receive a struct! value back from a routine, the returned struct! value must be fully specified. For example:

ret-struct: make routine! [
    return: [struct! [a [int] b [string!]]]
] lib "ret_struct"

In the above example, the routine returns a struct! containing and integer! and a string!. The respective library function, ret_struct, passes back a pointer to a struct containing an int and a char.

Routine Specification Block

The specification argument is either a structure or a block that describes the arguments passed to an external library function. If the specification is a block, REBOL translates it into a structure in the process of creating the routine and initializes the values to none. The routine specification is similar to the specification block used in defining all REBOL functions, as shown in the following example:

routine-name: make routine! [
    "description-string"
    argument [type] "Arg description string"
    return: [type]
] library func-id

The description-string is an optional specification element that describes the purpose of the external library function. Each argument in the specification must be followed by a block containing the argument datatype and can optionally be followed by a description string. The optional return: argument identifies the expected return value from the external library function. If declared, the return: argument must be followed with a block containing the return type. If the type of return value is not specified, then no value is returned to the REBOL script, even if the external library function normally provides a return value.

In the following example, the structure specification is created first and its values are initialized with none, then the structure is used to define the routine, getenvvar.

get-struct: make struct! [
    "Get an environment variable."
    in-string [string!] "Name of an environment variable"
    return: [string!]
] none

get-env: make routine! get-struct clib "getenv"

In the next example, a block is used as the specification argument, instead of a structure. REBOL automatically translates the block into a structure during creation of the routine:

get-env: make routine! [
    "Get an environment variable."
    in-string [string!] "Name of an environment variable"
    return: [string!]
] clib "getenv"

If the routine is passed an incorrect datatype, the routine returns an error message, as shown in the following example:

get-env 1234

*** Script Error: expected one of: string! - not: integer

Specifying Datatypes in Routines

In routine specification blocks, arguments can be declared using REBOL datatypes, as well as standard C language datatypes. Routines support only the datatypes shown in C to REBOL Datatype Conversions and REBOL to C Datatype Conversion tables below. These tables also show how each datatype is converted.

C to REBOL Datatype Conversions:

C Datatypes	REBOL Datatypes
char	char!
short	integer!
long	integer!
int	integer!
float	decimal!
double	decimal!
struct	struct!
char	string!

REBOL to C Datatype Conversion:

REBOL Datatypes	C Datatypes
char!	char
integer!	long
decimal!	double
struct!	struct

NOTE: While the C datatype void is not supported in routine! and struct!, char can be successfully substituted in its place.

Datatype Conversion Issues

REBOL/Command provides consistent and accurate conversion between REBOL datatypes and C datatypes. However, truncation, rounding, and precision errors can occur when converting datatypes that do not have the same bit precision in C as they do in REBOL.

Using C Pointers

Pointers reference memory locations allocated by external library functions. When an external library function returns pointers to REBOL, you can define the pointers as REBOL words and use them as arguments to other routines and functions.

External library functions that are passed pointers, generally expect the pointers to be char or void datatypes. Currently, REBOL routines do not support the void datatype. In most cases, you can safely pass char instead of void. For return values, use long instead of void.

When you pass pointers to an external library function, the function can change the values of the pointers. To determine which pointer values were changed by an external library function, use the following method:

• Use second on the routine! value, which returns a structure.
• Then, use first on the struct! returned from second. This returns a block of the potentially changed values that were passed to the external library function.

Obtaining a Routine's Specification

To obtain the specification of a routine, use any of the following methods:

1. Use first, which returns the routine specification as a struct! value, as shown in the following example:

myroutine: make routine! [
    "Do something interesting"
    Arg1 [string!] "Input String"
    Arg2 [integer!] "A number"
    return: [integer!]]
] lib "libroutine"

print mold first :myroutine

make struct! [
    "Do something interesting" Arg1 [string!]
    "Input String" Arg2 [integer!]
    "A number" return: [integer!]
] [none 0]

2. Use third, which returns the routine specification as a block, as shown in the following example.

print mold third :myroutine

[
"Do something interesting"
Arg1 [string!] "Input String"
Arg2 [integer!] "A number"
return: [integer!]
]

Routine Datatype Check

To determine whether an object is a routine! datatype, use one of the following methods:

1. Use routine?, which returns true if the value is a routine! datatype. For example:

routine? :fun

== true

2. Use the type? action, which returns the datatype of the object. For example:

type? :fun

== routine!

3. Use any-function?, which returns true if the object is a routine! data type. For example:

any-function? :fun

== true

Garbage Collection Issues

Some external library functions that are passed pointers take ownership of the memory referenced by those pointers. When an external library function takes ownership of memory, you may need to protect the REBOL values in those memory locations from the REBOL garbage collector. For example, if a REBOL word that was defined as a struct! is reused to make another struct!, the memory used by the first struct! becomes unreferenced. Any unreferenced struct! values may be reclaimed by the REBOL garbage collector, even though the memory is owned by an external function.

To save a REBOL structure from the garbage collector, include the word save in the struct! attribute block. The save attribute protects a REBOL structure from the garbage collector even when the structure is unreferenced. The struct! attributes are defined in an optional block placed at the beginning of the struct! specification, as shown in the following example:

struct: make struct! [
    "Description..."
    [save] ;- A protected structure
    name [char*] "Name's help string"
]["luke"]

If the save attribute is not specified as shown in the previous example, the garbage collector eventually frees the memory when nothing refers to the struct! value. Accessing a structure that has been freed by the garbage collector may cause REBOL/Command to crash.

Integrating External Libraries

You can develop REBOL mezzanine function shortcuts that simplify the definition of routines that call external library functions. Functions similar to func and function can be used in the following manner:

routine: func [specs lib id] [make routine! specs lib id]

This routine can be used to define new library functions:

fun: make routine! [argument [type]] lib "libfun"

A helper function can be created to define multiple functions. For example:

gfx-lib: load %gfx.dll
func-gfx: func [specs identifier][
    make routine! specs gfx-lib identifier
]
draw-line: func-gfx [...] "DrawLine"
draw-rect: func-gfx [...] "DrawRect"

In this example, the gfx-lib library is used to create each function using the func-gfx function.

Another way to create multiple functions for a library is shown in the following example:

gfx-funcs: [
    draw-line [specs...] "DrawLine"
    draw-rect [specs...] "DrawRect"
    draw-circle [specs...] "DrawCircle"
    ...
]

foreach [name specs lib-fun] gfx-funcs [
    set name func-gfx specs lib-fun
]

In this example, the function name, specifications, and library function name are stored in gfx-funcs. The foreach loop reads three items at a time setting the first item to the routine! value returned by func-gfx. The second and third items define the routine specification and external library function name, respectively.

Issues with Microsoft Windows

When using the External Library Interface to access Windows libraries, be aware of the following issues:

• Microsoft Windows uses wrapper functions in many external libraries. Wrapper functions frequently hide the actual function name making it difficult to find and load the desired functions. Many of the actual library function names specified in the Windows documentation end with an A or a W. For example, the Windows function that creates a directory is documented as CreateDirectory. However, the actual library function corresponding to this wrapper function is either CreateDirectoryA or CreateDirectoryW. Microsoft does this to support both ANSI and wide character compatibility in certain functions. Consult the Microsoft Windows documentation for the actual names of the functions you want to use.
• The Windows function for creating a window, CreateWindow, requires both the class name and the window name to create a child window. In a REBOL application, the resulting window would be a child to the main REBOL console, which is opened by REBOL/Command when the application is started. To successfully specify the class name and window name, use the string REBOL as both the class name and window name in the CreateWindow call.