REBOL 3 Concepts: Protocols: REBOL Networking Basics

Pending Revision

This document was written for R2 and has yet to be revised for R3.

Contents

Modes of Operation

There are two basic modes of network operation: atomic and port-based.

''Atomic network operations are those that are accomplished in a single function. For instance, you can read an entire Web page with a single call to the read function. There is no need to separately open a connection or set up the read. All of that is done automatically as part of the read. For example, you can type:

print read http://www.rebol.com

The host is found and opened, its Web page transferred, and the connection closed.

The port-based mode of operation is one that uses a more traditional programming approach. It involves opening a port and performing various series operations on the port. For instance, if you want to read your email from a POP server one message at a time, you would use this method. Here is an example that reads and displays all of your email:

pop: open pop://user:pass@mail.example.com
forall pop [print first pop]
close pop

The atomic method of operation is easier, but it is also more limited. The port-based method allows more types of operations, but also requires a greater understanding of networking.

Specifying Network Resources

REBOL provides two approaches for specifying network resources: URLs and port specifications.

''Uniform Resource Locators (URL) are used on the Internet to identify a network resource, such as a Web page, FTP site, email address, file, or other resource or service. URLs are integral to the operation of REBOL, and they can be expressed directly in the language.

The standard notation for URLs consists of a scheme followed by a specification:

scheme:specification

The scheme is often the name of a protocol, such as HTTP, FTP, SMTP, and POP; however, that is not a requirement. A scheme can be any name that identifies the method used to access a resource.

The format of a scheme's specification depends on the scheme; however, most schemes share a common format for identifying network hosts, user names, passwords, port numbers, and file paths. Here are a few commonly used formats:

scheme://host

scheme://host:port

scheme://user@host

scheme://user:pass@host

scheme://user:pass@host:port

scheme://host/path

scheme://host:port/path

scheme://user@host/path

scheme://user:pass@host/path

scheme://user:pass@host:port/path

[bad-link:concepts/network.txt] Resource Specification lists the fields used in the above formats.

scheme	The name used to identify the type of resource, often the same as the protocol. For example, HTTP, FTP, and POP.
host	The network name or address for a machine. For example, www.rebol.com, cnn.com, accounting.
port	Port number on the host machine for the scheme being used. Normally there is a default for this, so it is not required most of the time. Examples: 21, 23, 80, 8000.
user	A user name to access the resource.
pass	A password to verify the user name.
path	A file path or some other method for referencing the resource. This is scheme dependent. Some schemes include patterns and script arguments (such as CGI).

Another way to identify a resource is with a REBOL port! specification. In fact, when a URL is used, it is automatically converted into a port specification. A port specification can accept many more arguments than a URL, but it requires multiple lines to express.

A port specification is written as an object block definition that provides each of the parameters necessary to access the network resource. For instance, the URL to access a Web site is:

read http://www.rebol.com/developer.html

but, it can also be written as:

read [
    scheme: 'HTTP
    host: "www.rebol.com"
    target: %/developer.html
]

The URL for an FTP read can be:

read ftp://bill:vbs@ftp.example.com:8000/file.txt

but, it can also be written as:

read [
    scheme: 'FTP
    host: "ftp.example.com"
    port-id: 8000
    target: %/file.txt
    user: "bill"
    pass: "vbs"
]

In addition, there are many other port fields that can be specified, such as timeout, type of access, and security.

Schemes, Handlers, and Protocols

REBOL networking operates by using schemes to identify handlers that communicate with protocols.

In REBOL a scheme is used to identify the method of accessing a resource. That method uses a code object that is called a handler. Each of the URL schemes that are supported by REBOL (such as HTTP, FTP) has a handler. The list of schemes can be obtained with:

probe next first system/schemes
[default Finger Whois Daytime SMTP POP HTTP FTP NNTP]

In addition, there are lower level scheme names that are not shown here. For instance, the TCP and UDP schemes are used for direct, lower level communication.

New schemes can be added to this list. For instance, you can define your own scheme, called FTP2, that provides special features for FTP access, such as automatically supplying your username and password so it does not need to be included in every FTP URL.

Most handlers are used to provide an interface to a network protocol. A protocol is used to communicate between various devices, including clients and servers.

Although each protocol is quite different in how it communicates, it does have some things in common with other protocols. For instance, most protocols require a network connection to be opened, read, written, and closed. These common operations are performed by a default handler in REBOL. This handler makes protocols like finger, whois, and daytime almost trivial to implement.

Scheme handlers are written as objects. The default handler serves as the root object for all the other handlers. When a handler requires a particular field, such as a timeout value to use for reading data, if the value is not defined in the specific handler, it will be provided by the default handler. Hence, handlers overlay one another with their fields and value. You can also create handlers that use other handlers for default values. For instance, you can create an FTP2 handler that looks for missing fields first in the FTP handler, then in the default handler.

When a port is used to access network resources, it is linked to a specific handler. The handler and the port together form the unit that is used to provide the data, code, and state information to process all protocols.

The source code to handlers can be obtained from the system/scheme object. This can be useful if you want to modify the behavior of a handler or build your own handler. For instance, to view the code for the whois handler, type:

probe get in system/schemes 'whois

Note that what you are seeing is a composite of the default handler with the whois handler. The actual source code that is used to create the whois handler is only a few lines:

make Root-Protocol [
    open-check:  [[any [port/user ""]] none]
    net-utils/net-install Whois make self [] 43
]

Monitoring Handlers

For debugging purposes, you can monitor the actions of any handler. Each handler has its own debugging output to indicate what operations are being performed. To enable network debugging, turn network tracing on with the line:

trace/net on

To turn network debugging off, use:

trace/net off

Here is an example:

read pop://carl:poof@zen.example.com
URL Parse: carl poof zen.example.com none none none
Net-log: ["Opening tcp for" POP]
connecting to: zen.example.com
Net-log: [none "+OK"]
Net-log: {+OK QPOP (version 2.53) at zen.example.com starting.}
Net-log: [["USER" port/user] "+OK"]
Net-log: "+OK Password required for carl."
Net-log: [["PASS" port/pass] "+OK"]
** User Error: Server error: tcp -ERR Password supplied for "carl"
is incorrect.
** Where: read pop://carl:poof@zen.example.com