Handles, Pipes and Devices

VIOS borrows a small idea from Unix and Plan 9: files, pipes, the console and your own services are all reached through handles with the same four words, OPEN, READ, WRITE and CLOSE. A task's standard input and output can be any handle, so small words can be joined into pipelines.

The handle model

A handle is a small number, 1 to 8. Each one refers to one of:

Kind Opened with Notes
file OPEN of an SD path such as /VIOS/LOG.TXT 4 file streams at a time; see Files and the SD Card
host OPEN of /host/... a file on the Mac, served by the harness over USB
pipe PIPE, or OPEN of a user device 128-byte buffer, 4 pipes at a time
null OPEN of /dev/null reads end at once, writes vanish
cons OPEN of /dev/cons (write only) bytes go to the console
snapshot OPEN of /dev/tasks (read only) a text list of the task slots

Every handle has a direction, given when it is opened: mode 1 reads, mode 2 writes. There is no read-write mode and no seek; a file opened for writing is created or truncated.

Every handle also has an owner: the task that opened it (0 is the foreground). When a task ends, is killed or fails, VIOS closes every handle it owns. Handles opened at the console stay open until you close them.

Limits: 8 handles, 4 SD file streams, 4 pipes, 4 user devices, paths up to 127 printable characters (no spaces).

OPEN, READ, WRITE, CLOSE

Word Stack Result
OPEN ( addr u mode -- h ) handle 1..8, or a negative error code
READ ( addr u h -- n ) bytes read, 0 at end of input, or a negative error code
WRITE ( addr u h -- n ) bytes accepted (may be fewer than u), or a negative error code
CLOSE ( h -- n ) 0, or a negative error code

These primitives never block. Error codes are small negative numbers:

Code Meaning
-1 try again: the pipe is empty (read) or full (write)
-2 broken pipe: no reader is left
-3 bad handle: not open
-4 nothing free: no handle, pipe, file stream or task slot
-5 path: invalid path, file not found, no such device
-6 I/O error reported by the SD card or the host
-7 wrong mode or direction

The text of the last error is shown by HANDLES as io-error ....

Write a file and read it back:

CREATE BUF 128 ALLOT
S" /VIOS/NOTE.TXT" 2 OPEN CONSTANT H
S" remember the milk" H WRITE .
H CLOSE .
S" /VIOS/NOTE.TXT" 1 OPEN CONSTANT H2
BUF 128 H2 READ DUP .  BUF SWAP TYPE CR
BUF 128 H2 READ .
H2 CLOSE .
17 0 17 remember the milk
0 0

The second READ returns 0: end of file.

Blocking: READ-WAIT and WRITE-ALL

Blocking is a policy, so it lives in Forth on top of the primitives, and you can redefine it:

  • READ-WAIT ( addr u h -- n ) calls READ; while it returns -1 it calls PAUSE and tries again. It returns the byte count, 0 at end of input, or another error code.
  • WRITE-ALL ( addr u h -- n ) writes until all u bytes are taken, calling PAUSE whenever the pipe is full. It returns 0, or the first error code.
: READ-WAIT >R BEGIN 2DUP R@ READ DUP -1 = WHILE DROP PAUSE REPEAT R> DROP >R 2DROP R> ;

Because they PAUSE, other tasks run while you wait.

Pipes

PIPE ( -- r w ) creates a pipe and returns its read handle and its write handle (write handle on top). If nothing is free it returns an error code and 0 instead.

PIPE CONSTANT W CONSTANT R
S" hello" W WRITE .
BUF 128 R READ .  BUF 5 TYPE CR
BUF 128 R READ .
W CLOSE .
BUF 128 R READ .
R CLOSE .
5 5 hello
-1 0 0 0

The pipe holds up to 128 bytes. Reading an empty pipe returns -1 while a writer exists and 0 (end of input) once the last writer has closed. Writing to a pipe whose reader has closed returns -2, broken pipe:

PIPE CONSTANT W2 CONSTANT R2
R2 CLOSE .
S" x" W2 WRITE .
W2 CLOSE .
0 -2 0

Task input and output

TASK-OUTPUT ( id h -- ) makes h a task's standard output; TASK-INPUT ( id h -- ) makes h its standard input. The task then owns the handle. Its EMIT, TYPE, . and CR go to the output handle (a full pipe makes the task wait), and its KEY reads the input handle (an empty pipe makes it wait; end of input gives -1).

Wire a task right after SPAWN, inside CRITICAL( ... )CRITICAL, so that it cannot run before it is connected:

: NUMBERS 10 0 DO I . LOOP CR ;
PIPE CONSTANT PW CONSTANT PR
CRITICAL( ' NUMBERS SPAWN DUP PW TASK-OUTPUT )CRITICAL DROP
BUF 128 PR READ-WAIT  BUF SWAP TYPE
BUF 128 PR READ-WAIT .
PR CLOSE .
0 1 2 3 4 5 6 7 8 9
0 0

When NUMBERS returns, its task ends and VIOS closes the write end it owned, so the second READ-WAIT sees end of input.

PIPELINE

PIPELINE ( xt1 xt2 -- id1 id2 ) does all of that for two words: it creates a pipe, starts xt2 with SPAWN-CON reading the pipe, starts xt1 with SPAWN writing it, and returns both task ids (xt2's on top). It is the Forth version of producer | filter:

: SAY S" hello pipes" TYPE CR ;
: UPPER BEGIN KEY DUP -1 <> WHILE DUP 97 123 WITHIN IF 32 - THEN EMIT REPEAT DROP ;
' SAY ' UPPER PIPELINE 2DROP
HELLO PIPES

UPPER is a classic filter: it reads KEY until -1 and writes what it wants. Its output reaches the console as soon as the console is idle. The variables PL-R and PL-W hold the pipe's two handles; they are internal to PIPELINE.

Built-in devices

Path Mode Behaviour
/dev/null read or write reads return 0 (end); writes report every byte as written
/dev/cons write bytes go to the console
/dev/tasks read a snapshot of the task slots, one line each: id state name slices steps error
/dev/win, /dev/win/..., /dev/ui/tree read; write where the file accepts it the display server's windows and widgets as text files: see Windows as files: /dev/win
S" /dev/cons" 2 OPEN CONSTANT CON
S" straight to the console" CON WRITE .
CON CLOSE .

/dev/tasks is taken when you open it (up to 512 bytes of text) and only one reader can have it open at a time. Unlike TASKS, it shows names in plain text:

100 MS
S" /dev/tasks" 1 OPEN CONSTANT DT
BUF 128 DT READ  BUF SWAP TYPE
DT CLOSE .
1 done UPPER 3 318 -
2 done SAY 1 7 -
3 free - 0 0 -
4 free - 0 0 -
0

Your own devices

DEVICE ( addr u xt -- ) creates /dev/NAME, served by a Forth word, Plan 9 style. Each OPEN of the device creates a pipe and starts the word as a task on the other end:

  • opened for reading, the task's output is what you read;
  • opened for writing, what you write is the task's KEY input.

A read device that produces ten numbers:

: TEN 10 0 DO I . LOOP ;
S" ten" ' TEN DEVICE
S" /dev/ten" 1 OPEN CONSTANT TD
BUF 128 TD READ-WAIT  BUF SWAP TYPE CR
TD CLOSE .

A write device that counts the bytes it is sent:

VARIABLE GOT
: TALLY BEGIN KEY DUP -1 <> WHILE DROP 1 GOT +! REPEAT DROP ;
S" tally" ' TALLY DEVICE
S" /dev/tally" 2 OPEN CONSTANT TH
S" twelve bytes" TH WRITE-ALL .
TH CLOSE .
100 MS  GOT @ .
0 0 12

Closing your end gives the server end of input (KEY returns -1), so it finishes. Device names are 1 to 11 characters; at most 4 devices exist. S" ten" 0 DEVICE removes one, and calling DEVICE again with the same name replaces its word. Each open needs a free pipe, two free handles and a free task slot, otherwise OPEN returns -4.

HANDLES

HANDLES lists every open handle, every device and the last I/O error:

HANDLES
device /dev/ten xt 266
device /dev/tally xt 269
io-error pipe empty

With handles open, each one gets a line such as handle 1 kind file write owner 0 bytes 17 path /VIOS/NOTE.TXT, with buffered N added for a pipe.

Handles from the host

Two commands let host tools move bytes through any handle without Forth buffers: HANDLE-PUT h HEX writes bytes given in hex and prints put N, and HANDLE-GET h n reads up to 64 bytes and prints get HEX (get - at end of input). The CLI's :put and :get file copies are built on them.

vios> S" /VIOS/LOG.TXT" 2 OPEN .
1 ok
vios> HANDLE-PUT 1 48656C6C6F0A
put 6
ok

Mac files as handles

While the harness runs on the Mac, any path starting with /host/ is a file in the harness's host directory (~/VIOS-host by default; the harness option --host-root changes it, and --host-root "" turns the service off). OPEN, READ, WRITE, CLOSE, LOAD and INCLUDE work on them; each operation is one request over USB (up to 200 bytes), and a request times out after about a second if no harness answers (OPEN returns -5).

S" /host/hello.txt" 2 OPEN CONSTANT HH
S" written on the Agon" HH WRITE-ALL .
HH CLOSE .

Edit a program on the Mac and run it on the Agon:

LOAD /host/app.fth