24.2.1 Defining a new stream class

Streams can be capable of input or output (or both), and may deal with characters or with binary elements. The stream package provides a number of stream classes with different capabilities from which user defined streams can inherit. In our example the stream must be capable of input and output, and must read characters. The following code defines our stream class appropriately:

(defclass unicode-ls-stream 

          (stream:fundamental-character-input-stream

           stream:fundamental-character-output-stream)

  ((file-stream :initform nil 

                :initarg :file-stream

                :accessor ls-stream-file-stream)))

The new class, unicode-ls-stream, has fundamental-character-input-stream and fundamental-character-output-stream as its superclasses, which means it inherits the relevant default character I/O methods. We shall be overriding some of these with more relevant and efficient implementations later.

Note that we have also provided a file-stream slot. When making an instance of unicode-ls-stream we can create an instance of a Common Lisp file stream in this slot. This allows us to use the Common Lisp file stream functionality for reading from and writing to a file.

24.2.2 Recognizing the stream element type

We know that the stream will read from a file using file-stream functionality and that the stream element type will be character. The following defines a method on stream-element-type to return the correct element type.

(defmethod stream-element-type ((stream unicode-ls-stream))

  'character)

24.2.3 Stream directionality

Streams can be defined for input only, output only, or both. In our example, the unicode-ls-stream class needs to be able to read from a file and write to a file, and we therefore defined it to inherit from an input and an output stream class. We could have defined disjoint classes instead, one inheriting from fundamental-character-input-stream and the other from fundamental-character-output-stream. This would have allowed us to rely on the default methods for the direction predicates.

However, given that we have defined one bi-directional stream class, we must define our own methods for the direction predicates. To allow this, the Common Lisp predicates input-stream-p and output-stream-p are implemented as generic functions.

(defmethod input-stream-p ((stream unicode-ls-stream))

  (input-stream-p (ls-stream-file-stream stream)))

(defmethod output-stream-p ((stream unicode-ls-stream))

  (output-stream-p (ls-stream-file-stream stream)))

The above code allows us to "trampoline" the correct direction predicate functionality from file-stream, using the ls-stream-file-stream accessor we defined previously.

24.2.4 Stream input

The following method for stream-read-char reads a character from the stream. If the character read is a #\Line-Separator, then the method returns #\Newline, otherwise the character read is returned. stream-read-char returns :eof at the end of the file.

(defmethod stream:stream-read-char ((stream unicode-ls-stream))

  (let ((char (read-char (ls-stream-file-stream stream) 

               nil :eof)))

    (if (eql char #\Line-Separator)

        #\Newline

      char)))

There is no need to define a new method for stream-read-line as the default method uses stream-read-char repeatedly to read a line, and our implementation of stream-read-char ensures that this will work.

We also need to make sure that if a #\Newline is unread, it is unread as a #\Line-Separator. The following method for stream-unread-char uses the Common Lisp file stream function unread-char to achieve this.

(defmethod stream:stream-unread-char ((stream unicode-ls-stream)

                                      char)

  (unread-char (if (eql char #\Newline) #\Line-Separator char)

               (ls-stream-file-stream stream)))

Finally, although the default methods for stream-listen and stream-clear-input would work for our stream, it is faster to use the functions provided by file-stream, again using our accessor ls-stream-file-stream.

(defmethod stream:stream-listen ((stream unicode-ls-stream))

  (listen (ls-stream-file-stream stream)))

(defmethod stream:stream-clear-input ((stream unicode-ls-stream))

  (clear-input (ls-stream-file-stream stream)))

24.2.5 Stream output

The following method for stream-write-char uses write-char to write a character to the stream. If the character written to unicode-ls-stream is a #\Newline, then the method writes a #\Line-Separator to the file stream.

(defmethod stream:stream-write-char ((stream unicode-ls-stream) 

                                     char)

  (write-char (if (eql char #\Newline)

                   #\Line-Separator

               char)

              (ls-stream-file-stream stream)))

The default method for stream-write-string calls stream-write-char repeatedly to write a string to the stream. However, the following is a more efficient implementation for our stream.

(defmethod stream:stream-write-string ((stream unicode-ls-stream)

                                        string &optional (start 0)

                                        (end (length string)))

  (loop with i = start

        until (>= i end)

        do (let* ((newline (position #\Newline 

                            string :start i :end end))

                  (this-end (or newline end)))

             (write-string string (ls-stream-file-stream stream) 

                           :start i :end this-end)

             (incf i this-end)

             (when newline 

                   (stream:stream-terpri stream)

                   (incf i)))

        finally (return string)))

We do not need to define our own method for stream-terpri , as the default uses stream-write-char , and therefore works appropriately

To be useful, the stream-line-column and stream-start-line-p generic functions need to know the number of characters preceding a #\Line-Separator. However, since the LispWorks file stream records line position only by #\Newline characters, this information is not available. Hence we define the two generic functions to return nil:

(defmethod stream:stream-line-column 

  ((stream unicode-ls-stream))

  nil)

(defmethod stream:stream-start-line-p 

  ((stream unicode-ls-stream))

  nil)

Finally, the methods for stream-force-output , stream-finish-output and stream-clear-output are "trampolined" from the standard force-output, finish-output and clear-output functions.

(defmethod stream:stream-force-output ((stream 

                                        unicode-ls-stream))

  (force-output (ls-stream-file-stream stream)))

(defmethod stream:stream-finish-output ((stream 

                                         unicode-ls-stream))

  (finish-output (ls-stream-file-stream stream)))

(defmethod stream:stream-clear-output ((stream 

                                        unicode-ls-stream))

  (clear-output (ls-stream-file-stream stream)))

24.2.6 Instantiating the stream

Now that the stream class has been defined, and all the methods relevant to it have been set up, we can create an instance of our user defined stream to test it. The following function takes a filename and optionally a stream direction as its arguments and makes an instance of unicode-ls-stream. It ensures that the file-stream slot of the stream contains a Common Lisp file-stream capable of reading from or writing to a file given by the filename argument.

(defun open-unicode-ls-file (filename &key (direction :input))

  (make-instance 'unicode-ls-stream :file-stream 

    (open filename 

          :direction direction 

          :external-format :unicode 

          :element-type 'character)))

The following macro uses open-unicode-ls-stream in a similar manner to the Common Lisp macro with-open-file:

(defmacro with-open-unicode-ls-file ((var filename

                                        &key (direction :input)) 

                                        &body body)

  `(let ((,var (open-unicode-ls-file ,filename 

                :direction ,direction)))

     (unwind-protect

         (progn ,@body)

       (close ,var))))

We now have the required functions and macros to test our user defined stream. The following code uses config.sys as a source of input to an instance of our stream, and outputs it to the file unicode-ls.out, changing all occurrences of #\Newline to #\Line-Separator in the process.

(with-open-unicode-ls-file (ss "C:\\unicode-ls.out" 

                              :direction :output)

   (write-line "-*- Encoding: Unicode; -*-" ss)

   (with-open-file (ii "C:\\config.sys")   ; Don't edit this file!

     (loop with line = nil

           while (setf line (read-line ii nil nil))

           do (write-line line ss))))

After running the above code, if your load the file C:\unicode-ls.out into an editor (for example, a LispWorks editor), you can see the line separator used instead of CR/LF. Most editors do not yet recognize the Unicode Line Separator character yet. In some editors it appears as a blank glyph, whereas in the LispWorks editor it appears as <2028>. In LispWorks you can use Alt+X What Cursor Position or Ctrl+X = to identify the unprintable characters.

You can also use the follow code to print out the contents of the new file line by line.

(with-open-unicode-ls-file (ss "C:\\unicode-ls.out")

   (loop while (when-let (line (read-line ss nil nil))

                 (write-line line))))