Remote debugging allows you to debug a LispWorks process that is running on one machine using a LispWorks IDE that is running on another machine. It is intended to make it easier to debug applications running on machines that do not have the LispWorks IDE, mainly mobile device applications on iOS and Android, but also applications running on servers where you cannot run the LispWorks IDE.
In the discussion below, the process being debugged is referred to as the "client", and the process running the LispWorks IDE is referred to the "IDE".
With remote debugging you can:
When you look at the source code from an IDE tool that is displaying client side data (for example by using the Find Source menu item) or look at the class of a remote object, the IDE finds the matching source or class on its side. You need to ensure that the IDE and the client sides have the same sources and class definitions for that to work.
Remote debugging is based on "connections", which are implemented on top of streams connecting the two sides. In normal usage, LispWorks will open a TCP socket stream for a connection, but you can also create connections with your own streams or sockets.
A LispWorks process that has loaded the remote debugging module can be connected to several IDE processes simultaneously, and any IDE process can be connected to several clients. The same IDE process can act as the client side and the IDE side at the same time. However, the most common usage is expected to be one client and one IDE, and the interface is designed towards this simple usage.
Communication across the connection is architecture-independent, and either side can be any architecture. It relies on there being a working Common Lisp reader.
The client side should load the client code by calling:
(require "remote-debugger-client")
Note that if the client is a delivered application, the call to require needs to happen at load time, before calling deliver. On the IDE side, the module "remote-debugger-full" (which includes "remote-debugger-client") is loaded automatically when needed.
In the simple usage scenario, you have one IDE and one client. To create the connection between them, you need to tell LispWorks how to create the TCP socket stream, which requires one side to be a TCP server, and the other side to have the address (or name) of the server-side machine to connect to. Therefore, in the simple case you will need to make a function call on both sides. Once you perform the two function calls, you can use most of the power of remote debugging.
There are two ways to specify the connection: one with the IDE acting as the TCP server, and one with the client acting as the TCP server.
On the IDE side, you should call:
(dbg:start-ide-remote-debugging-server)
On the client side, you should call:
(dbg:configure-remote-debugging-spec "ide-hostname")
After making these calls, whenever the debugger is entered on the client side, it will automatically display a GUI debugger on the IDE side. In addition, calls to start-remote-listener and remote-inspect from the client side will automatically display tools on the IDE side.
These functions use TCP port 21101 by default (the initial value of *default-ide-remote-debugging-server-port*).
By default, configure-remote-debugging-spec delays opening the connection until it is actually needed (by entering the debugger, or a call to start-remote-listener or remote-inspect).
Note that within the LispWorks IDE, you can make the call to start-ide-remote-debugging-server using the Start IDE Remote Debugging Server button in the Preferences dialog Debugger options Remote tab.
On the client side you should call:
(dbg:start-client-remote-debugging-server)
On the IDE side you should call:
(dbg:ide-connect-remote-debugging "client-hostname")
The call on the IDE side opens a connection, which the client will use when entering the debugger and in calls to start-remote-listener and remote-inspect.
These functions use TCP port 21102 by default (the initial value of *default-client-remote-debugging-server-port*).
Note that within the LispWorks IDE, you can make the call to ide-connect-remote-debugging using the Connect To Debugging Client button in the Preferences dialog Debugger options Remote tab.
The client side remote debugging API is intended to minimize the amount of work you need to do for simple configurations.
Once you have either specified the connection by calling configure-remote-debugging-spec on the client side and called start-ide-remote-debugging-server on the IDE side, or called start-client-remote-debugging-server on the client side and the IDE has connected to it using ide-connect-remote-debugging, entering the debugger automatically opens a Debugger window on the IDE side (unless you are already inside a Remote Listener or Remote Debugger).
If you want to open a Remote Listener on the IDE side from the client side, you can call start-remote-listener. Also, you can call remote-inspect on the client side to inspect an object on the IDE side.
The interface allows you to have more complex configurations, as detailed by configure-remote-debugging-spec, create-client-remote-debugging-connection and start-client-remote-debugging-server.
The behavior of the Debugger, Listener and Inspector tools is described in the LispWorks IDE User Guide.
Remote Debugger windows are opened automatically when the client side enters the debugger.
Remote Listener windows are opened on request, either by using the IDE's menus, by calling ide-open-a-listener, or by calling ide-connect-remote-debugging with :open-a-listener t (or from the client side by start-remote-listener).
The Inspector inspects a remote object when you tell it to inspect in the same way as you would tell it to inspect an ordinary object (typically from the Debugger or Listener), or by calling remote-inspect on the client side.
Remote (client side) values can be used on the IDE side and the type of object affects how it is represented.
Remote numbers and characters are represented on the IDE side as their actual IDE side values.
Remote strings are represented on the IDE side as IDE side strings, which are copies of the string of the same element type. Note that, as a result, two separate occurrences in the IDE of the same client side string are not necessarily the same object, and that modifying the characters in these strings does not affect the string on the client side.
Most other remote objects are represented in the IDE by remote handles (see below for exceptions). Handles are specific to a connection, so accessing the same remote object in the IDE multiple times over the same connection will always use the same (by eq) handle. However, accessing the same client object through different connections will use different handles, which are not equal at all, and there is no way to find if two handles from different connections refer to the same remote object.
Remote handles are printed like this:
#<Remote ... >
where the ... is the printing of the remote object by the client side.
Handles are opaque objects. The predicate remote-object-p can be used to check if an object is a remote object, and remote-object-connection returns the connection that the handle is associated with. If two handles are associated with the same connection, then they are eq if and only if they refer to the same object on the client side.
The generic function get-inspector-values has a method that specializes on handles to invoke get-inspector-values on the client side and return the results. Note that get-inspector-values also returns a setter, which allows you to set values inside the client's object. This method makes the IDE Inspector and the CL functions inspect and describe work on remote handles.
Apart from the interface in the previous paragraphs, there is no useful way to access handles on the IDE side. However, you can access the underlying remote object by using ide-eval-form-in-remote or ide-funcall-in-remote, by sending a form containing the handle. For example, assuming the value of my-remote-simple-vector is a remote handle for a simple-vector, you can read its first element by:
(dbg:ide-eval-form-in-remote `(svref ,my-remote-simple-vector 0))
This will call svref on the client's object that my-remote-simple-vector is a handle for, because the client side call receives the underlying object rather than the handle.
Each call to ide-eval-form-in-remote and ide-funcall-in-remote is associated with a specific connection, and only remote objects that are associated with the connection can be used in form arguments. Trying to use remote objects that are associated with another connection signals an error.
LispWorks represents certain client side conses/symbols as conses/symbols on the IDE side in cases where there is no need to access the remote object. For example, the lists that get-inspector-values returns are IDE side conses, and the symbols in the slot-names list are IDE side symbols (for symbols in packages that exist on the IDE side). By default, ide-eval-form-in-remote and ide-funcall-in-remote return handles to the values returned by the form, except for numbers, characters, strings and the top-level of lists. They have a keyword :encoded-result which gives you some control over whether the values are returned as handles or not.
When displaying the source code of a function, LispWorks uses source location information on the IDE side to find the source file. That means that the IDE side needs to have the same source files loaded as the client side. To find a subform inside the definition of a function, the debugger uses the information from the client side, which must be compiled with source-level-debugging (and kept if it is delivered) for this to work.
Invoking the Class Browser in the IDE for a remote object handle shows the class on the IDE side that has the same class-name as the class of the object on the client side. Calling class-of (and type-of) on the IDE side on a remove object handle return the internal class (and class name) of remote handles, which you should not be accessing.
The functions ide-eval-form-in-remote and ide-funcall-in-remote can be used to call functions on the client side (ide-eval-form-in-remote is used by the editor commands).
The function ide-set-remote-symbol-value can be used to set the global value of a symbol on the client side, which is a common operation. It is equivalent to calling ide-funcall-in-remote with set.
The function ide-attach-remote-output-stream can be used to create an output stream on the client side, such that any output into it will go to a stream on the IDE side. It returns a remote object handle for the client side stream, which can then be used it calls to ide-eval-form-in-remote etc.
The sections below describe some unexpected problems that you might encounter when using remote debugging and suggest ways to solve them.
There are some basic things to check first.
start-client-remote-debugging-server on the client side and ide-connect-remote-debugging on the IDE side.
Or:
configure-remote-debugging-spec on the client side and start-ide-remote-debugging-server on the IDE side.
When using start-client-remote-debugging-server and ide-connect-remote-debugging, start-client-remote-debugging-server must be called first.
When using configure-remote-debugging-spec and start-ide-remote-debugging-server, start-ide-remote-debugging-server must be called before the connection is opened by the client. However, by default, configure-remote-debugging-spec delays opening the connection until it is needed so can be called first.
(example-file "android/OthelloDemo/app/src/main/AndroidManifest.xml").:log-stream argument for both sides, and check if anything is written to it.
If you cannot find the problem, then check that the connection works at the TCP level.
(setq *log-stream* <somewhere-that-you-can-see-it>)
(comm:start-up-server
:function #'(lambda (socket)
(format *log-stream* "Connected from ~a~%"
(comm:get-socket-peer-address socket))
(finish-output *log-stream*))
:service dbg:*default-client-remote-debugging-server-port*)
and then on the IDE side evaluate:
(comm:open-tcp-stream "<client-hostname>"
dbg:*default-client-remote-debugging-server-port*)
The call to open-tcp-stream should return a stream, and the "Connected from" message should be printed to *log-stream* on the client side.
(comm:start-up-server
:function #'(lambda (socket)
(format mp:*background-standard-output*
"Connected from ~a~%"
(comm:get-socket-peer-address socket))
(finish-output mp:*background-standard-output*))
:service dbg:*default-ide-remote-debugging-server-port*)
and then on the client side evaluate:
(comm:open-tcp-stream "<ide-hostname>"
dbg:*default-ide-remote-debugging-server-port*)
The call to open-tcp-stream should return a stream, and the "Connected from" message should be printed to Output tab of the Listener or Editor in the IDE.
If you cannot connect as above then you need to fix the configuration of your machines to make it work. If you can connect as above, but the remote debugging does not connect after you did all the checks then contact LispWorks support for help.
The most likely explanation for this is that the underlying connection was closed. Check if the remote object prints with "{closed connection}".
By default, the connection opened by the client side functions is reused whenever a connection is needed. This is normally all that is required, but sometimes it is useful to have a better control.
The client has a default connection (a global value), and a switch that enables using the default connection (the enabling switch). Both configure-remote-debugging-spec and start-client-remote-debugging-server have keyword arguments :setup-default and :enable, which control setting the default connection, and whether to enable using it. The default for both is true. The default connection and switch are used by the APIs that need a connection (mainly when the debugger is entered, but also remote-inspect and start-remote-listener), so by default the connection that was opened last is used.
The value of the enabling switch can be set globally by set-remote-debugging-connection or in a dynamic extent by with-remote-debugging-connection. These functions also allow you to specify a specific connection to use, rather than the default. Note that with-remote-debugging-connection and set-remote-debugging-connection do not affect the default connection, only the enabling switch.
The function set-default-remote-debugging-connection can be used to set the default connection.
start-client-remote-debugging-server and configure-remote-debugging-spec take also a keyword :open-callback, which specifies a callback that is called whenever a connection is created. You can use this callback to store the connection somewhere for later use, for example in a call to set-default-remote-debugging-connection, set-remote-debugging-connection or with-remote-debugging-connection.
The value of the :setup-default keyword to configure-remote-debugging-spec can also be :delayed, which means that the connection is not opened immediately, but when it is opened, it is set as the default.
When the debugger is entered, it first checks the value of the enabling switch. If it is set to a connection then that connection is used. If it is set to t, then the default connection is used. Otherwise, it checks if a host was configured (globally by configure-remote-debugging-spec or in a dynamic extent by with-remote-debugging-spec) and tries to open a connection to it. If that succeeds, it decides according to the :setup-default and :enable arguments whether to set the default connection and the enabling switch, and then use this connection. If :setup-default is nil, then the connection is closed when the Remote Debugger is closed.
When a connection is closed, all remote object handles in the IDE that were created using it become invalid. For example, if you use configure-remote-debugging-spec with :setup-default nil, and later the Remote Debugger was raised and you inspected an argument of a function and then closed the Remote Debugger, then the Inspector will fail to access slots in this object. For this reason, it is usually better to have a permanently open default connection, so the Inspector can be still be used after the Remote Debugger has been closed. A single connection is also more efficient, but the effect of this is small.
start-remote-listener behaves the same as the debugger when it tries to find a connection.
remote-inspect first checks if it can use an existing connection the same way as the debugger. However, if it cannot, it tries to use a connection used previously by an Inspector. If this does not work, it tries the default connection (if any) even if the enabling switch is nil, and if all these fail, it opens a connection using the configured host if any, and remembers it for the next time (unless configured to set this connection as the default). The Inspector behaves differently to the Debugger/Listener is because there is no obvious place where a temporary connection should be closed in the case of the Inspector, compared to the Debugger and Listener where closing the GUI tool is the natural place to close the connection if it is temporary.
In the simplest usage, you will have one connection that is used for everything. The next level of complexity is to have one connection, but control dynamically whether to use it or not, either globally by calls to set-remote-debugging-connection, or in a dynamic extent using with-remote-debugging-connection.
For more complex usage, you can use the :open-callback to record the connections that you have opened, and then use them in set-default-remote-debugging-connection or set-remote-debugging-connection or with-remote-debugging-connection to tell the debugger/listener/inspector which connection to use.
The functions described in 3.7.5.3 Common (both IDE and client) connection functions can also be used on the client side.
Normally you do not need to manage remote debugging connections on the IDE side, but sometimes it may be useful.
ide-list-remote-debugging-connections returns a list of connections.
ide-find-remote-debugging-connection can be used to find a connection. This is used by default by the IDE side functions that need a connection (ide-eval-form-in-remote, ide-funcall-in-remote, ide-set-remote-symbol-value and ide-attach-remote-output-stream), and the Editor commands.
ide-set-default-remote-debugging-connection can be used to set the default connection, which is what the Editor commands use, and affects what ide-find-remote-debugging-connection returns.
You can get the connection from a remote object handle by using remote-object-connection.
The functions described in 3.7.5.3 Common (both IDE and client) connection functions can also be used on the IDE side.
You can close a connection by using close-remote-debugging-connection. Closing a connection causes the other side to be closed too. Closing the default connection causes the default to be set to nil.
You can use ensure-remote-debugging-connection to check if a connection is alive. ensure-remote-debugging-connection can be called with any object and returns nil for any non-connection object or a connection object that is closed.
You can use remote-debugging-connection-add-close-cleanup to add a "cleanup", which is a callback function that is called when the connection is closed, and remote-debugging-connection-remove-close-cleanup to remove a previously added cleanup.
The function remote-debugging-connection-peer-address can be used to check what machine is on the other side.
The function remote-debugging-connection-name can be used to find the name of a connection.
Each opened connection has a dedicated Lisp process that handles communications through it, which you can see by listing processes using (mp:ps) or in the Process Browser tool. The name of the connection appears in the name of the process. You can forcibly close a connection by using process-terminate on the process or from the Process Browser.
The variables *default-client-remote-debugging-server-port* (when the client is the TCP server) and *default-ide-remote-debugging-server-port* (when the IDE is the TCP server) specify the default port to use. You can overwrite the default port by supplying the port argument to the functions that establish the connection, or change it in the IDE when using the GUI.
You can also override the values in *default-client-remote-debugging-server-port* and *default-ide-remote-debugging-server-port* by configuring the machine to map a network service name to port number (for example /etc/services on Linux or %WINDIR%\system32\etc\services on Windows).
Before LispWorks uses the value of *default-client-remote-debugging-server-port*, it looks up the network service name lw-remote-debug-client. If this service is registered, then LispWorks uses the registered port number instead of *default-client-remote-debugging-server-port*. Note that this applies both when starting the TCP server on the remote debugging client and when connecting to such a server from the IDE side.
Similarly, before LW LispWorks the value of *default-ide-remote-debugging-server-port*, it looks up the network service name lw-remote-debug-ide. If this service is registered, then LispWorks uses the registered port number instead of *default-ide-remote-debugging-server-port*. This applies both when starting the TCP server on the remote debugging IDE and when connecting to such server from the client side.
In all situations, the IDE and client must end up using the same port number.
You can use SSL for the remote debugging connections by supplying the :ssl keyword to the functions ide-connect-remote-debugging, start-ide-remote-debugging-server, configure-remote-debugging-spec and start-client-remote-debugging-server and the macro with-remote-debugging-spec.
LispWorks® User Guide and Reference Manual - 01 Dec 2021 19:30:18