Each Lisp mp:process has a separate native thread and in LispWorks 6.0 and later versions these threads can run simultaneously.
Note: In LispWorks 5.1 and earlier versions, you can have many runnable mp:process objects/native threads, but Lisp code can only run in one thread at a time and a lock is used to enforce this. This can limit performance on a computer with multiple CPU cores. When a foreign function is called using the FLI, the lock is released until the function returns. This allows other Lisp threads to run, for instance while waiting for a database query to execute.
You can call back into Lisp using fli:define-foreign-callable in any thread, without any other setup.
Threads running Lisp code can be rescheduled preemptively, so if you call into Lisp from more than one thread simultaneously and one request takes a long time then it will not delay the requests in other threads.
When foreign code creates a native thread (a "foreign thread") and code running on this thread calls into Lisp, then Lisp needs to associate a Lisp process object with this thread to be able to work properly.
When there is a call on a foreign thread into Lisp which is not a recursive call (an "outer call"), Lisp first checks if there is a process associated with this thread, and if there is it uses it. Otherwise, it creates a new process and associates it with the foreign thread. Recursive calls into Lisp (when Lisp calls foreign code which calls back into Lisp) are processed in the same way as recursive calls in Lisp threads.
When the outer call returns, Lisp by default keeps the process associated with the thread, but this is not guaranteed. Keeping the process means that next call into Lisp requires less work, but comes at the cost of using more memory. Lisp eliminates the process if it detects that the thread has died, if there is call to last-callback-on-thread inside the outer call or if the process is killed by process-terminate.
Once Lisp has a process associated with the thread, it establishes it as the current process, as returned by calling get-current-process, and then calls the foreign callable Lisp code.
Part of establishing the process involves binding the variables in *process-initial-bindings*. Note that this binding happen repeatedly for each outer call. The computation of the bound value is done when the process is created, so if the process is not eliminated between outer calls (the default behavior), this happens only once. The computation of the value occurs in the dynamic environment of the new process.
Compatibility note: Before LispWorks 7.1, the computation occurred in a "no-process" scope, and an error would have entered the debugger in the console without an option to abort.
Keeping the process between outer calls (the default behavior), makes each call faster, but uses memory. For few processes, this is probably the best approach. With many processes, the memory usage may become an issue.
There is an overhead for an outer call, which is larger than a recursive call. A few outer calls per second should not be a problem, but it should be avoided inside a heavy computation.
LispWorks® User Guide and Reference Manual - 01 Dec 2021 19:30:21