Spin up dedicated queues for some operations.

Without this, libDispatch on Linux waits for up to 1 second to spin up new threads for handlers when needed. This tremendously increases latency (by more than 1 second) on incoming requests if there have been no requests for a few seconds. This is because no-longer-used threads are discarded after a few seconds, so the 1-second wait time mentioned earlier applies.

Given that we need to spawn two fresh threads for each incoming request, anyway (one of the event-processing CompletionQueue and one for the actual code that responds to the request), we avoid these wait times by creating fresh dispatch queues (which each spin up a dedicated thread immediately instead of waiting) for each request handler.

This reduces the initial latency for "cold" requests from ~1.5s down to ~60ms. The remaining latency is probably the time needed by libDispatch to spin up new pthreads for the fresh queues. Still more than I'd like, but the need for fresh pthreads will only cease once we switch to a non-blocking event handling system like SwiftNIO.

Sources/SwiftGRPC/Core/CompletionQueue.swift

@@ -113,7 +113,8 @@ class CompletionQueue {
   /// - Parameter completion: a completion handler that is called when the queue stops running
   func runToCompletion(completion: (() -> Void)?) {
     // run the completion queue on a new background thread
-    DispatchQueue.global().async {
+    let spinloopThreadQueue = DispatchQueue(label: "SwiftGRPC.CompletionQueue.runToCompletion.spinloopThread")
+    spinloopThreadQueue.async {
       spinloop: while true {
         let event = cgrpc_completion_queue_get_next_event(self.underlyingCompletionQueue, 600)
         switch event.type {

Sources/SwiftGRPC/Core/Server.swift

@@ -66,7 +66,9 @@ public class Server {
                   handlerFunction: @escaping (Handler) -> Void) {
     cgrpc_server_start(underlyingServer)
     // run the server on a new background thread
-    dispatchQueue.async {
+    let spinloopThreadQueue = DispatchQueue(label: "SwiftGRPC.CompletionQueue.runToCompletion.spinloopThread")
+    let handlerDispatchQueue = DispatchQueue(label: "SwiftGRPC.Server.run.dispatchHandler", attributes: .concurrent)
+    spinloopThreadQueue.async {
       spinloop: while true {
         do {
           let handler = Handler(underlyingServer: self.underlyingServer)
@@ -86,12 +88,12 @@ public class Server {
                 _ = strongHandlerReference
                 // this will start the completion queue on a new thread
                 handler.completionQueue.runToCompletion {
-                  dispatchQueue.async {
+                  handlerDispatchQueue.async {
                     // release the handler when it finishes
                     strongHandlerReference = nil
                   }
                 }
-                dispatchQueue.async {
+                handlerDispatchQueue.async {
                   // dispatch the handler function on a separate thread
                   handlerFunction(handler)
                 }