Browse Source

Add single connection performance scenarios

Motivation:

Running the unconstrained performance benchmark on a development
machines yields results with a large amount of variance (I have seen
over 20% between runs in some cases) since all cores become fully
loaded. As such it's much harder to determine whether a change was
actually beneficial.

Modifications:

- Add a single connection unary scenario, a copy of the unconstrained
  unary scenario but uses a single client such that only two threads are
  used: one for the server and one for the client.

Result:

- Results with less variance between runs.
George Barnett 5 years ago
parent
commit
dcfaaae3cc

+ 3 - 3
Performance/QPSBenchmark/README.md

@@ -64,6 +64,6 @@ For examples of running benchmarking tests proceed as follows.
 - `scenarios/unary-unconstrained.json`: will run a test with unary RPCs
 - `scenarios/unary-unconstrained.json`: will run a test with unary RPCs
   using all cores on the machine. 64 clients will connect to the server, each
   using all cores on the machine. 64 clients will connect to the server, each
   enqueuing up to 100 requests.
   enqueuing up to 100 requests.
-- `scenarios/bidirectional-ping-pong.json`: will run bidirectional streaming
-  RPCs.
-
+- `scenarios/unary-1-connection.json`: as above with a single client.
+- `scenarios/bidirectional-ping-pong-1-connection.json`: will run bidirectional
+  streaming RPCs using a single client.

+ 0 - 0
Performance/QPSBenchmark/scenarios/bidirectional-ping-pong.json → Performance/QPSBenchmark/scenarios/bidirectional-ping-pong-1-connection.json


+ 53 - 0
Performance/QPSBenchmark/scenarios/unary-1-connection.json

@@ -0,0 +1,53 @@
+{
+  "scenarios": [
+    {
+      "name": "swift_protobuf_async_unary_qps_unconstrained_insecure",
+      "warmup_seconds": 5,
+      "benchmark_seconds": 30,
+      "num_servers": 1,
+      "server_config": {
+        "async_server_threads": 0,
+        "channel_args": [
+          {
+            "str_value": "throughput",
+            "name": "grpc.optimization_target"
+          }
+        ],
+        "server_type": "ASYNC_SERVER",
+        "security_params": null,
+        "threads_per_cq": 0,
+        "server_processes": 0
+      },
+      "client_config": {
+        "security_params": null,
+        "channel_args": [
+          {
+            "str_value": "throughput",
+            "name": "grpc.optimization_target"
+          }
+        ],
+        "async_client_threads": 0,
+        "outstanding_rpcs_per_channel": 100,
+        "rpc_type": "UNARY",
+        "payload_config": {
+          "simple_params": {
+            "resp_size": 0,
+            "req_size": 0
+          }
+        },
+        "client_channels": 1,
+        "threads_per_cq": 0,
+        "load_params": {
+          "closed_loop": {}
+        },
+        "client_type": "ASYNC_CLIENT",
+        "histogram_params": {
+          "max_possible": 60000000000,
+          "resolution": 0.01
+        },
+        "client_processes": 0
+      },
+      "num_clients": 0
+    }
+  ]
+}