Home ...

Perf Service - Load testing for gRPC or http calls

Ankali
Ankali

PerfService

Service to enable load testing on different types of network calls.

Welcome to the PerfService wiki!

PerfService is a whole suite of dotnet projects to run performance test(s)

a. PerfRunner b. PerfLoader c. PerfRunnerTests

under one repository.

PerfRunner

The actual hoster and runner of service which has a folder called tests which contain the tests. Simply copy the TestBase.cs in it to your new test and update the RunTest method in it to suit your performance test needs. The http client has been typed DIed into _httpClient which can be used in the test(s) to make the http calls along with an end point.

To run on a specific port, when running multiple instances.

dotnet run --no-build Debug urls=http://0.0.0.0:5278

PS - Need to update docker-compose.yml for more than one PerfRunner

PerfLoader

Front end or the web interface to initiate performance test run(s) and control or to stop them. Screen

PerfRunnerTests

Unit tests for the PerfRunner to be kept updated with any new features added to the PerfRunner project.

Architecture

Architecture

Getting Started

Docker

Once in this directory. Should be as straight as

sudo docker build -t perfrunner .

and

docker compose up --no-build

The mcr.microsoft.com/dotnet/sdk:6.0 and redis:alpine images are used.

Redis Cache manager

Redis is used as a common user store to store users and are pulled and pushed into a queue there say when one or multiple PerfRunners are running.

PerfRunner can alone be run without the redis as well, in which case the user store should default to the in-memory queue and running multiple Perfrunners can cause the same user to call the api's from each of the PerfRunners.

Planned features

....

Existing features

  1. Abitlity to create and run http test(s).
  2. UI to run, view and stop test(s).
  3. Ability to update rate per sec during test run.
  4. Abitlity to create and run gRPC test(s).
  5. Account data support with User object having state has well. Each user have their own state and are in the queue per state and updated by the test(s) accordingly.
  6. User state management - as user queue per each state.
  7. Ability to transfer data across test(s) - can be achieved by adding new data properties to the User object itself, say like the phone number, library card number, health number and so on. This is updated and be used by each test(s) accordingly by dequeing that state queue with the user with that updated data. Say the user just registeted at the front desk with payment, library no, his state is now authenticated and library no. is the new data and he is in the authenticated state queue. Whenever its his turn, that user gets popped from that queue.
  8. Support for flag for even or uneven load distribution per second. 8a. If each action has a rate that is used for rate otherwise default to test rate.
  9. Try using interface inherit implementation.
  10. Monitor traffic in prometheus or Grafana cloud. Check the Grafana dashboard json to import it to the Grafana dashboard.
  11. Update rate for individual action.
  12. Pause/ Unpause individual action.
  13. Set duration for action and test. Whichever comes first will stop first.
  14. Updating duration for action will run the action for the next n update seconds since update.
  15. Update distribution for each action.
  16. Run, update and monitor test(s) on selected runner(s). If no runner is selected, all are probed*.
  17. PerfLoader dynamically maintains the PerfRunner(s) list using Polly polling every minute.

Planned features - Loader

  1. Clean front end interface for interaction. The app needs a lot of fron end improvement yet. Basic operations such as rate, duration and distribution updates are instantanious.

Sample Web app

For this use case, I am using the Bowling alley web app, assuming when a user goes to the ally, he has to say, get autheneticated, next wait for the lane and if lane is available, play can be initiated. For each stage, state can be represented, which is defined in UserState.

Monitoring

There are several ways to monitor like below.

CLI

The easiest way is with dotnet-counters. You can observe the distribution when you flip from Even to Uneven or vice - versa.

dotnet-counters monitor -n PerfRunner --counters PerfService.PerfRunner

Prometheus

Prometheus is used to more or less display or relay the instrumentation metrics to apps like Grafana or similar. To install you may have to download and run it on your distribution. The config file for it is here or append the scrape config like below.

  - job_name: 'PerfRunner'
    scrape_interval: 1s # poll very quickly for a more responsive demo
    static_configs:
      - targets: ['localhost:9186']

Grafana

To run Grafana as a docker image, run.

docker run -d --name=grafana -p 3000:3000 grafana/grafana-enterprise
admin

Run results

Please refer here for a sample run with upto 300 users per second with 12 core processor and having about 17% CPU usage and about 1.5 GB memory usage.

Sample commands

To run from tests directly CLI with PerfRunner running.

$ grpcurl -plaintext localhost:5277 describe
perf.Perf is a service:
service Perf {
  rpc Ping ( .perf.PingRequest ) returns ( .perf.PingReply );
  rpc RunTest ( .perf.TestRequest ) returns ( .perf.TestReply );
  rpc StopAllTests ( .perf.StopAllTestsRequest ) returns ( .perf.StopAllTestsReply );
  rpc StopTest ( .perf.StopTestRequest ) returns ( .perf.StopTestReply );
  rpc UpdateRate ( .perf.UpdateRateRequest ) returns ( .perf.UpdateRateReply );
}
grpc.reflection.v1alpha.ServerReflection is a service:
service ServerReflection {
  rpc ServerReflectionInfo ( stream .grpc.reflection.v1alpha.ServerReflectionRequest ) returns ( stream .grpc.reflection.v1alpha.ServerReflectionResponse );
}

$ grpcurl -plaintext localhost:5277 describe perf.TestRequest
perf.TestRequest is a message:
message TestRequest {
  string guid = 1;
  string name = 2;
  int32 rate = 3;
  repeated .perf.ActionOption actions = 4;
}

License

Free for non-commercial use, but please read LICENSE for commercial use, other(s) and support.

Issues

Please report any issues here. This can be ranging from a minor defect to a valid feature request.

Contribution

If you would like to contribute to thie repository, please read CONTRIBUTING before creating your PR.

Customizations/ Support

G Sys

Donations

If you like using this repository and like to donate, please visit the below link. This work is made possible with donations like yours. PM for customizations and implementations .

paypal