Crank 是微软新出的一个性能测试框架，集成了多种基准测试工具，如bombardier、wrk等。
Crank通过统一的配置，可以转换成不同基准测试工具命令进行测试。可参考Bombardier Job实现。
安装Crank
运行如下两个命令分别安装Crank的cli(Controller)和Agent。
- dotnet tool update Microsoft.Crank.Controller --version "0.2.0-*" --global
- dotnet tool update Microsoft.Crank.Agent--version "0.2.0-*" --global
需要.Net SDK 5.0环境
安装完成后执行命令crank，会打印出如下可以配置的参数和介绍，或者github查看相关参数介绍。
- PS C:\Users\Stack\Desktop> crank
- Crank Benchmarks Controller
- The Crank controller orchestrates benchmark jobs on Crank agents.
- Usage: Crank [command] [options]
- Options:
- -?|-h|--help Show help information
- -c|--config Configuration file or url
- -s|--scenario Scenario to execute
- -j|--job Name of job to define
- --profile Profile name
- --script Execute a named script available in the configuration files. Can be used multiple times.
- -j|--json Saves the results as json in the specified file.
- --csv Saves the results as csv in the specified file.
- --compare An optional filename to compare the results to. Can be used multiple times.
- --variable Variable
- --sql Connection string of the SQL Server Database to store results in
- --table Table name of the SQL Database to store results in
- --session A logical identifier to group related jobs.
- --description A string describing the job.
- -p|--property Some custom key/value that will be added to the results, .e.g. --property arch=arm --property
- os=linux
执行crank-agent，启动基准测试所需的代理服务。github查看相关参数介绍。
- PS C:\Users\Stack\Desktop> crank-agent
- Hosting environment: Production
- Content root path: C:\Users\Stack\.dotnet\tools\.store\microsoft.crank.agent\0.2.0-alpha.21567.1\microsoft.crank.agent\0.2.0-alpha.21567.1\tools\net5.0\any\
- Now listening on: http://[::]:5010
创建Crank配置文件
配置文件参考官方hello.benchmarks.yml示例
示例文件中引入了bombardier.yml，由于大环境对githubusercontent.com域名不太友好，可以考虑将bombardier.yml下载到本地，imports引入本地路径或者直接将文件内容加入到新建的配置文件。
- imports:
- -https://raw.githubusercontent.com/dotnet/crank/main/src/Microsoft.Crank.Jobs.Bombardier/bombardier.yml
生产中使用推荐imports文件的方式，crank配置文件中做少量配置就可以完成基准测试，并且可以引入不同的Microsoft.Crank.Jobs.XXX/XXX.yml,基于不同的基准测试工具进行测试。
其他测试工具配置文件都在https://github.com/dotnet/crank/blob/main/src/Microsoft.Crank.XXX下。
- variables:
- headers:
- none: ''
- plaintext: '--header "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
- html: '--header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header "Connection: keep-alive"'
- json: '--header "Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
- connectionclose: '--header "Connection: close"'
- presetHeaders: none
- jobs:
- bombardier:
- source:
- repository: https://github.com/dotnet/crank.git
- branchOrCommit: main
- project: src/Microsoft.Crank.Jobs.Bombardier/Microsoft.Crank.Jobs.Bombardier.csproj
- sourceKey: bombardier
- noBuild: true
- readyStateText: Bombardier Client
- waitForExit: true
- variables:
- connections: 256 #设置连接数
- warmup: 15 #设置测试预热次数
- duration: 15 #设置测试时间
- requests: 0 #设置测试请求实例数
- rate: 0 #设置每秒请求频率
- transport: fasthttp # | http1 | http2 设置使用golang的fasthttp库发送http请求
- serverScheme: http
- serverAddress: localhost
- serverPort: 5000
- path:
- bodyFile: # path or url for a file to use as the body content
- verb: # GET when nothing is specified
- customHeaders: [ ] # list of headers with the format: '<name1>: <value1>', e.g. [ 'content-type: application/json' ]
- arguments: "-c {{connections}} -w {{warmup}} -d {{duration}} -n {{requests}} --insecure -l {% if rate != 0 %} --rate {{ rate }} {% endif %} {% if transport %} --{{ transport}} {% endif %} {{headers[presetHeaders]}} {% for h in customHeaders %}{% assign s = h | split : ':' %}--header \"{{ s[0] }}: {{ s[1] | strip }}\" {% endfor %} {% if serverUri == blank or serverUri == empty %} {{serverScheme}}://{{serverAddress}}:{{serverPort}}{{path}} {% else %} {{serverUri}}:{{serverPort}}{{path}} {% endif %} {% if bodyFile != blank and bodyFile != empty %} -f {{bodyFile}} {% endif %} {% if verb != blank and verb != empty %} -m {{verb}} {% endif %}"
- onConfigure:
- # - job.timeout = Number(job.variables.duration) + Number(job.variables.warmup) + 10;
- server:
- source: #指定需要测试的项目，本文直接使用本地路径
- localFolder: .
- project: crank_demo.csproj
- readyStateText: Application started.
- # source: 指定测试项目的远程仓库地址，并通过branchOrCommit指定分支
- # repository: https://github.com/dotnet/crank
- # branchOrCommit: main
- # project: samples/hello/hello.csproj
- # readyStateText: Application started.
- scenarios: #配置基准测试场景
- crank_demo: #定义方案名，执行crank命令时指定该名称
- application:
- job: server # 指定测试项目为上面定义的server
- load:
- job: bombardier # 指定测试工具bombardier
- variables:
- serverPort: 5000 #配置http服务端口
- path: / #配置http服务地址
- profiles:
- local:
- variables:
- serverAddress: localhost
- jobs:
- application:
- endpoints:
- - http://localhost:5010
- load:
- endpoints:
- - http://localhost:5010
启动Crank-Agent
启动agent后执行crank，会有一个如下安装sdk的动作
- [09:29:05.261] Runtime: 6.0.0 (Current)
- [09:29:05.262] SDK: 6.0.100 (Current)
- [09:29:05.263] ASP.NET: 6.0.0 (Current)
- [09:29:05.265] Creating custom global.json
- [09:29:05.266] Desktop: 6.0.0 (Current)
- [09:29:05.266] Installing SDK '6.0.100' ...
所以启动agent时多指定一个dotnethome参数，避免重复安装sdk。
- crank-agent --dotnethome 'C:\Program Files\dotnet'
启动Crank
crank --config .\demo.benchmarks.yml --scenario crank_demo --profile local
--scenario crank_demo: 指定定义的测试场景
--profile local :设置输出结果到本地，即控制台输出
可以通过参数指定结果输出到本地json文件(--output results.json )或者数据库(--sql [connection-string] --table [table-name])
结果输出
这里省略压测期间的日志输出，直接列出结果。
- | application | |
- | --------------------- | ------------- |
- | CPU Usage (%) | 56 |
- | Cores usage (%) | 447 |
- | Working Set (MB) | 140 |
- | Private Memory (MB) | 157 |
- | Build Time (ms) | 7,232 |
- | Start Time (ms) | 501 |
- | Published Size (KB) | 91,292 |
- | .NET Core SDK Version | 6.0.100 |
- | ASP.NET Core Version | 6.0.0+ae1a6cb |
- | .NET Runtime Version | 6.0.0+4822e3c |
- | load | |
- | --------------------- | -------------- |
- | CPU Usage (%) | 46 |
- | Cores usage (%) | 370 |
- | Working Set (MB) | 29 |
- | Private Memory (MB) | 30 |
- | Build Time (ms) | 11,891 |
- | Start Time (ms) | 226 |
- | Published Size (KB) | 68,228 |
- | .NET Core SDK Version | 3.1.415 |
- | ASP.NET Core Version | 3.1.21+458d974 |
- | .NET Runtime Version | 3.1.21+df8abc0 |
- | First Request (ms) | 185 |
- | Requests | 912,005 |
- | Bad responses | 0 |
- | Mean latency (us) | 4,207 |
- | Max latency (us) | 138,999 |
- | Requests/sec | 60,305 |
- | Requests/sec (max) | 128,523 |
更多
crank readme： https://github.com/dotnet/crank/blob/main/docs/README.md
Benchmarks crank为Benchmarks重构版本
本文示例代码： https://github.com/MayueCif/crank_demo
