doc/index.html in bigbench-0.0.4 vs doc/index.html in bigbench-0.0.5
- old
+ new
@@ -75,26 +75,24 @@
<li><a href="./BigBench/Bot.html">BigBench::Bot</a>
<li><a href="./BigBench/Configuration.html">BigBench::Configuration</a>
- <li><a href="./BigBench/Configuration/Config.html">BigBench::Configuration::Config</a>
-
- <li><a href="./BigBench/Configuration/InvalidOptions.html">BigBench::Configuration::InvalidOptions</a>
-
<li><a href="./BigBench/Executor.html">BigBench::Executor</a>
- <li><a href="./BigBench/Executor/InvalidCommand.html">BigBench::Executor::InvalidCommand</a>
+ <li><a href="./BigBench/Executor/Executable.html">BigBench::Executor::Executable</a>
<li><a href="./BigBench/Fragment.html">BigBench::Fragment</a>
<li><a href="./BigBench/Fragment/Fragment.html">BigBench::Fragment::Fragment</a>
<li><a href="./BigBench/Output.html">BigBench::Output</a>
<li><a href="./BigBench/PostProcessor.html">BigBench::PostProcessor</a>
+ <li><a href="./BigBench/PostProcessor/Context.html">BigBench::PostProcessor::Context</a>
+
<li><a href="./BigBench/PostProcessor/Environment.html">BigBench::PostProcessor::Environment</a>
<li><a href="./BigBench/PostProcessor/Environment/Appearings.html">BigBench::PostProcessor::Environment::Appearings</a>
<li><a href="./BigBench/PostProcessor/Environment/AttributeCluster.html">BigBench::PostProcessor::Environment::AttributeCluster</a>
@@ -121,10 +119,12 @@
<li><a href="./BigBench/PostProcessor/Statistics.html">BigBench::PostProcessor::Statistics</a>
<li><a href="./BigBench/PostProcessor/Test.html">BigBench::PostProcessor::Test</a>
+ <li><a href="./BigBench/PostProcessor/TestPretty.html">BigBench::PostProcessor::TestPretty</a>
+
<li><a href="./BigBench/Runner.html">BigBench::Runner</a>
<li><a href="./BigBench/Runner/NoBenchmarksDefined.html">BigBench::Runner::NoBenchmarksDefined</a>
<li><a href="./BigBench/Store.html">BigBench::Store</a>
@@ -195,16 +195,16 @@
<h2 id="label-Test+Receipts">Test Receipts</h2>
<p>How do the test receipts look like? As easy as possible. For example like
this in <code>example.rb</code>:</p>
-<pre class="ruby"><span class="ruby-identifier">configure</span> =<span class="ruby-operator">></span> {
- :<span class="ruby-identifier">duration</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>.<span class="ruby-identifier">minutes</span>,
- :<span class="ruby-identifier">output</span> =<span class="ruby-operator">></span> <span class="ruby-string">"example.ljson"</span>,
- :<span class="ruby-identifier">users</span> =<span class="ruby-operator">></span> <span class="ruby-value">5</span>,
- :<span class="ruby-identifier">basic_auth</span> =<span class="ruby-operator">></span> [<span class="ruby-string">'username'</span>, <span class="ruby-string">'password'</span>]
-}
+<pre class="ruby"><span class="ruby-constant">BigBench</span>.<span class="ruby-identifier">configure</span> <span class="ruby-keyword">do</span> <span class="ruby-operator">|</span><span class="ruby-identifier">config</span><span class="ruby-operator">|</span>
+ <span class="ruby-identifier">config</span>.<span class="ruby-identifier">duration</span> = <span class="ruby-value">2</span>.<span class="ruby-identifier">minutes</span>
+ <span class="ruby-identifier">config</span>.<span class="ruby-identifier">output</span> = <span class="ruby-string">"example.ljson"</span>
+ <span class="ruby-identifier">config</span>.<span class="ruby-identifier">users</span> = <span class="ruby-value">5</span>
+ <span class="ruby-identifier">config</span>.<span class="ruby-identifier">basic_auth</span> = [<span class="ruby-string">'username'</span>, <span class="ruby-string">'password'</span>]
+<span class="ruby-keyword">end</span>
<span class="ruby-identifier">benchmark</span> <span class="ruby-string">"default website pages"</span> =<span class="ruby-operator">></span> <span class="ruby-string">"http://localhost:3000"</span> <span class="ruby-keyword">do</span>
<span class="ruby-identifier">get</span> <span class="ruby-string">"/"</span>
<span class="ruby-identifier">get</span> <span class="ruby-string">"/blog"</span>
<span class="ruby-identifier">get</span> <span class="ruby-string">"/imprint"</span>
@@ -217,10 +217,16 @@
<span class="ruby-keyword">end</span>
<span class="ruby-identifier">post_process</span> :<span class="ruby-identifier">statistics</span>
</pre>
+<h3 id="label-Generator">Generator</h3>
+
+<p>You can have your test receipts generated!</p>
+
+<pre>bigbench generate sample</pre>
+
<h2 id="label-Single+Host+vs.+Multiple+Hosts+Testing">Single Host vs. Multiple Hosts Testing</h2>
<p>You can either test with a single machine right from your local host, or
with multiple machines using bots. No matter what, the test receipt will
stay the same.</p>
@@ -228,33 +234,33 @@
<h3 id="label-Single+Host">Single Host</h3>
<p><a href="BigBench.html">BigBench</a> allows you to run your tests against
every host from your local machine. The command for this looks like this:</p>
-<pre>bigbench run local example.rb</pre>
+<pre>bigbench local example.rb</pre>
<h3 id="label-Multiple+Hosts+with+Bots">Multiple Hosts with Bots</h3>
<p><a href="BigBench.html">BigBench</a> uses a bot design pattern which means
you can run your tests from multiple hosts. Everything you need for this is
a redis that is reachable from all testing hosts. Every host simply starts
a bot that is checking for a new test receipt every minute like this:</p>
-<pre>bigbench start bot redis_url:port redis_password</pre>
+<pre>bigbench bot redis_url:port redis_password</pre>
<p>Then to run the tests from all hosts simply use the same receipt as you
would use for a local run and call it like this:</p>
-<pre>bigbench run bots example.rb redis_url:port redis_password</pre>
+<pre>bigbench bots example.rb redis_url:port redis_password</pre>
<p>This will upload the test receipt to all bots and make them run it. Every
bot reports its results back to the redis and the local machine then
combines, and writes them to the output file. So you test with the same
receipts and get the same results, no matter if your testing from the local
host or with multiple bots.</p>
-<p><img src="http://southdesign.github.com/bigbench/images/structure.png" /></p>
+<p><img src="http://southdesign.github.com/bigbench/graphs/structure.png" /></p>
<h2 id="label-Output">Output</h2>
<p>How does the recorded output look like? It’s in the <code>*.ljson</code>
format which is nothing else but a textfile with a complete JSON object on
@@ -303,11 +309,12 @@
<span class="ruby-constant">Twitter</span>.<span class="ruby-identifier">post</span> <span class="ruby-node">"Just run BigBench with #{total_trackings} trackings and #{total_errors} errors."</span>
<span class="ruby-keyword">end</span>
</pre>
-<p>It’s very easy to write a post processor. The basic structure is like this:</p>
+<p>It’s also very easy to write an own post processor. The basic structure is
+like this:</p>
<pre class="ruby"><span class="ruby-keyword">module</span> <span class="ruby-constant">BigBench</span>
<span class="ruby-keyword">module</span> <span class="ruby-constant">PostProcessor</span>
<span class="ruby-keyword">module</span> <span class="ruby-constant">SamplePostProcessor</span>
@@ -324,291 +331,45 @@
<pre>post_process :sample_post_processor
# or
post_process BigBench::PostProcessor::SamplePostProcessor</pre>
-<h3 id="label-Post+Processor+Environment">Post Processor Environment</h3>
+<h3 id="label-Post+Processor+Environment+API">Post Processor Environment API</h3>
-<p>Post processors by default offer a great load of functionality that helps
-to evaluate the benchmarks. The available methods are:</p>
+<p><a href="BigBench.html">BigBench</a> automatically supports a great load of
+functionality for every post processor it would need anyways. This
+functionality is offered through the</p>
+<ul><li>
+<p><a
+href="https://github.com/southdesign/bigbench/wiki/Post-Processor-Environment-API">Post
+Processor Environment API</a></p>
+</li></ul>
-<h4 id="label-each_tracking">each_tracking</h4>
+<h3 id="label-Included+Post+Processors">Included Post Processors</h3>
-<p>Iterate over each of the tracking elements. The trackings are read
-line-by-line. This is the fastest approach and should be used for huge
-datasets because the trackings are not loaded completely into memory.</p>
+<p>By default <a href="BigBench.html">BigBench</a> ships with a few very
+useful post processors that might already fit your needs perfectly. The
+full list of included post processors is shown in the wiki:</p>
+<ul><li>
+<p><a href="https://github.com/southdesign/bigbench/wiki/Post-Processors">Post
+Processors</a></p>
+</li></ul>
-<pre class="ruby"><span class="ruby-identifier">total_trackings</span>, <span class="ruby-identifier">total_errors</span> = <span class="ruby-value">0</span>, <span class="ruby-value">0</span>
-<span class="ruby-identifier">each_tracking</span> <span class="ruby-keyword">do</span> <span class="ruby-operator">|</span><span class="ruby-identifier">tracking</span><span class="ruby-operator">|</span>
- <span class="ruby-identifier">total_trackings</span> <span class="ruby-operator">+=</span> <span class="ruby-value">1</span>
- <span class="ruby-identifier">total_errors</span> <span class="ruby-operator">+=</span> <span class="ruby-value">1</span> <span class="ruby-keyword">unless</span> <span class="ruby-identifier">tracking</span>[:<span class="ruby-identifier">status</span>] <span class="ruby-operator">==</span> <span class="ruby-value">200</span>
-<span class="ruby-keyword">end</span>
-</pre>
-
-<h4 id="label-trackings">trackings</h4>
-
-<p>An array with all tracking hashes in it. The creation might take some time
-at the first usage, afterwards the array is cached automatically.</p>
-
-<pre class="ruby"><span class="ruby-identifier">trackings</span>.<span class="ruby-identifier">size</span> <span class="ruby-comment"># => 650456</span>
-<span class="ruby-identifier">trackings</span>.<span class="ruby-identifier">each</span> <span class="ruby-keyword">do</span> <span class="ruby-operator">|</span><span class="ruby-identifier">tracking</span><span class="ruby-operator">|</span>
- <span class="ruby-identifier">puts</span> <span class="ruby-identifier">tracking</span>[:<span class="ruby-identifier">duration</span>]
-<span class="ruby-keyword">end</span>
-</pre>
-
-<h4 id="label-statistics">statistics</h4>
-
-<p>Computes the default statistics for any attribute</p>
-
-<pre class="ruby"><span class="ruby-comment"># Unclustered statistics</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">max</span> <span class="ruby-comment"># => 78.2</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">min</span> <span class="ruby-comment"># => 12.3</span>
-
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">mean</span> <span class="ruby-comment"># => 45.2</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">average</span> <span class="ruby-comment"># => 45.2</span>
-
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">standard_deviation</span> <span class="ruby-comment"># => 11.3</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">sd</span> <span class="ruby-comment"># => 11.3</span>
-
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">squared_deviation</span> <span class="ruby-comment"># => 60.7</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">variance</span> <span class="ruby-comment"># => 60.7</span>
-
-<span class="ruby-comment"># Time clustered statistics - 1.second</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">max</span> <span class="ruby-comment"># => 42.1</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">min</span> <span class="ruby-comment"># => 12.3</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">max</span> <span class="ruby-comment"># => 42.1</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">average</span> <span class="ruby-comment"># => 33.1</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">benchmark</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">average</span> <span class="ruby-comment"># => 32.9</span>
-<span class="ruby-identifier">statistics</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">average</span> <span class="ruby-comment"># => 12.5</span>
-</pre>
-
-<h4 id="label-cluster">cluster</h4>
-
-<p>Clusters the resulting trackings by a timebase. The default timebase is
-<code>1.second</code> which means, that it groups all trackings to full
-seconds and calculates the amount of requests and the average duration.</p>
-
-<pre class="ruby"><span class="ruby-comment"># Duration is 120 seconds for this example</span>
-
-<span class="ruby-comment"># 1.second</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">timesteps</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [43, 96, ..., 41]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [503, 541, ..., 511]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [200, 204, ..., 209]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">post</span>) <span class="ruby-comment"># => [201, 102, ..., 401]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>) <span class="ruby-comment"># => [501, 502, ..., 102]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">404</span>) <span class="ruby-comment"># => [3, 1, ..., 0]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>) <span class="ruby-comment"># => [401, 482, ..., 271]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/logout"</span>) <span class="ruby-comment"># => [56, 51, ..., 38]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">benchmark</span>(<span class="ruby-string">"index"</span>) <span class="ruby-comment"># => [342, 531, ..., 234]</span>
-<span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">benchmark</span>(<span class="ruby-string">"user"</span>) <span class="ruby-comment"># => [22, 41, ..., 556]</span>
-
-<span class="ruby-comment"># 1.minute</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">timesteps</span> <span class="ruby-comment"># => [0, 1]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [42, 44]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [24032, 21893]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [200, 204]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>) <span class="ruby-comment"># => [501, 502]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>) <span class="ruby-comment"># => [401, 482]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">benchmark</span>(<span class="ruby-string">"user"</span>) <span class="ruby-comment"># => [22, 41]</span>
-
-<span class="ruby-comment"># 30.seconds</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">timesteps</span> <span class="ruby-comment"># => [0, 1, 2, 3]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [42, 44, 41, 40]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [11023, 10234, 12345, 13789]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [200, 204, 34, 124]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>) <span class="ruby-comment"># => [501, 502, 243, 57]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>) <span class="ruby-comment"># => [401, 482, 124, 234]</span>
-<span class="ruby-identifier">cluster</span>(<span class="ruby-value">30</span>.<span class="ruby-identifier">seconds</span>).<span class="ruby-identifier">benchmark</span>(<span class="ruby-string">"user"</span>) <span class="ruby-comment"># => [22, 41, 12, 51]</span>
-</pre>
-
-<h4 id="label-appearing">appearing</h4>
-
-<p>Lists the unique attribute values that appeared in all trackings or the
-selected tracking scope.</p>
-
-<pre class="ruby"><span class="ruby-identifier">appearing</span>.<span class="ruby-identifier">statuses</span> <span class="ruby-comment"># => [200, 404]</span>
-<span class="ruby-identifier">appearing</span>.<span class="ruby-identifier">methods</span> <span class="ruby-comment"># => ["get", "post"]</span>
-<span class="ruby-identifier">appearing</span>.<span class="ruby-identifier">paths</span> <span class="ruby-comment"># => ["/", "/basic/auth"</span>
-</pre>
-
-<h4 id="label-polynomial_regression">polynomial_regression</h4>
-
-<p>The polynomial regression creates a function that tries to map the test
-data best. With this function you have the ability do derivate and thereby
-plot the changes of the tested system over time. The degree of the
-regression can be freely chosen.</p>
-
-<pre class="ruby"><span class="ruby-comment"># Linear regression by default</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [45, 23, ..., 36]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [78, 12, ..., 63]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [12, 45, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">degree</span> <span class="ruby-comment"># => 1</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">formula</span> <span class="ruby-comment"># => "43.00886000234 + 0.0167548964060689x^1"</span>
-
-<span class="ruby-comment"># 1. Derivation</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [0.01, 0.01, ..., 0.01]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [405, 405, ..., 406]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [23, 62, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [78, 12, ..., 63]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">derivation</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => [12, 45, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">formula</span>(<span class="ruby-value">1</span>) <span class="ruby-comment"># => "0.0167548964060689"</span>
-
-<span class="ruby-comment"># Quadratic regression</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 41, ..., 44]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 41, ..., 44]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [78, 12, ..., 63]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [12, 45, ..., 23]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">formula</span> <span class="ruby-comment"># => "33.00886000234 + 0.0167548964060689x^1 + 0.0167548964060689x^2"</span>
-
-<span class="ruby-comment"># Different timebase clustering</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [0, 1]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [24032, 21893]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 41]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 41]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [78, 12]</span>
-<span class="ruby-identifier">polynomial_regression</span>(:<span class="ruby-identifier">degree</span> =<span class="ruby-operator">></span> <span class="ruby-value">2</span>, :<span class="ruby-identifier">timebase</span> =<span class="ruby-operator">></span> <span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [12, 45]</span>
-</pre>
-
-<h4 id="label-normal_distribution">normal_distribution</h4>
-
-<p>The normal distribution method creates a Gaussian bell function that
-visualizes the distribution of a special attribute. If you want to know if
-all your requests take about the same time, or if they vary a lot this is
-method to use. The x-values are automatically scaled to 4-times the
-variance around the mean, so it should map the whole bell all the time.</p>
-
-<pre class="ruby"><span class="ruby-comment"># Normal distribution without time clustering</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [45, 23, ..., 36]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">formula</span> <span class="ruby-comment"># => "(1 / (10.242257627240862 * sqrt(2*PI))) * exp(-1 * ((x - 2.04671984377919)^2) / (2*10.242257627240862))"</span>
-
-<span class="ruby-comment"># Normal distribution with default time slicing of 1.second</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [78, 12, ..., 63]</span>
-<span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [12, 45, ..., 23]</span>
-
-<span class="ruby-comment"># Normal distribution with custom time slicing</span>
-<span class="ruby-identifier">normal_distribution</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
-<span class="ruby-identifier">normal_distribution</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
-<span class="ruby-identifier">normal_distribution</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-identifier">normal_distribution</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">paths</span>(<span class="ruby-string">"/"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [78, 12, ..., 63]</span>
-<span class="ruby-identifier">normal_distribution</span>(<span class="ruby-value">1</span>.<span class="ruby-identifier">minute</span>).<span class="ruby-identifier">benchmarks</span>(<span class="ruby-string">"index page"</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [12, 45, ..., 23]</span>
-</pre>
-
-<h4 id="label-scope_to_benchmark">scope_to_benchmark</h4>
-
-<p>The scope_to_benchmark method lets you scope any result to a single
-benchmark. The values computed in this block have entirely been created by
-this benchmark.</p>
-
-<pre class="ruby"><span class="ruby-comment"># Results for the index page benchmark</span>
-<span class="ruby-identifier">scope_to_benchmark</span> <span class="ruby-string">"index page"</span> <span class="ruby-keyword">do</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [43, 96, ..., 41]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [503, 541, ..., 511]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [200, 204, ..., 209]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">post</span>) <span class="ruby-comment"># => [201, 102, ..., 401]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [45, 23, ..., 36]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-keyword">end</span>
-
-<span class="ruby-comment"># Results for the login and logout benchmark</span>
-<span class="ruby-identifier">scope_to_benchmark</span> <span class="ruby-string">"login and logout"</span> <span class="ruby-keyword">do</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [43, 96, ..., 41]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [300, 141, ..., 511]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [100, 204, ..., 209]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">post</span>) <span class="ruby-comment"># => [101, 102, ..., 401]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [45, 23, ..., 36]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-keyword">end</span>
-</pre>
-
-<h4 id="label-each_benchmark">each_benchmark</h4>
-
-<p>Iterates over all benchmarks and automatically scopes the results at each
-iteration to the current benchmark. This is useful if you want to access
-the detailed differences of each benchmark.</p>
-
-<pre class="ruby"><span class="ruby-comment"># Iterate over all benchmarks and calculate the results</span>
-<span class="ruby-identifier">each_benchmark</span> <span class="ruby-keyword">do</span> <span class="ruby-operator">|</span><span class="ruby-identifier">benchmark</span><span class="ruby-operator">|</span>
- <span class="ruby-identifier">benchmark</span>.<span class="ruby-identifier">name</span> <span class="ruby-comment"># => "index page" then "login and logout"</span>
-
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">durations</span> <span class="ruby-comment"># => [43, 96, ..., 41]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">requests</span> <span class="ruby-comment"># => [300, 141, ..., 511]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>) <span class="ruby-comment"># => [100, 204, ..., 209]</span>
- <span class="ruby-identifier">cluster</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">post</span>) <span class="ruby-comment"># => [101, 102, ..., 401]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">x</span> <span class="ruby-comment"># => [1, 2, ..., 120]</span>
- <span class="ruby-identifier">polynomial_regression</span>.<span class="ruby-identifier">durations</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [45, 23, ..., 36]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">requests</span>.<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [43, 45, ..., 62]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">methods</span>(:<span class="ruby-identifier">get</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [23, 62, ..., 23]</span>
- <span class="ruby-identifier">normal_distribution</span>.<span class="ruby-identifier">statuses</span>(<span class="ruby-value">200</span>).<span class="ruby-identifier">y</span> <span class="ruby-comment"># => [51, 22, ..., 15]</span>
-<span class="ruby-keyword">end</span>
-</pre>
-
<h3 id="label-Running+Post+Processors+separately">Running Post Processors separately</h3>
<p>You can also re-run the currently defined post processors or run a separate
post processor you never even defined in the first place without collecting
the test data again like this:</p>
<pre># Re-run the postprocessors defined in example.rb
-bigbench run postprocessors example.rb
+bigbench process example.rb
# Run a separate post processor independently - the already defined post processors are ignored
-bigbench run postprocessor example.rb statistics</pre>
+bigbench process example.rb statistics</pre>
<p>Contribute, create great post processors and send me a pull request!</p>
-<h3 id="label-Statistics">Statistics</h3>
-
-<p>The statistics post processor computes a simple overview of the benchmark
-and prints it to the terminal like this:</p>
-
-<pre>BigBench Statistics
-+---------------------------+------------------+---------+
-| Name | Value | Percent |
-+---------------------------+------------------+---------+
-| Total Requests: | 52,469 | 100% |
-| Total Errors: | 0 | 0.0% |
-| | | |
-| Average Requests/Second: | 437 Requests/sec | |
-| Average Request Duration: | 1 ms | |
-| | | |
-| Max Request Duration: | 181 ms | |
-| Min Request Duration: | 1 ms | |
-| | | |
-| Status Codes: | | |
-| 200 | 52469 | 100.0% |
-| | | |
-| HTTP Methods | | |
-| get | 34980 | 66.7% |
-| post | 17489 | 33.3% |
-| | | |
-| URL Paths: | | |
-| / | 34979 | 66.7% |
-| /basic/auth | 17490 | 33.3% |
-+---------------------------+------------------+---------+
-19 rows in set</pre>
-
<h2 id="label-Load+Comparison">Load Comparison</h2>
<p><a href="BigBench.html">BigBench</a> is awfully good at creating high loads
on web servers. A quick benchmark comparison to <a
href="http://jmeter.apache.org/">Apache's JMeter</a> shows that <a
@@ -684,9 +445,18 @@
</tr>
</table>
<h2 id="label-Version+History">Version History</h2>
+
+<h3 id="label-0.5">0.5</h3>
+<ul><li>
+<p>Changed <code>configure</code> syntax to a common ruby pattern block style</p>
+</li><li>
+<p>Refactored and simplified command line usage with thor</p>
+</li><li>
+<p>Added a generator for test files</p>
+</li></ul>
<h3 id="label-0.4">0.4</h3>
<ul><li>
<p>Added command line tool to run the post processors again</p>
</li><li>