# ProxyTester - testing with rspec for http and socks proxies

The `proxy_tester` helps you maintaining your proxy infrastructure by writing
tests which can be run automatically. It uses `rspec` in the background to make
that thing possible and adds some helper methods to make testing proxies
easier.

Today `proxy_tester` supports "HTTP"- and "SOCKS5"-Proxies.

## Installation

Add this line to your application's Gemfile:

    gem 'proxy_tester'

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install proxy_tester

## Usage

### Quick start

**Initialize proxy tester**

This will create all files and directories needed to run proxy tester. Thanks
to `--pre-seed` it will also add some example files to your test
infrastructure, e.g. `spec_helper`, `example_spec.rb`.

```bash
% proxy_tester init --pre-seed
```

**Modify example spec**

1. Add your proxy, e.g. `localhost:3128` to the spec
2. Add a user + password to `users.csv` and to the `example_spec.rb` if your proxy needs authentication

**Run tests**

```bash
% proxy_tester test rspec
```

**Getting in touch with the screenshots taken during test**

```bash
# Please choose your preferred viewer, e.g. feh and the correct date/time for your test
% feh ~/.local/share/proxy_tester/reports.d/2014-03-19_17:22:00/*.png
```

### Getting started (WIP)

**Getting help**

```bash
% proxy_tester help
% proxy_tester help <cmd>
```

**Prepare a configuration file for proxy_tester**

For more details please see the chapter [Writing Configuration File](#write_config) for
that.

**Initialize proxy tester**

This will create all files and directories needed to run proxy tester.

```bash
% proxy_tester init
```

By default it creates the following directories/files:

* `$HOME/.local/share/proxy_tester`
* `$HOME/.local/share/proxy_tester/reports.d`
* `$HOME/.config/proxy_tester`
* `$HOME/.config/proxy_tester/testcases.d`
* `$HOME/.config/proxy_tester/config.yaml`
* `$HOME/.config/proxy_tester/users.csv`

`proxy_tester` supports different options for init


```bash
% proxy_tester help init

# Usage:
#   proxy_tester init
# 
# Options:
#   [--force]                                      # Overwrite existing files?
#   [--pre-seed]                                   # Add examples to test cases directory
#   [--test-cases-directory=TEST_CASES_DIRECTORY]  # Directory with test cases
#   [--user-file=USER_FILE]                        # Path to user file
#   [--create-config-file]                         # Create config file
#                                                  # Default: true
#   [--create-user-file]                           # Create user file
#                                                  # Default: true
#   [--create-test-cases-directory]                # Create directory to store test cases
#                                                  # Default: true
#   [--config-file=CONFIG_FILE]                    # Config file
#   [--log-level=LOG_LEVEL]                        # Log level for ui logging
#   [--debug-mode]                                 # Run application in debug mode
```

**Adding users to users.csv**

If your proxy needs user authentication you can use the `use_user`-method
together with the `users.csv`. Make sure you keep the users.csv safe (and out
of a central vcs) because the passwords need to be stored in plain text.


```csv
"name","password","description"
"user1","password1","test user abc"
"user2","password2","test user efg"
```

In your spec you can refer to the users by using the following code snippet.
There is no need to write the password into the spec file. `proxy_helper` will
look it up.

```ruby
it 'blocks www.example.org' do
  use_proxy 'localhost:3128'
  use_user 'user1'

  use_timeout(100) do
    visit 'http://www.example.org'
  end

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

**Writing specs**

For more details please see the chapter [Writing Specs](#writing_specs) for
that.

**Run the tests**

Run the tests on commandline

```bash
% proxy_tester test
```

It supports several different options:

```bash
% proxy_tester help test

# Usage:
#   proxy_tester test
# 
# Options:
#   [--test-cases-directory=TEST_CASES_DIRECTORY]  # Directory with test cases
#   [--tags=one two three]                         # Filter tests based on tags
#   [--config-file=CONFIG_FILE]                    # Config file
#   [--log-level=LOG_LEVEL]                        # Log level for ui logging
#   [--debug-mode]                                 # Run application in debug mode
# 
```

If you want to limit the tests which should be run by `proxy_tester` you can
use the `--tags`-switch. Make sure you've flagged your examples correctly - see
[rspec documentation](https://www.relishapp.com/rspec/rspec-core/v/2-4/docs/command-line/tag-option)
for more about this topic.


## Writing Specs
<a name="writing_specs"></a>

### Introduction

**Recommended structure for testcases**

For my specs I use the following structure. Additionally I use `git` for
version control.

```
testcase.d
  infrastructure1
    infrastructure1_abc_spec.rb
    infrastructure1_def_spec.rb
    spec_helper.rb
    support
      rspec.rb
      shared_examples-base.rb
      [...]
```

**The spec file(s)**

I took screenshots for all my tests in an `after`-hook to be able to provide reports after
testing. For each environment, e.g. `production`, `staging`, I create a diffent
context adding a flag, e.g `:production`, for easier filterung. For each proxy
I add a context and set the subject to the proxy hostname/ip address + port.
The `it_behaves_like`-method is special because it references a shared example.
More on this can be found later.

Name: `infrastructure1_abc_spec.rb`

```ruby
# encoding: utf-8
describe 'Infrastructure XY' do
  after :each do
    take_screenshot
  end

  before :each do
    use_user 'user1'
  end

  context 'Production', :production do
    context 'proxy1' do
      subject { 'proxy1.localdomain:8080' }

      it_behaves_like 'a base proxy'
    end

    context 'proxy2' do
      subject { 'proxy2.localdomain:8080' }

      it_behaves_like 'a base proxy'
    end
  end

  context 'Staging', :staging do
    context 'proxy1-int' do
      subject { 'proxy1-int.localdomain:8080' }

      it_behaves_like 'a base proxy'
    end

    context 'proxy2-int' do
      subject { 'proxy2-int.localdomain:8080' }

      it_behaves_like 'a base proxy'
    end
  end
end
```

**The spec helper file**

In my spec helper file I load all files found in support to make shared
examples and other files available.

Name: `spec_helper.rb`

```ruby
# encoding: utf-8

# Loading support files
Dir.glob(::File.expand_path('../support/*.rb', __FILE__)).each { |f| require_relative f }

begin
  require 'bundler'
  Bundler.require :default
rescue
  $stderr.puts 'Please install ruby gem "bundler" before using this spec_helper.rb in your proxy_tester-spec files.'
  exit(-1)
end

# *open-uri*
# Easier use of urls in scripts
# Please see "http://www.ruby-doc.org/stdlib-2.1.1/libdoc/open-uri/rdoc/OpenURI.html" for further information
require 'open-uri'
```

**The spec configuration**

I want `rspec` to run only those examples which are flagged with `:focus`. If
there is no example with a flag `:focus`. It should run all examples. I use the
`:focus`-flag while developing a test or debugging existing tests.

Name: `rspec.rb`

```ruby
RSpec.configure do |c|
  c.filter_run_including :focus => true
  c.run_all_when_everything_filtered                = true
  c.treat_symbols_as_metadata_keys_with_true_values = true
end
```

**Shared examples: base**

To `DRY` my examples up I use shared examples. Please see [rspec
documentation](https://www.relishapp.com/rspec/rspec-core/docs/example-groups/shared-examples)
for more detailed information.

Name: `shared_examples-base.rb`

```ruby
# encoding: utf-8
require_relative 'shared_examples'

shared_examples 'a base proxy' do
  before :each do
    use_proxy subject
  end

  it 'works with http://www.example.org' do
    visit 'http://www.example.org'
    expect(page).to have_content('Example Domain')
    expect(page.status_code).to eq 200
  end

  it 'blocks eicar virus' do
    visit 'http://www.eicar.org/download/eicar.com'
    find('div#lk_info').trigger('click')

    expect(page).to have_content('Access blocked')
    expect(page.status_code).to eq 403
  end
end
```

### Helper methods

This gem includes all helper methods provided by
[`capybara`](https://github.com/jnicklas/capybara) and
[`poltergeist`](https://github.com/jonleighton/poltergeist) plus those listed
below. To see the most current list of available methods please look for
`lib/proxy_tester/rspec/helper.rb`.

* `runtime`

This method returns the time of `proxy_tester`-run.

```ruby
# time of run, YYYY-MM-DD_HH:mm:ss
runtime
```

* proxy

This method returns the proxy set earlier. It holds the proxy-object.

```ruby
# proxy host
proxy.host
proxy.host = 'host'

# proxy port
proxy.port
proxy.port = 3128

# proxy type :none, :http, :socks5
proxy.type
proxy.type = :none

# is blank?
proxy.blank?
```

* proxy_pac

This method returns the proxy pac set earlier. It holds the proxy_pac-object.

```ruby
# proxy host
proxy_pac.host

# proxy port
proxy_pac.port

# client ip
proxy_pac.client_ip
proxy_pac.client_ip = '127.0.0.1'

# url to proxy pac
proxy_pac.url
proxy_pac.url = 'http://www.example.org'
proxy_pac.url = 'www.example.org' # heuristic parsing via addressable

# path/url to proxy pac-file
proxy_pac.pac_file
proxy_pac.pac_file = 'http://pac.in.domain.org/proxy.pac'
proxy_pac.pac_file = '/path/to/file.pac'

# is blank? => port + host = blank
proxy_pac.blank?

# direct? => no proxy
proxy_pac.direct?

```

* cleanup_reports

The gem supports a method to create a screenshot for a visited website. To
keep the `reports.d`-directory clean, the last 5 report-directories are kept only.

```ruby
before :all do
  cleanup_reports
end
```

If you want to change this, set the `@__keep_report_directories` to the needed
value in an `before(:all)`-hook.

```ruby
before :all do
  # keep the last 10 report directories
  @__keep_report_directories = 10
  cleanup_reports
end
```

* take_screenshot

If you want to take a screenshot of a visited website, you can use
`take_screenshot`. By default all screenshots are stored in `reports.d`.

```ruby
after :each do
  take_screenshot
end
```

* view_screenshot (alias: show_screenshot)

While debugging an example it might be handy to view the fetched website. This
can be done using the `view_screenshot`-helper. It opens the screenshot in your
preferred image viewer.

```ruby
context 'proxy1' do
  subject { 'proxy1.localdomain:8080' }

  it 'works with http://www.example.org' do
    use_proxy subject

    visit 'http://www.example.org'

    view_screenshot

    expect(page).to have_content('Example Domain')
    expect(page.status_code).to eq 200
  end

end
```

* use_user_agent

Set the user agent used by the driver.

```ruby
context 'proxy1' do
  subject { 'proxy1.localdomain:8080' }

  it 'works with http://www.example.org' do
    use_proxy subject

    use_user_agent 'Mozilla/5.0 (X11; Linux x86_64; rv:27.0) Gecko/20100101 Firefox/27.0'

    visit 'http://www.example.org'

    expect(page).to have_content('Example Domain')
    expect(page.status_code).to eq 200
  end
end
```

* use_user

Set the user used to connect to the proxy.

*Lookup user password using `users.csv`*

```ruby
it 'blocks www.example.org for user "user1"' do
  use_proxy subject
  use_user 'user1'

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

*Ask user for user name and user password*

```ruby
it 'blocks www.example.org for user' do
  use_proxy subject
  use_user :ask

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

*Ask user for user password*

```ruby
it 'blocks www.example.org for user' do
  use_proxy subject
  use_user 'user', :ask_password

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

*Build a special user name based on ENV*

```ruby
it 'blocks www.example.org for user' do
  use_proxy subject
  use_user 'user', :credential_merging
  # => ENV['USER']-user

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

* use_client_ip

Set the client ip address used during proxy pac evaluation.

```ruby
it 'blocks www.example.org' do
  use_proxy :pac, 'http://localhost/proxy.pac'
  use_client_ip '127.0.0.1'

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

* use_time

Set the time used during proxy pac evaluation.

```ruby
it 'blocks www.example.org' do
  use_proxy :pac, 'http://localhost/proxy.pac'
  use_time '2014-03-07 05:12:20'

  visit 'http://www.example.org'

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end

```

* use_proxy

Set the proxy host or proxy pac used to forward the traffic.

```ruby
use_proxy :host, 'localhost:3128'
use_proxy 'localhost:3128'
use_proxy :pac, 'http://localhost/proxy.pac'
```

* use_timeout

Set the timeout in seconds for visiting a website.

```ruby
it 'blocks www.example.org' do
  use_proxy 'localhost:3128'
  use_time '2014-03-07 05:12:20'

  use_timeout(100) do
    visit 'http://www.example.org'
  end

  expect(page).to have_content('Access forbidden')
  expect(page.status_code).to eq 403
end
```

### Custom matchers

* have_requests_with_status_code

Checks if a page requested other objects with given status code(s). Because
some sites support `chunked` data, the status code needs to be an array.

```ruby
it 'requested given resources' do
  use_proxy 'proxy1.localdomain:8080'
  use_user 'user1', :ask_password

  visit 'http://www.example.org'
  
  domains_with_status_code = {
    'http://www.example.org' => [200],
  }
  
  expect(page).to have_requests_with_status_code domains_with_status_code
end
```

* reference_domains

Checks if a page requested objects from given domains.

```ruby
it 'requested given resources' do
  use_proxy 'proxy1.localdomain:8080'
  use_user 'user1', :ask_password

  visit 'http://www.example.org'
  domains = %w{ www.example.org }

  expect(page).to reference_domains domains
end
```

## Writing Configuration File
<a name="write_config"></a>

The configuration file of `proxy_tester` is a simple
[yaml](http://www.yaml.org)-file. Those configuration variables are overwritten
if you choose a corresponding commandline-switch, e.g. `--config-file`-switch
for `config_file`-option. If a configuration option is not defined in config
file and not given on commandline `proxy_tester` uses default hardcoded within
the application. It supports the following variables.

```yaml
# configuration options with defaults
# $HOME = '/home/user'
:config_file: /home/user/.config/proxy_tester/config.yaml
:user_file: /home/user/.config/proxy_tester/user.csv
:test_cases_directory: /home/user/.config/proxy_tester/test_cases.d
:examples_directory: /home/user/.config/proxy_tester/test_cases.d/examples
:reports_directory: /home/user/.share/proxy_tester/reports.d
```

## Testing Proxies Directly

### Introduction

`proxy_tester` does also provides a mode where you can test a proxy directly by opening a url.

```bash
% proxy_tester test url --proxy localhost:3128 http://www.example.net
```

### Options

```
Usage:
  proxy_tester url --proxy=one two three

Options:
  --proxy=one two three    # System(s) under test (required)
  [--count=N]              # Number of requests
                           # Default: 10
  [--wait=N]               # Wait n seconds before kill children
                           # Default: 60
  [--user=USER]            # User for authentication
                           # Default: xgvndeg
  [--log-level=LOG_LEVEL]  # Log level
  [--concurrent]           # Simulate concurrent requests
  [--output=OUTPUT]        # Write output to file
```

### Usage

**Simple**

`proxy_tester` requires `--proxy`-argument with `host:port` and at least one
url given.

```bash
% proxy_tester test url --proxy localhost:3128 http://www.example.net
```

**Fetching multiple urls**

```bash
% proxy_tester test url --proxy localhost:3128 http://example.net http://www.example.org
```

**Fetching multiple urls using shell expansion**

```bash
% proxy_tester test url --proxy localhost:3128 http://example{1,2,3,4}.net
```

**Fetching url via multiple proxies using shell expansion**

```bash
% proxy_tester test url --proxy proxy{1,2,3}.localdomain:3128 http://example.net
```

**Fetching url n-times**

Using the `--count`-option `proxy_tester` tries to fetch the given url
`--count`-times within `--timeout` (default: 60s).

```bash
% proxy_tester test url --proxy proxy1.localdomain:3128 http://example.net  --count 5
```

**Fetching url concurrently via proxy using shell expansion**

Using the `--concurrent`-option `proxy_tester` creates 10 Threads which try to
fetch the url as often as possible during 60 s. To change the number of threads
you can use the `--count`-option. To change the timeout value please use
`--timeout`.

```bash
% proxy_tester test url --proxy proxy1.localdomain:3128 http://example.net --concurrent
```

**Write output to file instead of stdout**

```bash
% proxy_tester test url --proxy proxy1.localdomain:3128 http://example.net --output /tmp/file.log
```

## Contributing

1. Fork it ( http://github.com/dg-vrnetze/proxy_tester/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request