Generated on Wed Jul 19 17:39:34 EDT 2006 with rcov 0.6.0
Code reported as executed by Ruby looks like this... and this: this line is also marked as covered. Lines considered as run by rcov, but not reported by Ruby, look like this, and this: these lines were inferred by rcov (using simple heuristics). Finally, here's a line marked as not executed.
| Name | Total lines | Lines of code | Total coverage | Code coverage | ||||||||
| lib/rfuzz/client.rb | 498 | 275 |
|
|
1 require 'http11_client' 2 require 'socket' 3 require 'stringio' 4 require 'rfuzz/stats' 5 6 module RFuzz 7 8 9 # A simple hash is returned for each request made by HttpClient with 10 # the headers that were given by the server for that request. Attached 11 # to this are four attributes you can play with: 12 # 13 # * http_reason 14 # * http_version 15 # * http_status 16 # * http_body 17 # 18 # These are set internally by the Ragel/C parser so they're very fast 19 # and pretty much C voodoo. You can modify them without fear once you get 20 # the response. 21 class HttpResponse < Hash 22 # The reason returned in the http response ("OK","File not found",etc.) 23 attr_accessor :http_reason 24 25 # The HTTP version returned. 26 attr_accessor :http_version 27 28 # The status code (as a string!) 29 attr_accessor :http_status 30 31 # The http body of the response, in the raw 32 attr_accessor :http_body 33 34 # When parsing chunked encodings this is set 35 attr_accessor :http_chunk_size 36 end 37 38 # A mixin that has most of the HTTP encoding methods you need to work 39 # with the protocol. It's used by HttpClient, but you can use it 40 # as well. 41 module HttpEncoding 42 43 # Converts a Hash of cookies to the appropriate simple cookie 44 # headers. 45 def encode_cookies(cookies) 46 result = "" 47 cookies.each do |k,v| 48 if v.kind_of? Array 49 v.each {|x| result += encode_field("Cookie", encode_param(k,x)) } 50 else 51 result += encode_field("Cookie", encode_param(k,v)) 52 end 53 end 54 return result 55 end 56 57 # Encode HTTP header fields of "k: v\r\n" 58 def encode_field(k,v) 59 "#{k}: #{v}\r\n" 60 end 61 62 # Encodes the headers given in the hash returning a string 63 # you can use. 64 def encode_headers(head) 65 result = "" 66 head.each do |k,v| 67 if v.kind_of? Array 68 v.each {|x| result += encode_field(k,x) } 69 else 70 result += encode_field(k,v) 71 end 72 end 73 return result 74 end 75 76 # URL encodes a single k=v parameter. 77 def encode_param(k,v) 78 escape(k) + "=" + escape(v) 79 end 80 81 # Takes a query string and encodes it as a URL encoded 82 # set of key=value pairs with & separating them. 83 def encode_query(uri, query) 84 params = [] 85 86 if query 87 query.each do |k,v| 88 if v.kind_of? Array 89 v.each {|x| params << encode_param(k,x) } 90 else 91 params << encode_param(k,v) 92 end 93 end 94 95 uri += "?" + params.join('&') 96 end 97 98 return uri 99 end 100 101 # HTTP is kind of retarded that you have to specify 102 # a Host header, but if you include port 80 then further 103 # redirects will tack on the :80 which is annoying. 104 def encode_host(host, port) 105 "#{host}" + (port.to_i != 80 ? ":#{port}" : "") 106 end 107 108 # Performs URI escaping so that you can construct proper 109 # query strings faster. Use this rather than the cgi.rb 110 # version since it's faster. (Stolen from Camping). 111 def escape(s) 112 s.to_s.gsub(/([^ a-zA-Z0-9_.-]+)/n) { 113 '%'+$1.unpack('H2'*$1.size).join('%').upcase 114 }.tr(' ', '+') 115 end 116 117 118 # Unescapes a URI escaped string. (Stolen from Camping). 119 def unescape(s) 120 s.tr('+', ' ').gsub(/((?:%[0-9a-fA-F]{2})+)/n){ 121 [$1.delete('%')].pack('H*') 122 } 123 end 124 125 # Parses a query string by breaking it up at the '&' 126 # and ';' characters. You can also use this to parse 127 # cookies by changing the characters used in the second 128 # parameter (which defaults to '&;'. 129 def query_parse(qs, d = '&;') 130 params = {} 131 (qs||'').split(/[#{d}] */n).inject(params) { |h,p| 132 k, v=unescape(p).split('=',2) 133 if cur = params[k] 134 if cur.class == Array 135 params[k] << v 136 else 137 params[k] = [cur, v] 138 end 139 else 140 params[k] = v 141 end 142 } 143 144 return params 145 end 146 end 147 148 149 # The actual HttpClient that does the work with the thinnest 150 # layer between you and the protocol. All exceptions and leaks 151 # are allowed to pass through since those are important when 152 # testing. It doesn't pretend to be a full client, but instead 153 # is just enough client to track cookies, form proper HTTP requests, 154 # and return HttpResponse hashes with the results. 155 # 156 # It's designed so that you create one client, and then you work it 157 # with a minimum of parameters as you need. The initialize method 158 # lets you pass in defaults for most of the parameters you'll need, 159 # and you can simple call the method you want and it'll be translated 160 # to an HTTP method (client.get => GET, client.foobar = FOOBAR). 161 # 162 # Here's a few examples: 163 # 164 # client = HttpClient.new(:head => {"X-DefaultHeader" => "ONE"}) 165 # resp = client.post("/test") 166 # resp = client.post("/test", :head => {"X-TestSend" => "Status"}, :body => "TEST BODY") 167 # resp = client.put("/testput", :query => {"q" => "test"}, :body => "SOME JUNK") 168 # client.reset 169 # 170 # The HttpClient.reset call clears cookies that are maintained. 171 # 172 # It uses method_missing to do the translation of .put to "PUT /testput HTTP/1.1" 173 # so you can get into trouble if you're calling unknown methods on it. By 174 # default the methods are PUT, GET, POST, DELETE, HEAD. You can change 175 # the allowed methods by passing :allowed_methods => [:put, :get, ..] to 176 # the initialize for the object. 177 # 178 # == Notifications 179 # 180 # You can register a "notifier" with the client that will get called when 181 # different events happen. Right now the Notifier class just has a few 182 # functions for the common parts of an HTTP request that each take a 183 # symbol and some extra parameters. See RFuzz::Notifier for more 184 # information. 185 # 186 # == Parameters 187 # 188 # :head => {K => V} or {K => [V1,V2]} 189 # :query => {K => V} or {K => [V1,V2]} 190 # :body => "some body" (you must encode for now) 191 # :cookies => {K => V} or {K => [V1, V2]} 192 # :allowed_methods => [:put, :get, :post, :delete, :head] 193 # :notifier => Notifier.new 194 # :redirect => false (give it a number and it'll follow redirects for that count) 195 # 196 class HttpClient 197 include HttpEncoding 198 199 TRANSFER_ENCODING="TRANSFER_ENCODING" 200 CONTENT_LENGTH="CONTENT_LENGTH" 201 SET_COOKIE="SET_COOKIE" 202 LOCATION="LOCATION" 203 HOST="HOST" 204 HTTP_REQUEST_HEADER="%s %s HTTP/1.1\r\n" 205 206 # Access to the host, port, default options, and cookies currently in play 207 attr_accessor :host, :port, :options, :cookies, :allowed_methods, :notifier 208 209 # Doesn't make the connect until you actually call a .put,.get, etc. 210 def initialize(host, port, options = {}) 211 @options = options 212 @host = host 213 @port = port 214 @cookies = options[:cookies] || {} 215 @allowed_methods = options[:allowed_methods] || [:put, :get, :post, :delete, :head] 216 @notifier = options[:notifier] 217 @redirect = options[:redirect] || false 218 end 219 220 221 # Builds a full request from the method, uri, req, and @cookies 222 # using the default @options and writes it to out (should be an IO). 223 # 224 # It returns the body that the caller should use (based on defaults 225 # resolution). 226 def build_request(out, method, uri, req) 227 ops = @options.merge(req) 228 query = ops[:query] 229 230 # merge head differently since that's typically what they mean 231 head = req[:head] || {} 232 head = ops[:head].merge(head) if ops[:head] 233 234 # setup basic headers we always need 235 head[HOST] = encode_host(@host,@port) 236 head[CONTENT_LENGTH] = ops[:body] ? ops[:body].length : 0 237 238 # blast it out 239 out.write(HTTP_REQUEST_HEADER % [method, encode_query(uri,query)]) 240 out.write(encode_headers(head)) 241 out.write(encode_cookies(@cookies.merge(req[:cookies] || {}))) 242 out.write("\r\n") 243 ops[:body] || "" 244 end 245 246 def read_chunks(input, out, parser) 247 begin 248 until input.closed? 249 parser.reset 250 chunk = HttpResponse.new 251 line = input.readline("\r\n") 252 nread = parser.execute(chunk, line, 0) 253 254 if !parser.finished? 255 # tried to read this header but couldn't 256 return :incomplete_header, line 257 end 258 259 size = chunk.http_chunk_size ? chunk.http_chunk_size.to_i(base=16) : 0 260 261 if size == 0 262 return :finished, nil 263 end 264 remain = size -out.write(input.read(size)) 265 return :incomplete_body, remain if remain > 0 266 267 line = input.read(2) 268 if line.nil? or line.length < 2 269 return :incomplete_trailer, line 270 elsif line != "\r\n" 271 raise HttpClientParserError.new("invalid chunked encoding trailer") 272 end 273 end 274 rescue EOFError 275 # this is thrown when the header read is attempted and 276 # there's nothing in the buffer 277 return :eof_error, nil 278 end 279 end 280 281 def read_chunked_encoding(resp, sock, parser) 282 out = StringIO.new 283 input = StringIO.new(resp.http_body) 284 285 # read from the http body first, then continue at the socket 286 status, result = read_chunks(input, out, parser) 287 288 case status 289 when :incomplete_trailer 290 if result.nil? 291 sock.read(2) 292 else 293 sock.read(result.length - 2) 294 end 295 when :incomplete_body 296 out.write(sock.read(result)) # read the remaining 297 sock.read(2) 298 when :incomplete_header 299 # push what we read back onto the socket, but backwards 300 result.reverse! 301 result.each_byte {|b| sock.ungetc(b) } 302 when :finished 303 # all done, get out 304 out.rewind; return out.read 305 when :eof_error 306 # read everything we could, ignore 307 end 308 309 # then continue reading them from the socket 310 status, result = read_chunks(sock, out, parser) 311 312 # and now the http_body is the chunk 313 out.rewind; return out.read 314 end 315 316 # Reads an HTTP response from the given socket. It uses 317 # readpartial which only appeared in Ruby 1.8.4. The result 318 # is a fully formed HttpResponse object for you to play with. 319 # 320 # As with other methods in this class it doesn't stop any exceptions 321 # from reaching your code. It's for experts who want these exceptions 322 # so either write a wrapper, use net/http, or deal with it on your end. 323 def read_response(sock) 324 data, resp = nil, nil 325 parser = HttpClientParser.new 326 resp = HttpResponse.new 327 328 notify :read_header do 329 data = sock.readpartial(1024) 330 nread = parser.execute(resp, data, 0) 331 332 while not parser.finished? 333 data += sock.readpartial(1024) 334 nread += parser.execute(resp, data, nread) 335 end 336 end 337 338 notify :read_body do 339 if resp[TRANSFER_ENCODING] and resp[TRANSFER_ENCODING].index("chunked") 340 resp.http_body = read_chunked_encoding(resp, sock, parser) 341 elsif resp[CONTENT_LENGTH] 342 cl = resp[CONTENT_LENGTH].to_i 343 if cl - resp.http_body.length > 0 344 resp.http_body += sock.read(cl - resp.http_body.length) 345 elsif cl < resp.http_body.length 346 STDERR.puts "Web site sucks, they said Content-Length: #{cl}, but sent a longer body length: #{resp.http_body.length}" 347 end 348 else 349 resp.http_body += sock.read 350 end 351 end 352 353 if resp[SET_COOKIE] 354 cookies = query_parse(resp[SET_COOKIE], ';,') 355 @cookies.merge! cookies 356 @cookies.delete "path" 357 end 358 359 notify :close do 360 sock.close 361 end 362 363 resp 364 end 365 366 # Does the socket connect and then build_request, read_response 367 # calls finally returning the result. 368 def send_request(method, uri, req) 369 begin 370 sock = nil 371 notify :connect do 372 sock = TCPSocket.new(@host, @port) 373 end 374 375 out = StringIO.new 376 body = build_request(out, method, uri, req) 377 378 notify :send_request do 379 sock.write(out.string + body) 380 sock.flush 381 end 382 383 return read_response(sock) 384 rescue Object 385 raise $! 386 ensure 387 sock.close unless (!sock or sock.closed?) 388 end 389 end 390 391 392 # Translates unknown function calls into PUT, GET, POST, DELETE, HEAD 393 # methods. The allowed HTTP methods allowed are restricted by the 394 # @allowed_methods attribute which you can set after construction or 395 # during construction with :allowed_methods => [:put, :get, ...] 396 def method_missing(symbol, *args) 397 if @allowed_methods.include? symbol 398 method = symbol.to_s.upcase 399 resp = send_request(method, args[0], args[1] || {}) 400 resp = redirect(symbol, resp) if @redirect 401 402 return resp 403 else 404 raise "Invalid method: #{symbol}" 405 end 406 end 407 408 # Keeps doing requests until it doesn't receive a 3XX request. 409 def redirect(method, resp, *args) 410 @redirect.times do 411 break if resp.http_status.index("3") != 0 412 413 host = encode_host(@host,@port) 414 location = resp[LOCATION] 415 416 if location.index(host) == 0 417 # begins with the host so strip that off 418 location = location[host.length .. -1] 419 end 420 421 @notifier.redirect(:begins) if @notifier 422 resp = self.send(method, location, *args) 423 @notifier.redirect(:ends) if @notifier 424 end 425 426 return resp 427 end 428 429 # Clears out the cookies in use so far in order to get 430 # a clean slate. 431 def reset 432 @cookies.clear 433 end 434 435 436 # Sends the notifications to the registered notifier, taking 437 # a block that it runs doing the :begins, :ends states 438 # around it. 439 # 440 # It also catches errors transparently in order to call 441 # the notifier when an attempt fails. 442 def notify(event) 443 @notifier.send(event, :begins) if @notifier 444 445 begin 446 yield 447 @notifier.send(event, :ends) if @notifier 448 rescue Object 449 @notifier.send(event, :error) if @notifier 450 raise $! 451 end 452 end 453 end 454 455 456 457 # This simple class can be registered with an HttpClient and it'll 458 # get called when different parts of the HTTP request happen. 459 # Each function represents a different event, and the state parameter 460 # is a symbol of consisting of: 461 # 462 # :begins -- event begins. 463 # :error -- event caused exception. 464 # :ends -- event finished (not called if error). 465 # 466 # These calls are made synchronously so you can throttle 467 # the client by sleeping inside them and can track timing 468 # data. 469 class Notifier 470 # Fired right before connecting and right after the connection. 471 def connect(state) 472 end 473 474 # Before and after the full request is actually sent. This may 475 # become "send_header" and "send_body", but right now the whole 476 # blob is shot out in one chunk for efficiency. 477 def send_request(state) 478 end 479 480 # Called whenever a HttpClient.redirect is done and there 481 # are redirects to follow. You can use a notifier to detect 482 # that you're doing to many and throw an abort. 483 def redirect(state) 484 end 485 486 # Before and after the header is finally read. 487 def read_header(state) 488 end 489 490 # Before and after the body is ready. 491 def read_body(state) 492 end 493 494 # Before and after the client closes with the server. 495 def close(state) 496 end 497 end 498 end
Generated using the rcov code coverage analysis tool for Ruby version 0.6.0.