# frozen_string_literal: true # See https://clojure.org/guides/spec (as of 2017-18-02) # Output generated by https://github.com/JoshCheek/seeing_is_believing require "set" require "date" require "speculation" S = Speculation extend S::NamespacedSymbols ## Predicates # Each spec describes a set of allowed values. There are several ways to build # specs and all of them can be composed to build more sophisticated specs. # A Ruby proc that takes a single argument and returns a truthy value is a # valid predicate spec. We can check whether a particular data value conforms # to a spec using conform: S.conform :even?.to_proc, 1000 # => 1000 # The conform function takes something that can be a spec and a data value. # Here we are passing a predicate which is implicitly converted into a spec. # The return value is "conformed". Here, the conformed value is the same as the # original value - we’ll see later where that starts to deviate. If the value # does not conform to the spec, the special value :"Speculation/invalid" is # returned. # If you don’t want to use the conformed value or check for # :"Speculation/invalid", the helper valid? can be used instead to return a # boolean. S.valid? :even?.to_proc, 10 # => true # Note that again valid? implicitly converts the predicate function into a # spec. The spec library allows you to leverage all of the functions you # already have - there is no special dictionary of predicates. Some more # examples: S.valid? :nil?.to_proc, nil # => true S.valid? ->(x) { x.is_a?(String) }, "abc" # => true S.valid? ->(x) { x > 5 }, 10 # => true S.valid? ->(x) { x > 5 }, 0 # => false # Regexps, Classes and Modules can be used as predicates. S.valid? /^\d+$/, "123" # => true S.valid? String, "abc" # => true S.valid? Enumerable, [1, 2, 3] # => true S.valid? Date, Date.new # => true # Sets can also be used as predicates that match one or more literal values: S.valid? Set[:club, :diamond, :heart, :spade], :club # => true S.valid? Set[:club, :diamond, :heart, :spade], 42 # => false S.valid? Set[42], 42 # => true ## Registry # Until now, we’ve been using specs directly. However, spec provides a central # registry for globally declaring reusable specs. The registry associates a # namespaced symbol with a specification. The use of namespaces ensures that # we can define reusable non-conflicting specs across libraries or # applications. # Specs are registered using def. It’s up to you to register the specification # in a namespace that makes sense (typically a namespace you control). S.def ns(:date), Date # => :"Object/date" S.def ns(:suit), Set[:club, :diamond, :heart, :spade] # => :"Object/suit" # A registered spec identifier can be used in place of a spec definition in the # operations we’ve seen so far - conform and valid?. S.valid? ns(:date), Date.new # => true S.conform ns(:suit), :club # => :club # You will see later that registered specs can (and should) be used anywhere we # compose specs. ## Composing predicates # The simplest way to compose specs is with and and or. Let’s create a spec # that combines several predicates into a composite spec with S.and: S.def ns(:big_even), S.and(Integer, :even?.to_proc, ->(x) { x > 1000 }) S.valid? ns(:big_even), :foo # => false S.valid? ns(:big_even), 10 # => false S.valid? ns(:big_even), 100000 # => true # We can also use S.or to specify two alternatives: S.def ns(:name_or_id), S.or(:name => String, :id => Integer) S.valid? ns(:name_or_id), "abc" # => true S.valid? ns(:name_or_id), 100 # => true S.valid? ns(:name_or_id), :foo # => false # This or spec is the first case we’ve seen that involves a choice during # validity checking. Each choice is annotated with a tag (here, between :name # and :id) and those tags give the branches names that can be used to # understand or enrich the data returned from conform and other spec functions. # When an or is conformed, it returns an array with the tag name and conformed # value: S.conform ns(:name_or_id), "abc" # => [:name, "abc"] S.conform ns(:name_or_id), 100 # => [:id, 100] # Many predicates that check an instance’s type do not allow nil as a valid # value (String, ->(x) { x.even? }, /foo/, etc). To include nil as a valid # value, use the provided function nilable to make a spec: S.valid? String, nil # => false S.valid? S.nilable(String), nil # => true ## Explain # explain is another high-level operation in spec that can be used to report # (to STDOUT) why a value does not conform to a spec. Let’s see what explain # says about some non-conforming examples we’ve seen so far. S.explain ns(:suit), 42 # >> val: 42 fails spec: :"Object/suit" predicate: [#, [42]] S.explain ns(:big_even), 5 # >> val: 5 fails spec: :"Object/big_even" predicate: [#, [5]] S.explain ns(:name_or_id), :foo # >> val: :foo fails spec: :"Object/name_or_id" at: [:name] predicate: [String, [:foo]] # >> val: :foo fails spec: :"Object/name_or_id" at: [:id] predicate: [Integer, [:foo]] # Let’s examine the output of the final example more closely. First note that # there are two errors being reported - spec will evaluate all possible # alternatives and report errors on every path. The parts of each error are: # - val - the value in the user’s input that does not match # - spec - the spec that was being evaluated # - at - a path (an array of symbols) indicating the location within the spec # where the error occurred - the tags in the path correspond to any tagged part # in a spec (the alternatives in an or or alt, the parts of a cat, the keys in # a map, etc) # - predicate - the actual predicate that was not satsified by val # - in - the key path through a nested data val to the failing value. In this # example, the top-level value is the one that is failing so this is # essentially an empty path and is omitted. # - For the first reported error we can see that the value :foo did not satisfy # the predicate String at the path :name in the spec ns(:name-or-id). The second # reported error is similar but fails on the :id path instead. The actual value # is a Symbol so neither is a match. # In addition to explain, you can use explain_str to receive the error messages # as a string or explain_data to receive the errors as data. S.explain_data ns(:name_or_id), :foo # => {:"Speculation/problems"=> # [{:path=>[:name], # :val=>:foo, # :via=>[:"Object/name_or_id"], # :in=>[], # :pred=>[String, [:foo]]}, # {:path=>[:id], # :val=>:foo, # :via=>[:"Object/name_or_id"], # :in=>[], # :pred=>[Integer, [:foo]]}]} ## Entity hashes # Ruby programs rely heavily on passing around hashes of data. (That may not be # true...) A common approach in other libraries is to describe each entity # type, combining both the keys it contains and the structure of their values. # Rather than define attribute (key+value) specifications in the scope of the # entity (the hash), specs assign meaning to individual attributes, then # collect them into shahes using set semantics (on the keys). This approach # allows us to start assigning (and sharing) semantics at the attribute level # across our libraries and applications. # This statement isn't quite true for Ruby's Ring equivalent, Rack: # ~~For example, most Ring middleware functions modify the request or response # map with unqualified keys. However, each middleware could instead use # namespaced keys with registered semantics for those keys. The keys could then # be checked for conformance, creating a system with greater opportunities for # collaboration and consistency.~~ # Entity maps in spec are defined with keys: email_regex = /^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$/ S.def ns(:email_type), S.and(String, email_regex) S.def ns(:acctid), Integer S.def ns(:first_name), String S.def ns(:last_name), String S.def ns(:email), ns(:email_type) S.def ns(:person), S.keys(:req => [ns(:first_name), ns(:last_name), ns(:email)], :opt => [ns(:phone)]) # This registers a ns(:person) spec with the required keys ns(:first-name), # ns(:last_name), and ns(:email), with optional key ns(:phone). The hash spec # never specifies the value spec for the attributes, only what attributes are # required or optional. # When conformance is checked on a hash, it does two things - checking that the # required attributes are included, and checking that every registered key has # a conforming value. We’ll see later where optional attributes can be useful. # Also note that ALL attributes are checked via keys, not just those listed in # the :req and :opt keys. Thus a bare S.keys is valid and will check all # attributes of a map without checking which keys are required or optional. S.valid? ns(:person), ns(:first_name) => "Elon", ns(:last_name) => "Musk", ns(:email) => "elon@example.com" # => true # Fails required key check S.explain ns(:person), ns(:first_name) => "Elon" # >> val: {:"Object/first_name"=>"Elon"} fails spec: :"Object/person" predicate: [#, [:"Object/last_name"]] # >> val: {:"Object/first_name"=>"Elon"} fails spec: :"Object/person" predicate: [#, [:"Object/email"]] # Fails attribute conformance S.explain ns(:person), ns(:first_name) => "Elon", ns(:last_name) => "Musk", ns(:email) => "n/a" # >> In: [:"Object/email"] val: "n/a" fails spec: :"Object/email_type" at: [:"Object/email"] predicate: [/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$/, ["n/a"]] # Let’s take a moment to examine the explain error output on that final example: # - in - the path within the data to the failing value (here, a key in the person instance) # - val - the failing value, here "n/a" # - spec - the spec that failed, here :my.domain/email # - at - the path in the spec where the failing value is located # - predicate - the predicate that failed, here (re-matches email-regex %) # Much existing Ruby code does not use hashes with namespaced keys and so keys # can also specify :req_un and :opt_un for required and optional unqualified # keys. These variants specify namespaced keys used to find their # specification, but the map only checks for the unqualified version of the # keys. # Let’s consider a person map that uses unqualified keys but checks conformance # against the namespaced specs we registered earlier: S.def :"unq/person", S.keys(:req_un => [ns(:first_name), ns(:last_name), ns(:email)], :opt_un => [ns(:phone)]) S.conform :"unq/person", :first_name => "Elon", :last_name => "Musk", :email => "elon@example.com" # => {:first_name=>"Elon", :last_name=>"Musk", :email=>"elon@example.com"} S.explain :"unq/person", :first_name => "Elon", :last_name => "Musk", :email => "n/a" # >> In: [:email] val: "n/a" fails spec: :"Object/email_type" at: [:email] predicate: [/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,63}$/, ["n/a"]] S.explain :"unq/person", :first_name => "Elon" # >> val: {:first_name=>"Elon"} fails spec: :"unq/person" predicate: [#, [:"Object/last_name"]] # >> val: {:first_name=>"Elon"} fails spec: :"unq/person" predicate: [#, [:"Object/email"]] # Unqualified keys can also be used to validate record attributes # TODO for objects/structs # Keyword args keys* - don't support # Sometimes it will be convenient to declare entity maps in parts, either # because there are different sources for requirements on an entity map or # because there is a common set of keys and variant-specific parts. The S.merge # spec can be used to combine multiple S.keys specs into a single spec that # combines their requirements. For example consider two keys specs that define # common animal attributes and some dog-specific ones. The dog entity itself # can be described as a merge of those two attribute sets: S.def :"animal/kind", String S.def :"animal/says", String S.def :"animal/common", S.keys(:req => [:"animal/kind", :"animal/says"]) S.def :"dog/tail?", ns(S, :boolean) S.def :"dog/breed", String S.def :"animal/dog", S.merge(:"animal/common", S.keys(:req => [:"dog/tail?", :"dog/breed"])) S.valid? :"animal/dog", :"animal/kind" => "dog", :"animal/says" => "woof", :"dog/tail?" => true, :"dog/breed" => "retriever" # => true ## Multi-spec - don't support ## Collections # A few helpers are provided for other special collection cases - coll_of, # tuple, and hash_of. # For the special case of a homogenous collection of arbitrary size, you can # use coll_of to specify a collection of elements satisfying a predicate. S.conform S.coll_of(Symbol), [:a, :b, :c] # => [:a, :b, :c] S.conform S.coll_of(Numeric), Set[5, 10, 2] # => # # Additionally, coll-of can be passed a number of keyword arg options: # :kind - a predicate or spec that the incoming collection must satisfy, such as `Array` # :count - specifies exact expected count # :min_count, :max_count - checks that collection has `count.between?(min_count, max_count)` # :distinct - checks that all elements are distinct # :into - one of [], {}, or Set[] for output conformed value. If :into is not specified, the input collection type will be used. # Following is an example utilizing some of these options to spec an array # containing three distinct numbers conformed as a set and some of the errors # for different kinds of invalid values: S.def ns(:vnum3), S.coll_of(Numeric, :kind => Array, :count => 3, :distinct => true, :into => Set[]) S.conform ns(:vnum3), [1, 2, 3] # => # S.explain ns(:vnum3), Set[1, 2, 3] # not an array # >> val: # fails spec: :"Object/vnum3" predicate: [Array, [#]] S.explain ns(:vnum3), [1, 1, 1] # not distinct # >> val: [1, 1, 1] fails spec: :"Object/vnum3" predicate: [#, [[1, 1, 1]]] S.explain ns(:vnum3), [1, 2, :a] # not a number # >> In: [2] val: :a fails spec: :"Object/vnum3" predicate: [Numeric, [:a]] # NOTE: Both coll-of and map-of will conform all of their elements, which may # make them unsuitable for large collections. In that case, consider every or # for maps every-kv. # While coll-of is good for homogenous collections of any size, another case is # a fixed-size positional collection with fields of known type at different # positions. For that we have tuple. S.def ns(:point), S.tuple(Float, Float, Float) S.conform ns(:point), [1.5, 2.5, -0.5] # => [1.5, 2.5, -0.5] # Note that in this case of a "point" structure with x/y/z values we actually # had a choice of three possible specs: # - Regular expression - S.cat :x => Float, :y => Float, :z => Float # - Allows for matching nested structure (not needed here) # - Conforms to hash with named keys based on the cat tags # - Collection - S.coll_of Float # - Designed for arbitrary size homogenous collections # - Conforms to an array of the values # - Tuple - S.tuple Float, Float, Float # - Designed for fixed size with known positional "fields" # - Conforms to an array of the values # In this example, coll_of will match other (invalid) values as well (like # [1.0] or [1.0 2.0 3.0 4.0]), so it is not a suitable choice - we want fixed # fields. The choice between a regular expression and tuple here is to some # degree a matter of taste, possibly informed by whether you expect either the # tagged return values or error output to be better with one or the other. # In addition to the support for information hashes via keys, spec also # provides hash_of for maps with homogenous key and value predicates. S.def ns(:scores), S.hash_of(String, Integer) S.conform ns(:scores), "Sally" => 1000, "Joe" => 300 # => {"Sally"=>1000, "Joe"=>300} # By default hash_of will validate but not conform keys because conformed keys # might create key duplicates that would cause entries in the map to be # overridden. If conformed keys are desired, pass the option # `:conform_keys => # true`. # You can also use the various count-related options on hash_of that you have # with coll_of. ## Sequences # Sometimes sequential data is used to encode additional structure. spec # provides the standard regular expression operators to describe the structure # of a sequential data value: # - cat - concatenation of predicates/patterns # - alt - choice among alternative predicates/patterns # - zero_or_more - 0 or more of a predicate/pattern # - one_or_more - 1 or more of a predicate/pattern # - zero_or_one - 0 or 1 of a predicate/pattern # Like or, both cat and alt tag their "parts" - these tags are then used in the # conformed value to identify what was matched, to report errors, and more. # Consider an ingredient represented by an array containing a quantity (number) # and a unit (symbol). The spec for this data uses cat to specify the right # components in the right order. Like predicates, regex operators are # implicitly converted to specs when passed to functions like conform, valid?, # etc. S.def ns(:ingredient), S.cat(:quantity => Numeric, :unit => Symbol) S.conform ns(:ingredient), [2, :teaspoon] # => {:quantity=>2, :unit=>:teaspoon} # The data is conformed as a hash with the tags as keys. We can use explain to # examine non-conforming data. # pass string for unit instead of keyword S.explain ns(:ingredient), [11, "peaches"] # >> In: [1] val: "peaches" fails spec: :"Object/ingredient" at: [:unit] predicate: [Symbol, ["peaches"]] # leave out the unit S.explain ns(:ingredient), [2] # >> val: [] fails spec: :"Object/ingredient" at: [:unit] predicate: [Symbol, []], "Insufficient input" # Let’s now see the various occurence operators zero_or_more, one_or_more, and zero_or_one: S.def ns(:seq_of_symbols), S.zero_or_more(Symbol) S.conform ns(:seq_of_symbols), [:a, :b, :c] # => [:a, :b, :c] S.explain ns(:seq_of_symbols), [10, 20] # >> In: [0] val: 10 fails spec: :"Object/seq_of_symbols" predicate: [Symbol, [10]] S.def ns(:odds_then_maybe_even), S.cat(:odds => S.one_or_more(:odd?.to_proc), :even => S.zero_or_one(:even?.to_proc)) S.conform ns(:odds_then_maybe_even), [1, 3, 5, 100] # => {:odds=>[1, 3, 5], :even=>100} S.conform ns(:odds_then_maybe_even), [1] # => {:odds=>[1]} S.explain ns(:odds_then_maybe_even), [100] # >> In: [0] val: 100 fails spec: :"Object/odds_then_maybe_even" at: [:odds] predicate: [#, [100]] # opts are alternating symbols and booleans S.def ns(:opts), S.zero_or_more(S.cat(:opt => Symbol, :val => ns(S, :boolean))) S.conform ns(:opts), [:silent?, false, :verbose, true] # => [{:opt=>:silent?, :val=>false}, {:opt=>:verbose, :val=>true}] # Finally, we can use alt to specify alternatives within the sequential data. # Like cat, alt requires you to tag each alternative but the conformed data is # a vector of tag and value. S.def ns(:config), S.zero_or_more(S.cat(:prop => String, :val => S.alt(:s => String, :b => ns(S, :boolean)))) S.conform ns(:config), ["-server", "foo", "-verbose", true, "-user", "joe"] # => [{:prop=>"-server", :val=>[:s, "foo"]}, # {:prop=>"-verbose", :val=>[:b, true]}, # {:prop=>"-user", :val=>[:s, "joe"]}] # TODO: If you need a description of a specification, use describe to retrieve one. # Spec also defines one additional regex operator, `constrained`, which takes a # regex operator and constrains it with one or more additional predicates. This # can be used to create regular expressions with additional constraints that # would otherwise require custom predicates. For example, consider wanting to # match only sequences with an even number of strings: S.def ns(:even_strings), S.constrained(S.zero_or_more(String), ->(coll) { coll.count.even? }) S.valid? ns(:even_strings), ["a"] # => false S.valid? ns(:even_strings), ["a", "b"] # => true S.valid? ns(:even_strings), ["a", "b", "c"] # => false S.valid? ns(:even_strings), ["a", "b", "c", "d"] # => true # When regex ops are combined, they describe a single sequence. If you need to # spec a nested sequential collection, you must use an explicit call to spec to # start a new nested regex context. For example to describe a sequence like # [:names, ["a", "b"], :nums, [1 2 3]], you need nested regular expressions to # describe the inner sequential data: S.def ns(:nested), S.cat(:names_sym => Set[:names], :names => S.spec(S.zero_or_more(String)), :nums_sym => Set[:nums], :nums => S.spec(S.zero_or_more(Numeric))) S.conform ns(:nested), [:names, ["a", "b"], :nums, [1, 2, 3]] # => {:names_sym=>:names, # :names=>["a", "b"], # :nums_sym=>:nums, # :nums=>[1, 2, 3]} # If the specs were removed this spec would instead match a sequence like # [:names, "a", "b", :nums, 1, 2, 3]. S.def ns(:unnested), S.cat(:names_sym => Set[:names], :names => S.zero_or_more(String), :nums_sym => Set[:nums], :nums => S.zero_or_more(Numeric)) S.conform ns(:unnested), [:names, "a", "b", :nums, 1, 2, 3] # => {:names_sym=>:names, # :names=>["a", "b"], # :nums_sym=>:nums, # :nums=>[1, 2, 3]} ## Using spec for validation # Now is a good time to step back and think about how spec can be used for # runtime data validation. # One way to use spec is to explicitly call valid? to verify input data passed # to a function. ~~You can, for example, use the existing pre- and post-condition # support built into defn:~~ def self.person_name(person) raise "invalid" unless S.valid? ns(:person), person name = "#{person[ns(:first_name)]} #{person[ns(:last_name)]}" raise "invalid" unless S.valid? String, name name end person_name 43 rescue $! # => # person_name ns(:first_name) => "Elon", ns(:last_name) => "Musk", ns(:email) => "elon@example.com" # => "Elon Musk" # When the function is invoked with something that isn’t valid ns(:person) data, # the pre-condition fails. Similarly, if there was a bug in our code and the # output was not a string, the post-condition would fail. # Another option is to use S.assert within your code to assert that a value # satisfies a spec. On success the value is returned and on failure an # assertion error is thrown. By default assertion checking is off - this can be # changed by setting S.check_asserts or having the environment variable # "SPECULATION_CHECK_ASSERTS=true". def self.person_name(person) p = S.assert ns(:person), person "#{p[ns(:first_name)]} #{p[ns(:last_name)]}" end S.check_asserts = true person_name 100 rescue $! # => #, [100]] # Speculation/failure :assertion_failed # {:"Speculation/problems"=> # [{:path=>[], # :pred=>[#, [100]], # :val=>100, # :via=>[], # :in=>[]}], # :"Speculation/failure"=>:assertion_failed} # > # A deeper level of integration is to call conform and use the return value to # destructure the input. This will be particularly useful for complex inputs # with alternate options. # Here we conform using the config specification defined above: def self.set_config(prop, val) # dummy fn puts "set #{prop} #{val}" end def self.configure(input) parsed = S.conform(ns(:config), input) if parsed == ns(S, :invalid) raise "Invalid input\n#{S.explain_str(ns(:config), input)}" else parsed.each do |config| prop, val = config.values_at(:prop, :val) _type, val = val set_config(prop[1..-1], val) end end end configure ["-server", "foo", "-verbose", true, "-user", "joe"] # set server foo # set verbose true # set user joe # Here configure calls conform to destructure the config input. The result is # either the special :"Speculation/invalid" value or a destructured form of the # result: [{ :prop => "-server", :val => [:s, "foo"] }, { :prop => "-verbose", :val => [:b, true] }, { :prop => "-user", :val => [:s, "joe"] }] # In the success case, the parsed input is transformed into the desired shape # for further processing. In the error case, we call explain_str to generate # an error message. The explain string contains information about what # expression failed to conform, the path to that expression in the # specification, and the predicate it was attempting to match. ## Spec’ing methods # The pre- and post-condition example in the previous section hinted at an # interesting question - how do we define the input and output specifications # for a method. # Spec has explicit support for this using fdef, which defines specifications # for a function - the arguments and/or the return value spec, and optionally a # function that can specify a relationship between args and return. # Let’s consider a ranged-rand function that produces a random number in a # range: def self.ranged_rand(from, to) rand(from...to) end # We can then provide a specification for that function: S.fdef(method(:ranged_rand), :args => S.and(S.cat(:start => Integer, :end => Integer), ->(args) { args[:start] < args[:end] }), :ret => Integer, :fn => S.and(->(fn) { fn[:ret] >= fn[:args][:start] }, ->(fn) { fn[:ret] < fn[:args][:end] })) # This function spec demonstrates a number of features. First the :args is a # compound spec that describes the function arguments. This spec is invoked # with the args in an array, as if they were invoked like `method.call(*args)` # Because the args are sequential and the args are positional fields, they are # almost always described using a regex op, like cat, alt, or zero_or_more. # The second :args predicate takes as input the conformed result of the first # predicate and verifies that start < end. The :ret spec indicates the return # is also an integer. Finally, the :fn spec checks that the return value is >= # start and < end. # We’ll see later how we can use a function spec for development and testing. ## Higher order functions # Higher order functions are common in ~Clojure~ Ruby and spec provides fspec # to support spec’ing them. # For example, consider the adder function: def self.adder(x) ->(y) { x + y } end # adder returns a proc that adds x. We can declare a function spec for adder # using fspec for the return value: S.fdef method(:adder), :args => S.cat(:x => Numeric), :ret => S.fspec(:args => S.cat(:y => Numeric), :ret => Numeric), :fn => ->(fn) { fn[:args][:x] == fn[:ret].call(0) } # The :ret spec uses fspec to declare that the returning function takes and # returns a number. Even more interesting, the :fn spec can state a general # property that relates the :args (where we know x) and the result we get from # invoking the function returned from adder, namely that adding 0 to it should # return x. ## Macros - noop ## A game of cards # Here’s a bigger set of specs to model a game of cards: suit = Set[:club, :diamond, :heart, :spade] rank = Set[:jack, :queen, :king, :ace].merge(2..10) deck = rank.to_a.product(suit.to_a) S.def ns(:card), S.tuple(rank, suit) S.def ns(:hand), S.zero_or_more(ns(:card)) S.def ns(:name), String S.def ns(:score), Integer S.def ns(:player), S.keys(:req => [ns(:name), ns(:score), ns(:hand)]) S.def ns(:players), S.zero_or_more(ns(:player)) S.def ns(:deck), S.zero_or_more(ns(:card)) S.def ns(:game), S.keys(:req => [ns(:players), ns(:deck)]) # We can validate a piece of this data against the schema: kenny = { ns(:name) => "Kenny Rogers", ns(:score) => 100, ns(:hand) => [] } S.valid? ns(:player), kenny # => true # Or look at the errors we’ll get from some bad data: S.explain ns(:game), ns(:deck) => deck, ns(:players) => [{ ns(:name) => "Kenny Rogers", ns(:score) => 100, ns(:hand) => [[2, :banana]] }] # >> In: [:"Object/players", 0, :"Object/hand", 0, 1] val: :banana fails spec: :"Object/card" at: [:"Object/players", :"Object/hand", 1] predicate: [#, [:banana]] # The error indicates the key path in the data structure down to the invalid # value, the non-matching value, the spec part it’s trying to match, the path # in that spec, and the predicate that failed. # If we have a function `deal` that doles out some cards to the players we can # spec that function to verify the arg and return value are both suitable # data values. We can also specify a :fn spec to verify that the count of # cards in the game before the deal equals the count of cards after the deal. def self.total_cards(game) game, players = game.values_at(ns(:game), ns(:players)) players.map { |player| player[ns(:hand)].count }.reduce(deck.count, &:+) end def self.deal(game) # ... end S.fdef method(:deal), :args => S.cat(:game => ns(:game)), :ret => ns(:game), :fn => ->(fn) { total_cards(fn[:args][:game]) == total_cards(fn[:ret]) } ## Generators # A key design constraint of spec is that all specs are also designed to act as # generators of sample data that conforms to the spec (a critical requirement # for property-based testing). ## ~~Project Setup~~ # Nothing to do, Rantly is included by default. May look at removing the hard # dependency in the future. # In your code you also need to require the speculation/gen lib. require "speculation/gen" Gen = S::Gen ## Sampling Generators # The gen function can be used to obtain the generator for any spec. # Once you have obtained a generator with gen, there are several ways to use # it. You can generate a single sample value with generate or a series of # samples with sample. Let’s see some basic examples: Gen.generate S.gen(Integer) # => 372495152381320358 Gen.generate S.gen(NilClass) # => nil Gen.sample S.gen(String), 5 # => ["RhzOLQjmSjhWavH", "y", "", "O", "peoPwXHRBBAPjDxzEZQh"] Gen.sample S.gen(Set[:club, :diamond, :heart, :spade]), 5 # => [:heart, :spade, :club, :spade, :spade] Gen.sample S.gen(S.cat(:k => Symbol, :ns => S.one_or_more(Numeric))), 4 # => [[:csWKkimBORwN, # -298753312314306397, # -2303961522202434118, # 1679934373136969303, # -262631322747429978, # 1.7157706401801108e+308, # 1758361237993287532, # 712842522394861335, # -883871273503318653, # 1283229873044628318, # 1.5298057192258154e+308, # 1.7789073686150528e+308, # -2281793086040303873, # 120746116914138063, # -404134654833569820, # -54740933266507251, # 5.01892001701602e+307], # [:RetYrsJr, # 1.3391749738917395e+308, # 1.0920197216545966e+307, # 1.384947546752308e+307, # 1.3364975035426882e+308, # 327082393035103718, # 1.0209866964240673e+308, # 512415813150328683], # [:UdDv, # 3.0578102207508006e+307, # 1.1626478137534508e+308, # 1.7939796459941183e+308, # 1494374259430455477, # 1.342849042383955e+308, # -281429214092326237, # -552507314062007344, # 4.1453903880025765e+307, # -973157747452936365, # 1.1388886925899274e+308, # 2056792483501668313, # 999682663796411736, # 7.395274944717998e+306, # -1514851160913660499, # -2167762478595098510, # 824382210168550458, # 1614922845514653160], # [:s, # -234772724560973590, # 1.0042104238108253e+308, # 1.3942217537031457e+307, # -1553774642616973743, # -360282579504585923]] # What about generating a random player in our card game? Gen.generate S.gen(ns(:player)) # => {:"Object/name"=>"qrmY", # :"Object/score"=>-188402685781919929, # :"Object/hand"=> # [[10, :heart], # [:king, :heart], # [2, :spade], # [7, :heart], # [9, :club], # [7, :club], # [10, :diamond], # [:jack, :spade], # [2, :diamond], # [3, :diamond], # [:king, :spade], # [5, :spade], # [10, :heart], # [:king, :heart], # [:jack, :spade], # [:king, :spade], # [:queen, :club], # [6, :diamond], # [5, :club], # [6, :club]]} # What about generating a whole game? Gen.generate S.gen(ns(:game)) # it works! but the output is really long, so not including it here # So we can now start with a spec, extract a generator, and generate some data. # All generated data will conform to the spec we used as a generator. For specs # that have a conformed value different than the original value (anything using # S.or, S.cat, S.alt, etc) it can be useful to see a set of generated samples # plus the result of conforming that sample data. ## Exercise # For this we have `exercise`, which returns pairs of generated and conformed # values for a spec. exercise by default produces 10 samples (like sample) but # you can pass both functions a number indicating the number of samples to # produce. S.exercise S.cat(:k => Symbol, :ns => S.one_or_more(Numeric)), :n => 5 # => [[[:AXgNzoRmshVeKju, # -817925373115395462, # 1.5359311568381347e+308, # 1.1061449248034022e+308, # -235267876425474208, # 3.955857252356689e+307, # -889011872905836841, # 9.082764829559406e+307, # 3.8449893386631863e+307, # 1399473921337276004, # 1.1035252898212735e+308], # {:k=>:AXgNzoRmshVeKju, # :ns=> # [-817925373115395462, # 1.5359311568381347e+308, # 1.1061449248034022e+308, # -235267876425474208, # 3.955857252356689e+307, # -889011872905836841, # 9.082764829559406e+307, # 3.8449893386631863e+307, # 1399473921337276004, # 1.1035252898212735e+308]}], # [[:Nsndjayf, # 1.9984725870793707e+307, # 1.5323527859487139e+308, # 1.0526758425396865e+308, # 2187215078751341740, # 2000267805737910757, # 672724827310048814, # 7.353660057508847e+307, # -499603991431322628, # 823374880053618568, # 988019501395130231, # -85062962445868544, # 1208854825028261939, # -239585966232519771], # {:k=>:Nsndjayf, # :ns=> # [1.9984725870793707e+307, # 1.5323527859487139e+308, # 1.0526758425396865e+308, # 2187215078751341740, # 2000267805737910757, # 672724827310048814, # 7.353660057508847e+307, # -499603991431322628, # 823374880053618568, # 988019501395130231, # -85062962445868544, # 1208854825028261939, # -239585966232519771]}], # [[:kKknKqGtQjl, 1.781549997030396e+305, -2255917728752340059], # {:k=>:kKknKqGtQjl, # :ns=>[1.781549997030396e+305, -2255917728752340059]}], # [[:OknzgVGj, # -2263138309988902357, # 6.780757328421502e+307, # 1159675302983770930, # 8.619504625294373e+307, # -102111175606505256, # 3.1369602174703924e+307, # 714218663950371918, # 1072428045010760820, # 1.7120457957881442e+308, # 1.7220639025345156e+308, # 7.318059339504824e+307, # -627281432214439965, # 1285330282675190977, # 5.624663033422957e+307], # {:k=>:OknzgVGj, # :ns=> # [-2263138309988902357, # 6.780757328421502e+307, # 1159675302983770930, # 8.619504625294373e+307, # -102111175606505256, # 3.1369602174703924e+307, # 714218663950371918, # 1072428045010760820, # 1.7120457957881442e+308, # 1.7220639025345156e+308, # 7.318059339504824e+307, # -627281432214439965, # 1285330282675190977, # 5.624663033422957e+307]}], # [[:mifpKjpS, 3.8475669790437504e+307, 1.5541847940699583e+307], # {:k=>:mifpKjpS, # :ns=>[3.8475669790437504e+307, 1.5541847940699583e+307]}]] S.exercise S.or(:k => Symbol, :s => String, :n => Numeric), :n => 5 # => [[-1310754584514288, [:n, -1310754584514288]], # [872148706486332083, [:n, 872148706486332083]], # [:rHCoqRLZYhzSgOu, [:k, :rHCoqRLZYhzSgOu]], # [-395552003092497804, [:n, -395552003092497804]], # [:WoaPnjB, [:k, :WoaPnjB]]] # For spec’ed functions we also have exercise_fn, which generates sample args, # invokes the spec’ed function and returns the args and the return value. S.exercise_fn(method(:ranged_rand)) # => [[[-2128611012334186431, -1417738444057945122], -1635106169064592441], # [[1514518280943101595, 1786254628919354373], 1739796291756227578], # [[-46749061680797208, 822766248044755470], -7474228458851983], # [[-649513218842008808, 1875894039691321060], -390581384114488816], # [[858361555883341214, 1741658980258358628], 1374077212657449917], # [[-1258388171360603963, -985723099401376708], -1123010455669592843], # [[-1035489322616947034, 1688366643195138662], 441214083022620176], # [[-2229284211372056198, -893085296484913242], -1161469637076511831], # [[819684425123939548, 1044514159372510410], 971678102106589235], # [[366502776249932529, 1318835861470496704], 377553467194155955]] ## Using S.and Generators # All of the generators we’ve seen worked fine but there are a number of cases # where they will need some additional help. One common case is when the # predicate implicitly presumes values of a particular type but the spec does # not specify them: Gen.generate S.gen(:even?.to_proc) rescue $! # => #) {:"Speculation/failure"=>:no_gen, :"Speculation/path"=>[]}\n> # In this case spec was not able to find a generator for the even? predicate. # Most of the primitive generators in spec are mapped to the common type # predicates (classes, modules, built-in specs). # However, spec is designed to support this case via `and` - the first # predicate will determine the generator and subsequent branches will act as # filters by applying the predicate to the produced values. # If we modify our predicate to use an `and` and a predicate with a mapped # generator, the even? can be used as a filter for generated values instead: Gen.generate S.gen(S.and(Integer, :even?.to_proc)) # => 1875527059787064980 # We can use many predicates to further refine the generated values. For # example, say we only wanted to generate numbers that were positive multiples # of 3: def self.divisible_by(n) ->(x) { (x % n).zero? } end Gen.sample S.gen(S.and(Integer, :positive?.to_proc, divisible_by(3))) # => [1003257946641857673, # 1302633092686504620, # 1067379217208623728, # 882135641374726149, # 1933864978000820676, # 235089151558168077, # 470438340672134322, # 2268668240213030931, # 1061519505888350829, # 1868667505095337938] # However, it is possible to go too far with refinement and make something that # fails to produce any values. The Rantly `guard` that implements the # refinement will throw an error if the refinement predicate cannot be resolved # within a relatively small number of attempts. For example, consider trying to # generate strings that happen to contain the world "hello": # hello, are you the one I'm looking for? Gen.sample S.gen(S.and(String, ->(s) { s.include?("hello") })) rescue $! # => # # Given enough time (maybe a lot of time), the generator probably would come up # with a string like this, but the underlying `guard` will make only 100 # attempts to generate a value that passes the filter. This is a case where you # will need to step in and provide a custom generator. ## Custom Generators # Building your own generator gives you the freedom to be either narrower # and/or be more explicit about what values you want to generate. Alternately, # custom generators can be used in cases where conformant values can be # generated more efficiently than using a base predicate plus filtering. Spec # does not trust custom generators and any values they produce will also be # checked by their associated spec to guarantee they pass conformance. # There are three ways to build up custom generators - in decreasing order of # preference: # - Let spec create a generator based on a predicate/spec # - Create your own generator using Rantly directly # First consider a spec with a predicate to specify symbols from a particular # namespace: S.def ns(:syms), S.and(Symbol, ->(s) { S::NamespacedSymbols.namespace(s) == "my.domain" }) S.valid? ns(:syms), :"my.domain/name" # => true Gen.sample S.gen(ns(:syms)) rescue $! # => # # The simplest way to start generating values for this spec is to have spec # create a generator from a fixed set of options. A set is a valid predicate # spec so we can create one and ask for it’s generator: sym_gen = S.gen(Set[:"my.domain/name", :"my.domain/occupation", :"my.domain/id"]) Gen.sample sym_gen, 5 # => [:"my.domain/id", # :"my.domain/name", # :"my.domain/occupation", # :"my.domain/name", # :"my.domain/id"] # To redefine our spec using this custom generator, use with_gen which takes a # spec and a replacement generator as a block: gen = S.gen(Set[:"my.domain/name", :"my.domain/occupation", :"my.domain/id"]) S.def(ns(:syms), S.with_gen(S.and(Symbol, ->(s) { S::NamespacedSymbols.namespace(s) == "my.domain" }), gen)) S.valid? ns(:syms), :"my.domain/name" Gen.sample S.gen(ns(:syms)), 5 # => [:"my.domain/name", # :"my.domain/id", # :"my.domain/occupation", # :"my.domain/occupation", # :"my.domain/id"] # TODO: make gens no-arg functions??? # Note that with_gen (and other places that take a custom generator) take a # one-arg function that returns the generator, allowing it to be lazily # realized. # One downside to this approach is we are missing what property testing is # really good at: automatically generating data across a wide search space to # find unexpected problems. # Rantly has a small library of generators that can be utilized. # In this case we want our keyword to have open names but fixed namespaces. # There are many ways to accomplish this but one of the simplest is to use fmap # to build up a keyword based on generated strings: sym_gen_2 = ->(rantly) do size = rantly.range(1, 10) string = rantly.sized(size) { rantly.string(:alpha) } :"my.domain/#{string}" end Gen.sample sym_gen_2, 5 # => [:"my.domain/hLZnEpj", :"my.domain/kvy", :"my.domain/VqWbqD", :"my.domain/imq", :"my.domain/eHeZleWzj"] # Returning to our "hello" example, we now have the tools to make that # generator: S.def ns(:hello), S.with_gen(->(s) { s.include?("hello") }, ->(rantly) { s1 = rantly.sized(rantly.range(0, 10)) { rantly.string(:alpha) } s2 = rantly.sized(rantly.range(0, 10)) { rantly.string(:alpha) } "#{s1}hello#{s2}" }) Gen.sample S.gen(ns(:hello)) # => ["XRLhtLshelloaY", # "tXxZQHhZOhelloJ", # "ExhellozzlPYz", # "MaiierIhelloel", # "WKZBJprQkhelloGdGToCbI", # "RDFCZhello", # "PXPsYwJLhellosYoYngd", # "SuhelloJ", # "wWhelloodQFFvdW", # "pNhello"] # Here we generate a tuple of a random prefix and random suffix strings, then # insert "hello" bewteen them. ## Range Specs and Generators # There are several cases where it’s useful to spec (and generate) values in a # range and spec provides helpers for these cases. # For example, in the case of a range of integer values (for example, a bowling # roll), use int_in to spec a range: S.def ns(:roll), S.int_in(0..10) Gen.sample S.gen(ns(:roll)) # => [10, 0, 8, 2, 6, 10, 1, 10, 10, 0] # spec also includes date_in for a range of dates: S.def ns(:the_aughts), S.date_in(Date.new(2000, 1, 1)..Date.new(2010)) Gen.sample S.gen(ns(:the_aughts)), 5 # => [#, # #, # #, # #, # #] # spec also includes time_in for a range of times: S.def ns(:the_aughts), S.time_in(Time.new(2000)..Time.new(2010)) Gen.sample S.gen(ns(:the_aughts)), 5 # => [2000-03-26 07:01:27 -0800, # 2009-03-28 12:00:18 -0700, # 2003-09-20 15:33:42 -0700, # 2007-06-26 20:07:13 -0700, # 2003-11-25 16:59:38 -0800] # Finally, float_in has support for double ranges and special options for # checking special float values like NaN (not a number), Infinity, and # -Infinity. S.def ns(:floats), S.float_in(:min => -100.0, :max => 100.0, :nan => false, :infinite => false) S.valid? ns(:floats), 2.9 # => true S.valid? ns(:floats), Float::INFINITY # => false Gen.sample S.gen(ns(:floats)), 5 # => [65.53711851243327, 67.31921045318401, -71.92560111608772, 81.66336359400515, -30.4921955594738] ## Instrumentation and Testing # spec provides a set of development and testing functionality in the # Speculation::Test namespace, which we can include with: require "speculation/test" STest = Speculation::Test ## Instrumentation # Instrumentation validates that the :args spec is being invoked on # instrumented functions and thus provides validation for external uses of a # function. Let’s turn on instrumentation for our previously spec’ed # ranged-rand function: STest.instrument method(:ranged_rand) # If the function is invoked with args that do not conform with the :args spec # you will see an error like this: ranged_rand 8, 5 rescue $! # => #8, :end=>5} fails at: [:args] predicate: [#, [{:start=>8, :end=>5}]] # Speculation/args [8, 5] # Speculation/failure :instrument # Speculation::Test/caller "/var/folders/4l/j2mycv0j4rx7z47sp01r93vc3kfxzs/T/seeing_is_believing_temp_dir20170304-91770-1jtmc1y/program.rb:901:in `
'" # {:"Speculation/problems"=> # [{:path=>[:args], # :val=>{:start=>8, :end=>5}, # :via=>[], # :in=>[], # :pred=> # [#, # [{:start=>8, :end=>5}]]}], # :"Speculation/args"=>[8, 5], # :"Speculation/failure"=>:instrument, # :"Speculation::Test/caller"=> # "/var/folders/4l/j2mycv0j4rx7z47sp01r93vc3kfxzs/T/seeing_is_believing_temp_dir20170304-91770-1jtmc1y/program.rb:901:in `
'"} # > # The error fails in the second args predicate that checks `start < end`. Note # that the :ret and :fn specs are not checked with instrumentation as # validating the implementation should occur at testing time. # Instrumentation can be turned off using the complementary function # unstrument. Instrumentation is likely to be useful at both development time # and during testing to discover errors in calling code. It is not recommended # to use instrumentation in production due to the overhead involved with # checking args specs. ## Testing # We mentioned earlier that ~~clojure.spec.test~~ Speculation provides tools # for automatically testing functions. When functions have specs, we can use # check, to automatically generate tests that check the function using the # specs. # check will generate arguments based on the :args spec for a function, invoke # the function, and check that the :ret and :fn specs were satisfied. STest.check method(:ranged_rand) # => [{:spec=>Speculation::FSpec(main.ranged_rand), # :"Speculation::Test/ret"=>{:num_tests=>1000, :result=>true}, # :method=>#}] # check also takes a number of options ~~that can be passed to test.check to # influence the test run~~, as well as the option to override generators for # parts of the spec, by either name or path. # Imagine instead that we made an error in the ranged-rand code and swapped # start and end: def self.ranged_rand(from, to) ## broken! (from + rand(to)).to_i end # This broken function will still create random integers, just not in the # expected range. Our :fn spec will detect the problem when checking the var: STest.abbrev_result STest.check(method(:ranged_rand)).first # >> {:spec=>"Speculation::FSpec(main.ranged_rand)", # >> :method=>#, # >> :failure=> # >> {:"Speculation/problems"=> # >> [{:path=>[:fn], # >> :val=>{:args=>{:start=>-1, :end=>0}, :block=>nil, :ret=>0}, # >> :via=>[], # >> :in=>[], # >> :pred=> # >> [#, # >> [{:args=>{:start=>-1, :end=>0}, :block=>nil, :ret=>0}]]}], # >> :"Speculation::Test/args"=>[-1, 0], # >> :"Speculation::Test/val"=> # >> {:args=>{:start=>-1, :end=>0}, :block=>nil, :ret=>0}, # >> :"Speculation/failure"=>:check_failed}} # check has reported an error in the :fn spec. We can see the arguments passed # were -1 and 0 and the return value was -0, which is out of the expected # range (it should be less that the `end` argument). # To test all of the spec’ed functions in a module/class (or multiple # module/classes), use enumerate-methods to generate the set of symbols # naming vars in the namespace: STest.summarize_results STest.check(STest.enumerate_methods(self)) # => {:total=>3, :check_failed=>2, :check_passed=>1} # And you can check all of the spec’ed functions by calling STest.check without any arguments. ## Combining check and instrument # While both `instrument` (for enabling :args checking) and `check` (for generating # tests of a function) are useful tools, they can be combined to provide even # deeper levels of test coverage. # `instrument` takes a number of options for changing the behavior of # instrumented functions, including support for swapping in alternate # (narrower) specs, stubbing functions (by using the :ret spec to generate # results), or replacing functions with an alternate implementation. # Consider the case where we have a low-level function that invokes a remote # service and a higher-level function that calls it. # code under test def self.invoke_service(service, request) # invokes remote service end def self.run_query(service, query) response = invoke_service(service, ns(:query) => query) result, error = response.values_at(ns(:result), ns(:error)) result || error end # We can spec these functions using the following specs: S.def ns(:query), String S.def ns(:request), S.keys(:req => [ns(:query)]) S.def ns(:result), S.coll_of(String, :gen_max => 3) S.def ns(:error), Integer S.def ns(:response), S.or(:ok => S.keys(:req => [ns(:result)]), :err => S.keys(:req => [ns(:error)])) S.fdef method(:invoke_service), :args => S.cat(:service => ns(S, :any), :request => ns(:request)), :ret => ns(:response) S.fdef method(:run_query), :args => S.cat(:service => ns(S, :any), :query => String), :ret => S.or(:ok => ns(:result), :err => ns(:error)) # And then we want to test the behavior of run_query while stubbing out # invoke_service with instrument so that the remote service is not invoked: STest.instrument method(:invoke_service), :stub => [method(:invoke_service)] invoke_service nil, ns(:query) => "test" # => {:"Object/result"=>["LtqDYvzOfVzCHWN", "ZASNKhtkwBAXyTF"]} invoke_service nil, ns(:query) => "test" # => {:"Object/result"=>["yvccRd", "xREXEgc"]} STest.summarize_results STest.check(method(:run_query)) # => {:total=>1, :check_passed=>1} # The first call here instruments and stubs invoke_service. The second and # third calls demonstrate that calls to invoke_service now return generated # results (rather than hitting a service). Finally, we can use check on the # higher level function to test that it behaves properly based on the generated # stub results returned from invoke_service. ## Wrapping Up # In this guide we have covered most of the features for designing and using # specs and generators. We expect to add some more advanced generator # techniques and help on testing in a future update. # Original author of clojure.spec guide: Alex Miller