r$Lc@sdZdZddkZdagZdefdYZdefdYZdefd YZ d efd YZ dd Z d e fdYZ dZdS(s:src/engine/SCons/Memoize.py 5023 2010/06/14 22:05:46 sconssMemoizer A metaclass implementation to count hits and misses of the computed values that various methods cache in memory. Use of this modules assumes that wrapped methods be coded to cache their values in a consistent way. Here is an example of wrapping a method that returns a computed value, with no input parameters: memoizer_counters = [] # Memoization memoizer_counters.append(SCons.Memoize.CountValue('foo')) # Memoization def foo(self): try: # Memoization return self._memo['foo'] # Memoization except KeyError: # Memoization pass # Memoization result = self.compute_foo_value() self._memo['foo'] = result # Memoization return result Here is an example of wrapping a method that will return different values based on one or more input arguments: def _bar_key(self, argument): # Memoization return argument # Memoization memoizer_counters.append(SCons.Memoize.CountDict('bar', _bar_key)) # Memoization def bar(self, argument): memo_key = argument # Memoization try: # Memoization memo_dict = self._memo['bar'] # Memoization except KeyError: # Memoization memo_dict = {} # Memoization self._memo['dict'] = memo_dict # Memoization else: # Memoization try: # Memoization return memo_dict[memo_key] # Memoization except KeyError: # Memoization pass # Memoization result = self.compute_bar_value(argument) memo_dict[memo_key] = result # Memoization return result At one point we avoided replicating this sort of logic in all the methods by putting it right into this module, but we've moved away from that at present (see the "Historical Note," below.). Deciding what to cache is tricky, because different configurations can have radically different performance tradeoffs, and because the tradeoffs involved are often so non-obvious. Consequently, deciding whether or not to cache a given method will likely be more of an art than a science, but should still be based on available data from this module. Here are some VERY GENERAL guidelines about deciding whether or not to cache return values from a method that's being called a lot: -- The first question to ask is, "Can we change the calling code so this method isn't called so often?" Sometimes this can be done by changing the algorithm. Sometimes the *caller* should be memoized, not the method you're looking at. -- The memoized function should be timed with multiple configurations to make sure it doesn't inadvertently slow down some other configuration. -- When memoizing values based on a dictionary key composed of input arguments, you don't need to use all of the arguments if some of them don't affect the return values. Historical Note: The initial Memoizer implementation actually handled the caching of values for the wrapped methods, based on a set of generic algorithms for computing hashable values based on the method's arguments. This collected caching logic nicely, but had two drawbacks: Running arguments through a generic key-conversion mechanism is slower (and less flexible) than just coding these things directly. Since the methods that need memoized values are generally performance-critical, slowing them down in order to collect the logic isn't the right tradeoff. Use of the memoizer really obscured what was being called, because all the memoized methods were wrapped with re-used generic methods. This made it more difficult, for example, to use the Python profiler to figure out how to optimize the underlying methods. iNtCountercBs)eZdZdZdZdZRS(s Base class for counting memoization hits and misses. We expect that the metaclass initialization will have filled in the .name attribute that represents the name of the function being counted. cCs,||_d|_d|_ti|dS(s iN(t method_namethittmisst CounterListtappend(tselfR((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyt__init__s   cCs%d}||i|i|ifGHdS(Ns %7d hits %7d misses %s()(RRtname(Rtfmt((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pytdisplayscCs2yt|i|iSWntj odSXdS(Ni(tcmpRtAttributeError(Rtother((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyt__cmp__s(t__name__t __module__t__doc__RR R(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs  t CountValuecBseZdZdZRS(s  A counter class for simple, atomic memoized values. A CountValue object should be instantiated in a class for each of the class's methods that memoizes its return value by simply storing the return value in its _memo dictionary. We expect that the metaclass initialization will fill in the .underlying_method attribute with the method that we're wrapping. We then call the underlying_method method after counting whether its memoized value has already been set (a hit) or not (a miss). cOsQ|d}|i|ijo|id|_n|id|_|i||S(Nii(Rt_memoRRtunderlying_method(Rtargstkwtobj((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyt__call__s  (RRRR(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs t CountDictcBs eZdZdZdZRS(s A counter class for memoized values stored in a dictionary, with keys based on the method's input arguments. A CountDict object is instantiated in a class for each of the class's methods that memoizes its return value in a dictionary, indexed by some key that can be computed from one or more of its input arguments. We expect that the metaclass initialization will fill in the .underlying_method attribute with the method that we're wrapping. We then call the underlying_method method after counting whether the computed key value is already present in the memoization dictionary (a hit) or not (a miss). cCsti||||_dS(s N(RRtkeymaker(RRR((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRscOs|d}y|i|i}Wn#tj o|id|_nEX|i||}||jo|id|_n|id|_|i||S(Nii(RRtKeyErrorRRRR(RRRRt memo_dicttkey((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs  (RRRRR(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs tMemoizercBseZdZdZRS(sMObject which performs caching of method calls for its 'primary' instance.cCsdS(N((R((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs(RRRR(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRscCs9|o |GHntixtD]}|iq!WdS(N(RtsortR (ttitletcounter((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pytDumps   tMemoized_MetaclasscBseZdZRS(cCstt|i|||xi|idgD]U}|i}|id||_|||_ti |d|}t |||q/WdS(Ntmemoizer_counterst.( tsuperR#RtgetRRRRttypest MethodTypetNonetsetattr(tclsRtbasestcls_dictR!Rtreplacement_method((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyRs  (RRR(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyR#scCs dadS(Ni(t use_memoizer(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pytEnableMemoizations(t __revision__RR(R*R0RtobjectRRRRR"ttypeR#R1(((s0install/lib/scons-2.0.0.final.0/SCons/Memoize.pyts` #