diff options
Diffstat (limited to 'doc/build/caching.rst')
| -rw-r--r-- | doc/build/caching.rst | 387 |
1 files changed, 387 insertions, 0 deletions
diff --git a/doc/build/caching.rst b/doc/build/caching.rst new file mode 100644 index 0000000..9f63750 --- /dev/null +++ b/doc/build/caching.rst @@ -0,0 +1,387 @@ +.. _caching_toplevel: + +======= +Caching +======= + +Any template or component can be cached using the ``cache`` +argument to the ``<%page>``, ``<%def>`` or ``<%block>`` directives: + +.. sourcecode:: mako + + <%page cached="True"/> + + template text + +The above template, after being executed the first time, will +store its content within a cache that by default is scoped +within memory. Subsequent calls to the template's :meth:`~.Template.render` +method will return content directly from the cache. When the +:class:`.Template` object itself falls out of scope, its corresponding +cache is garbage collected along with the template. + +The caching system requires that a cache backend be installed; this +includes either the `Beaker <http://beaker.readthedocs.org/>`_ package +or the `dogpile.cache <http://dogpilecache.readthedocs.org>`_, as well as +any other third-party caching libraries that feature Mako integration. + +By default, caching will attempt to make use of Beaker. +To use dogpile.cache, the +``cache_impl`` argument must be set; see this argument in the +section :ref:`cache_arguments`. + +In addition to being available on the ``<%page>`` tag, the caching flag and all +its options can be used with the ``<%def>`` tag as well: + +.. sourcecode:: mako + + <%def name="mycomp" cached="True" cache_timeout="60"> + other text + </%def> + +... and equivalently with the ``<%block>`` tag, anonymous or named: + +.. sourcecode:: mako + + <%block cached="True" cache_timeout="60"> + other text + </%block> + + +.. _cache_arguments: + +Cache Arguments +=============== + +Mako has two cache arguments available on tags that are +available in all cases. The rest of the arguments +available are specific to a backend. + +The two generic tags arguments are: + +* ``cached="True"`` - enable caching for this ``<%page>``, + ``<%def>``, or ``<%block>``. +* ``cache_key`` - the "key" used to uniquely identify this content + in the cache. Usually, this key is chosen automatically + based on the name of the rendering callable (i.e. ``body`` + when used in ``<%page>``, the name of the def when using ``<%def>``, + the explicit or internally-generated name when using ``<%block>``). + Using the ``cache_key`` parameter, the key can be overridden + using a fixed or programmatically generated value. + + For example, here's a page + that caches any page which inherits from it, based on the + filename of the calling template: + + .. sourcecode:: mako + + <%page cached="True" cache_key="${self.filename}"/> + + ${next.body()} + + ## rest of template + +On a :class:`.Template` or :class:`.TemplateLookup`, the +caching can be configured using these arguments: + +* ``cache_enabled`` - Setting this + to ``False`` will disable all caching functionality + when the template renders. Defaults to ``True``. + e.g.: + + .. sourcecode:: python + + lookup = TemplateLookup( + directories='/path/to/templates', + cache_enabled = False + ) + +* ``cache_impl`` - The string name of the cache backend + to use. This defaults to ``'beaker'``, indicating + that the 'beaker' backend will be used. + +* ``cache_args`` - A dictionary of cache parameters that + will be consumed by the cache backend. See + :ref:`beaker_backend` and :ref:`dogpile.cache_backend` for examples. + + +Backend-Specific Cache Arguments +-------------------------------- + +The ``<%page>``, ``<%def>``, and ``<%block>`` tags +accept any named argument that starts with the prefix ``"cache_"``. +Those arguments are then packaged up and passed along to the +underlying caching implementation, minus the ``"cache_"`` prefix. + +The actual arguments understood are determined by the backend. + +* :ref:`beaker_backend` - Includes arguments understood by + Beaker. +* :ref:`dogpile.cache_backend` - Includes arguments understood by + dogpile.cache. + +.. _beaker_backend: + +Using the Beaker Cache Backend +------------------------------ + +When using Beaker, new implementations will want to make usage +of **cache regions** so that cache configurations can be maintained +externally to templates. These configurations live under +named "regions" that can be referred to within templates themselves. + +.. versionadded:: 0.6.0 + Support for Beaker cache regions. + +For example, suppose we would like two regions. One is a "short term" +region that will store content in a memory-based dictionary, +expiring after 60 seconds. The other is a Memcached region, +where values should expire in five minutes. To configure +our :class:`.TemplateLookup`, first we get a handle to a +:class:`beaker.cache.CacheManager`: + +.. sourcecode:: python + + from beaker.cache import CacheManager + + manager = CacheManager(cache_regions={ + 'short_term':{ + 'type': 'memory', + 'expire': 60 + }, + 'long_term':{ + 'type': 'ext:memcached', + 'url': '127.0.0.1:11211', + 'expire': 300 + } + }) + + lookup = TemplateLookup( + directories=['/path/to/templates'], + module_directory='/path/to/modules', + cache_impl='beaker', + cache_args={ + 'manager':manager + } + ) + +Our templates can then opt to cache data in one of either region, +using the ``cache_region`` argument. Such as using ``short_term`` +at the ``<%page>`` level: + +.. sourcecode:: mako + + <%page cached="True" cache_region="short_term"> + + ## ... + +Or, ``long_term`` at the ``<%block>`` level: + +.. sourcecode:: mako + + <%block name="header" cached="True" cache_region="long_term"> + other text + </%block> + +The Beaker backend also works without regions. There are a +variety of arguments that can be passed to the ``cache_args`` +dictionary, which are also allowable in templates via the +``<%page>``, ``<%block>``, +and ``<%def>`` tags specific to those sections. The values +given override those specified at the :class:`.TemplateLookup` +or :class:`.Template` level. + +With the possible exception +of ``cache_timeout``, these arguments are probably better off +staying at the template configuration level. Each argument +specified as ``cache_XYZ`` in a template tag is specified +without the ``cache_`` prefix in the ``cache_args`` dictionary: + +* ``cache_timeout`` - number of seconds in which to invalidate the + cached data. After this timeout, the content is re-generated + on the next call. Available as ``timeout`` in the ``cache_args`` + dictionary. +* ``cache_type`` - type of caching. ``'memory'``, ``'file'``, ``'dbm'``, or + ``'ext:memcached'`` (note that the string ``memcached`` is + also accepted by the dogpile.cache Mako plugin, though not by Beaker itself). + Available as ``type`` in the ``cache_args`` dictionary. +* ``cache_url`` - (only used for ``memcached`` but required) a single + IP address or a semi-colon separated list of IP address of + memcache servers to use. Available as ``url`` in the ``cache_args`` + dictionary. +* ``cache_dir`` - in the case of the ``'file'`` and ``'dbm'`` cache types, + this is the filesystem directory with which to store data + files. If this option is not present, the value of + ``module_directory`` is used (i.e. the directory where compiled + template modules are stored). If neither option is available + an exception is thrown. Available as ``dir`` in the + ``cache_args`` dictionary. + +.. _dogpile.cache_backend: + +Using the dogpile.cache Backend +------------------------------- + +`dogpile.cache`_ is a new replacement for Beaker. It provides +a modernized, slimmed down interface and is generally easier to use +than Beaker. As of this writing it has not yet been released. dogpile.cache +includes its own Mako cache plugin -- see :mod:`dogpile.cache.plugins.mako_cache` in the +dogpile.cache documentation. + +Programmatic Cache Access +========================= + +The :class:`.Template`, as well as any template-derived :class:`.Namespace`, has +an accessor called ``cache`` which returns the :class:`.Cache` object +for that template. This object is a facade on top of the underlying +:class:`.CacheImpl` object, and provides some very rudimental +capabilities, such as the ability to get and put arbitrary +values: + +.. sourcecode:: mako + + <% + local.cache.set("somekey", type="memory", "somevalue") + %> + +Above, the cache associated with the ``local`` namespace is +accessed and a key is placed within a memory cache. + +More commonly, the ``cache`` object is used to invalidate cached +sections programmatically: + +.. sourcecode:: python + + template = lookup.get_template('/sometemplate.html') + + # invalidate the "body" of the template + template.cache.invalidate_body() + + # invalidate an individual def + template.cache.invalidate_def('somedef') + + # invalidate an arbitrary key + template.cache.invalidate('somekey') + +You can access any special method or attribute of the :class:`.CacheImpl` +itself using the :attr:`impl <.Cache.impl>` attribute: + +.. sourcecode:: python + + template.cache.impl.do_something_special() + +Note that using implementation-specific methods will mean you can't +swap in a different kind of :class:`.CacheImpl` implementation at a +later time. + +.. _cache_plugins: + +Cache Plugins +============= + +The mechanism used by caching can be plugged in +using a :class:`.CacheImpl` subclass. This class implements +the rudimental methods Mako needs to implement the caching +API. Mako includes the :class:`.BeakerCacheImpl` class to +provide the default implementation. A :class:`.CacheImpl` class +is acquired by Mako using a ``importlib.metatada`` entrypoint, using +the name given as the ``cache_impl`` argument to :class:`.Template` +or :class:`.TemplateLookup`. This entry point can be +installed via the standard `setuptools`/``setup()`` procedure, underneath +the `EntryPoint` group named ``"mako.cache"``. It can also be +installed at runtime via a convenience installer :func:`.register_plugin` +which accomplishes essentially the same task. + +An example plugin that implements a local dictionary cache: + +.. sourcecode:: python + + from mako.cache import Cacheimpl, register_plugin + + class SimpleCacheImpl(CacheImpl): + def __init__(self, cache): + super(SimpleCacheImpl, self).__init__(cache) + self._cache = {} + + def get_or_create(self, key, creation_function, **kw): + if key in self._cache: + return self._cache[key] + else: + self._cache[key] = value = creation_function() + return value + + def set(self, key, value, **kwargs): + self._cache[key] = value + + def get(self, key, **kwargs): + return self._cache.get(key) + + def invalidate(self, key, **kwargs): + self._cache.pop(key, None) + + # optional - register the class locally + register_plugin("simple", __name__, "SimpleCacheImpl") + +Enabling the above plugin in a template would look like: + +.. sourcecode:: python + + t = Template("mytemplate", + file="mytemplate.html", + cache_impl='simple') + +Guidelines for Writing Cache Plugins +------------------------------------ + +* The :class:`.CacheImpl` is created on a per-:class:`.Template` basis. The + class should ensure that only data for the parent :class:`.Template` is + persisted or returned by the cache methods. The actual :class:`.Template` + is available via the ``self.cache.template`` attribute. The ``self.cache.id`` + attribute, which is essentially the unique modulename of the template, is + a good value to use in order to represent a unique namespace of keys specific + to the template. +* Templates only use the :meth:`.CacheImpl.get_or_create()` method + in an implicit fashion. The :meth:`.CacheImpl.set`, + :meth:`.CacheImpl.get`, and :meth:`.CacheImpl.invalidate` methods are + only used in response to direct programmatic access to the corresponding + methods on the :class:`.Cache` object. +* :class:`.CacheImpl` will be accessed in a multithreaded fashion if the + :class:`.Template` itself is used multithreaded. Care should be taken + to ensure caching implementations are threadsafe. +* A library like `Dogpile <http://pypi.python.org/pypi/dogpile.core>`_, which + is a minimal locking system derived from Beaker, can be used to help + implement the :meth:`.CacheImpl.get_or_create` method in a threadsafe + way that can maximize effectiveness across multiple threads as well + as processes. :meth:`.CacheImpl.get_or_create` is the + key method used by templates. +* All arguments passed to ``**kw`` come directly from the parameters + inside the ``<%def>``, ``<%block>``, or ``<%page>`` tags directly, + minus the ``"cache_"`` prefix, as strings, with the exception of + the argument ``cache_timeout``, which is passed to the plugin + as the name ``timeout`` with the value converted to an integer. + Arguments present in ``cache_args`` on :class:`.Template` or + :class:`.TemplateLookup` are passed directly, but are superseded + by those present in the most specific template tag. +* The directory where :class:`.Template` places module files can + be acquired using the accessor ``self.cache.template.module_directory``. + This directory can be a good place to throw cache-related work + files, underneath a prefix like ``_my_cache_work`` so that name + conflicts with generated modules don't occur. + +API Reference +============= + +.. autoclass:: mako.cache.Cache + :members: + :show-inheritance: + +.. autoclass:: mako.cache.CacheImpl + :members: + :show-inheritance: + +.. autofunction:: mako.cache.register_plugin + +.. autoclass:: mako.ext.beaker_cache.BeakerCacheImpl + :members: + :show-inheritance: + |