Tasks¶

TaskQueue.add_task(fn, *args, **kwargs)[source]¶

Add the function fn to the queue to be invoked with args and kwargs as arguments. If the TaskQueue is not currently running, it will be started now.

TaskQueue.yield_task(fn, *args, **kwargs)[source]¶

Like add_task but will call the function as a coroutine, allowing you to yield the return value from within a function decorated with @tornado.gen.coroutine or @tornado.gen.engine.

Usage:

def add(arg1, arg2):
  return arg1 + arg2

@tornado.gen.engine
def caller():
  value = yield TaskQueue.instance('myqueue').yield_task(add, 1, 2)
  print value #prints 3

TaskQueue.run()[source]¶

Start processing jobs in the queue. You should not need to call this as add_task automatically starts the queue. Processing threads will stop when there are no jobs available in the queue for at least idle_timeout seconds.

TaskQueue.__len__()[source]¶

Returns the number of active threads plus the number of queued tasks/

AwaitableInstance¶

class toto.tasks.AwaitableInstance(instance, queue_name='pool')[source]¶

Use AwaitableInstance(instance) to create a wrapper around instance that may be used to automatically execute any of instance methods in a TaskQueue. Use queue_name to specify the name of the TaskQueue to use.

Usage:

class MyClass(object):
  def do_something(self, arg):
    ...
    return result

awaitable = AwaitableInstance(MyClass())
result = yield awaitable.do_something(args)

The above will call do_something() in a TaskQueue and await the result.

InstancePool¶

class toto.tasks.InstancePool(instances, default_queue='pool')[source]¶

Create a pool wrapping one or more instances of a class. The pool has convenience methods for asynchronous execution and transactions. May be used e.g. to share non-threadsafe classes among various threads.

Invoking a method other than those documented below will remove an instance from the pool, invoke the method, then return it to the pool.

instances should be either a single instance or a collection of instances.

default_queue is the name of the TaskQueue that will be used for all asynchronous operations.

InstancePool.await(queue_name=None)[source]¶

Create an AwaitableInstance wrapping this queue.

InstancePool.instance()[source]¶

For use with the with operator. Allows multiple methods to be called on a single instance instance in the pool and returns it when complete.

For example:

with pool.instance() as i:
  i.operation1()
  i.operation2()

Will remove an instance from the pool, call two methods on it, then return it to the queue.

InstancePool.transaction(function)[source]¶

Calls function with an instance from the pool as the single argument. Returns the result of function(instance)

InstancePool.async_transaction(function, queue_name=None)[source]¶

Like transaction() but runs the operation in a TaskQueue. queue_name may be specified to override the default.

InstancePool.await_transaction(function, queue_name=None)[source]¶

Like transaction() but runs the operation in a TaskQueue and returns a Future that may be used to retrieve the result. Must be called from a Tornado IOLoop. queue_name may be specified to override the default.