Version 0.6.2 in initial development: new features coming!

I have recently been working on version 0.6.2. I am excited about the changes. The biggest one is that I have made the queue system definition fully modular and moved it to the batch_systems sub-package, so adding a new queue system is now very easy.

I also completely changed the local queue system and made it a database using daemon process instead. This adds a few new requirements, but it is worth it, the local queue is now extremely fast and stable.

Also, on a whim I added the ability to submit files and shell scripts with the fyrd console script, so you can now do things like::

fyrd run --extra_vars 'output:input:\(.*\).txt:\1.out' "my_algorithm {input} > {output}" "data/{input}"


fyrd submit --wait script.qsub

Finally, the other big change is the appearance of a decorator called jobify that massively simplifies submitting functions:

>>> import fyrd
>>> @fyrd.jobify(name='test_job', mem='1GB')
... def test(string, iterations=4):
...     """This does basically nothing!"""
...     outstring = ""
...     for i in range(iterations):
...         outstring += "Version {0}: {1}".format(i, string)
...     return outstring
>>> j = test('hi')
>>> j.get()
'Version 0: hiVersion 1: hiVersion 2: hiVersion 3: hi'
>>> @fyrd.jobify()
... def raise_me(num, power=10):
...     return num**power
>>> raise_me(10).get()

I hope to get an alpha version of 0.6.2 out shortly.