News
Version 0.6.2-beta.1 Released
06 Mar 2018
This version brings a major overhaul to the structure of the code, while leaving the API mostly intact. Importantly, it fixes the issue that resulted in the local mode not using 100% of CPU power.
Major Changes
- Completely rewrote the local mode to use Pyro4 and SQLAlchemy, it is now much, much faster and more stable
- Batch system definitions now fully modular and are contained in the
fyrd.batch_systemspackage.options.pyhas also been moved into this package, which allows any programmer to add a new batch system definition to fyrd by just editing the contents of that small subpackaged - Updated console script to allow running arbitrary shell scripts on the console with
fyrd runor submitting any number of existing job files usingfyrd sub. Added the new alias scriptsfrunandfsubfor those new modes also. Both new modes will accept the--waitargument, meaning that they will block until the jobs complete. - Documentation overhauled to update API and add instructions on creating a new batch system,
these instructions are duplicated in the README within the
batch_systemspackage folder. - Full support for array job parsing for both torque and slurm. We now create on job entry
for each array job child, instead of for each array job. To manage this, the
fyrd.queue.Queue.QueueJobclass was moved tofyrd.queue.QueueJoband split to add a child class,fyrd.queue.QueueChild. All array jobs not have onefyrd.queue.QueueJobjob, plus onefyrd.queue.QueueChildjob for each of their children, which are stored in thechildrendictionary in thefyrd.queue.QueueJobclass. - Added a
getmethod to thefyrd.queue.Queueclass to allow a user to get outputs from a list of jobs, loops continuously through the jobs so that jobs are not lost. - Added tqdm as a requirement and enabled progressbars in multi-job wait and get
Minor Changes
- Updated the documentation to include a changelog, which will only contain change information for version 0.6.2a1 onwards.
- Added additional tests to cover the new changes as well as generally increase test suite coverage.
- Several small bug fixes
Version 0.6.2 in initial development: new features coming!
16 Aug 2017
I have recently been working on version 0.6.2. I am excited about the changes. read more
Version 0.6.1b9 Released
03 Aug 2017
This fixes several bugs in documentation and packaging but makes no changes to the code.
Version 0.6.1-beta.8 Released
03 Aug 2017
I decided to put fyrd on PyPI so that it can now be installed directly with pip. This required a few changes to the documentation and setup.py though, so this release just bundles all of those changes to make v0.6.1-beta.8 consistent between PyPI and github.
Version 0.6.1-beta.7 Released
02 Aug 2017
I have not had much time to work on fyrd in the last few months, so this release represents fairly minor fixes.
read morePersistent Problems with Local Mode
30 Nov 2016
It has come to my attention that the local mode is not very stable, it sometimes takes a very long time to run jobs, and occasionally fails completely for no good reason. I don’t have enough time to fix this in the next few days, it will probably require a large rewrite. The batch system functionality is intact though, I will post again when the local mode works.
Version 0.6.1-beta.6 Released
14 Nov 2016
All scripts that used to be in bin/ have been moved into a single
entry-point script called fyrd and are managed as subcommands instead
of as separate scripts.
Scripts with the same name of those previously in bin/ still remiain, but
they have thus all been converted to shell scripts that are simple aliases of
the relevant fyrd subcommands.
Version 0.6.1-beta.5 Released
02 Nov 2016
This version represents a large API break and a significant step towards making fyrd more similar to the multiprocessing library.
The largest change in this version is that way the Job objects return
outputs. Previously, Job.get() would return (exitcode, stdout, stderr), now
it just returns the output, which is either the function return value, or the
contents of STDOUT.