A powerful workflow engine implemented in pure Python
This is just a very quick release to fix an issue with 1.1.5 that could prevent people from deserializing their existing workflows.
We've made some significant improvements to our documentation and heavily refactored our code around events. We made some minor performance improvements by reducing the size of our serialization (but we are in the midst of a much larger refactor of serialization with even larger payoffs). More details follow.
Full Changelog: https://github.com/sartography/SpiffWorkflow/compare/v1.1.3...v1.1.5
Correcting an issue with data creation in a multi-instance sub-workflow / call-activity.
This release just updates the documentation, and gets PiPy and GitHub in sync. Please see Release 1.1.1 for additional release notes as these will occur on top of each other.
Full Changelog: https://github.com/sartography/SpiffWorkflow/compare/1.1...1.1.1
Full Changelog: https://github.com/sartography/SpiffWorkflow/compare/1.0...1.1
After a year of extensive development to get SpiffWorkflow to support our effort at building a BPMN based workflow application, Sartography (our software development company) is pleased to announce an official release of Version 1.0 of SpiffWorkflow.
That's an auspicious jump from the previous 0.6, but we think it merits the version. We have used Spiff Workflow in roughly the present state for several months in a production environment supporting real users. We believe it is ready for this 1.0 moniker. We look forward to the rapid release of many subsequent versions over the coming years.
Preserved Features - Most of the previously-existing tests were maintained, so these changes will hopefully still support any existing users. We’ve kept dependencies on external dependencies very low. And where we added things like Box and Levenshtein along the way, we pulled them back out, replacing them with internal implementations.
DMN Support - While it is not eye-watering perfect, we have a baseline of support for DMN processing that is fully integrated into the existing Execution Engine, so your DMN rules can be applied in the same language as your preferred execution language for BPMN expressions.
Forms - We’ve added support for Camunda-specific web forms, which are currently parsed and returned as JSON data structures. We’ve used this to generate Formly (an NPM package for Angular) forms for web pages and it works great, but we hope to extract it in the future and pass this through directly, to make SpiffWorkflow more general-purpose and allow just about any additional complex data to accompany a user task.
BPMN Additional Support - We made nearly innumerable enhancements to BPMN support. These, at a glance, include:
Improved Logging and Better Errors - We spent a lot of time improving and standardizing error messages in Spiff so it is easier to tell what is going on where, when and why, particularly as you attempt to debug issues that occur in Script Tasks. We can even tell you the line in the script that failed, and offer “did you mean…” suggestions through a simple implementation of the Levenshtein distance algorithm. We also added the ability to halt a workflow at a specific engine_step, which can be useful if you want to run tests of your workflow, but have it stop at a specific moment.
Dot Notation - Our one major departure from Python syntax, we added the ability to use dot notation when referencing values deep within the Task Data, we found that using MyData.key was easier than MyData[“key”] for many areas in BPMN and DMN. Resets - We added the ability to reset the workflow to any previously completed task, and resume work from that point.
Navigation - We created a rich and complex navigation generator that is capable of describing any point in the tree of tasks. We found we desperately needed a way to tell where we were when parsing complex diagrams. While it didn’t prove terribly useful in the user interface so far, it’s a ludicrously-robust tool for debugging. At any moment, you can now print out the full data structure in a meaningful, visual way directly from the command line.
Tests - We added hundreds of tests, many of which load up example BPMN diagrams and clearly show expectations of how Spiff should behave in a large array of situations and edge cases.
Performance Improvements - As we used this in production we found numerous performance issues, which we have tried to address - mostly related to serializing and deserializing the workflows which happens a lot in a stateless web-based architecture. We added and iteratively refined some tools for gathering and reporting performance metrics, opening these up to the larger application, so you can debug and get a consistent log and tell if it is your app that is slow, or if things are slow within Spiff.
My God, It’s Full of Edge Cases - Some 40% of our commits are related to edge cases we encountered as we developed dozens of real-world BPMN/DMN-based workflows and fixing issues as we attempted to use complex BPMN elements together.
We did need to remove a few items from Spiff along the way, the most notable is a drop in support for Python 2. Python 3.6, 3.7 and 3.8 are all supported and a part of our automated test suite.
Sartography (Sartography.com) undertook these efforts to support the development of a custom web application for the University of Virginia. We would like to thank UVA for their support and trust in allowing us to take on the mammoth task of building a general-purpose workflows system, and contributing something back to the open source community.
Bruce Silver, the author of BPMN Quick and Easy Using Method and Style, whose work we referenced extensively as we made implementation decisions and educated ourselves on the BPMN and DMN standards.
Samuel Abels (@knipknap) for keeping SpiffWorkflow alive for the past few years, and offering us commit access to make these contributions, and Matthew Hampton for kicking this project off and giving us sound footing on which to build.
Camunda, for whom, without their open source Modeler and the BPMN.js library, we would not have the tools to effectively build out our models, embed an editor in our application, and pull this mad mess together.
Kelly McDonald (@w4kpm) who dove deeper into the core of SpiffWorkflow than anyone else, and was instrumental in helping us get some of these major enhancements working correctly.
We would like to thank Denny Weinberg for his early contributions to DMN support, which we used as a baseline and then extended.
Clean-up: Moving forward, we want to drop direct support and parsing of Camunda-specific elements. We would like to pass non-BPMN parsing on to the calling application, allowing users to build plugin-based architectures that can address domain-specific objectives.
Cleaner Serialization: While our serialization and deserialization is rigorously tested, the JSON it produces is not easy to understand on its own. We would love to take time to clean this up, and make it easier to parse and extract data from a running workflow’s JSON representation.
We are also working on a layered system architecture that will make the development of web-based applications for authoring and running BPMN-based workflows easier to accomplish. We’ve built a full REST based API on top of SpiffWorkflow for our pilot application, but it isn’t general enough to release. We’ve also created Javascript applications for both editing and running workflows specifications in a browser, but these still need to be generalized.
A full stack of interconnected open source tools is the direction we are headed, along with a plug-in based architecture that will work seamlessly across that stack. We believe such grand ambitions are possible. Our ultimate ambition is the development of a Platform as a Service Business that facilitates the use of an increasingly standards compliant suite of powerful open source libraries and components.