The Scheer PAS Process Engine

When it comes to automation and orchestration of workflows or even complex end-2-end business processes, it is beneficial to look under the hood of potential solution providers and understand how the process concept is supported in practice.

This includes not only the implementation phase, i.e. designing the process with a standardized modelling notation, but especially the way how processes are executed. Most business processes involve asynchronous processing, require the persistence of state information and proper error handling. It is important that the underlying technological concepts of the engines executing the processes support these requirements.

In this blog post, I will explain the basic concept of the Scheer PAS Process Engine.

Process Design and Implementation

The Scheer PAS Designer is the Low-Code tool of the PAS platform to design and develop end to end processes and their implementation. The modelling of the process is done in BPMN (Business Process Model and Notation) which is a standardized graphical notation, that provides a visual representation of the steps and activities involved in a business process. BPMN has the advantage that it is easily understandable by process experts, business users and developers which makes it a powerful notation to not only model but also automate processes. Automating a process with Scheer PAS always means drawing a BPMN in the first step. The figure below shows a simple example using the most common notation elements:

During process implementation, the modeler connects these elements (task, gateway, event, …) with activities to be carried out, which can be pre-built functional building blocks from libraries, user forms , or specific logic captured in a variety of ways (script, mapping diagram, activity diagram), for which Scheer PAS brings its own Low-Code editors.

The Process Execution

Process models are translated by Scheer PAS into semantically equivalent state machines, which is a very powerful approach to define systems with complex and dynamic behaviour. It describes the behaviour of systems that can exist in different states and transitions between them based on certain conditions or events. Execution of these state machines is where the Scheer PAS Runtime comes into play. Related to BPMN process execution, the key elements of this process state machine can be described as follows:

States: A state represents a specific condition or situation that a process instance can be in at a given point in time.

Transitions: Transitions define the movement of a process instance from one state to another in response to a trigger or event.

Events: Events are occurrences or stimuli that can trigger a transition from one state to another.

Actions: Actions are tasks or operations associated with a state or a transition.

Following this state machine approach results in a set of advantages that our process engine can provide when it comes to the execution of a business process:

Asynchronous processing: The state machine concept is inherently asynchronous. This means that it is not necessary for previously started process instances to have ended in order to start a new process instance. Or in other words, several process instances can run in parallel, each one in different stages of the process.

Well-defined state: Each running process instance is in a well-defined state, at all times. It is always clear and transparent which actions have been performed, which events have occurred and what is still to happen.

Data safety: Information about the current state of a process instance as well as instance related process data can be persisted. This means, even if a running process instance is aborted due to unforeseeable reasons (e.g. system crash, power outage, …), the data is not lost, and the process instance can be restarted exactly in the same state with the same data.

In addition to the process state machine for executing the BPMN process, the Scheer PAS process engine also contains a service state machine, which encloses the process state machine.

This service state machine consists of a small set of states surrounding the process state machine and defining the Initialized, Aborted, Error and Done states of the process. This allows transitions from each process state into an Error, Abort or Done state without the need to explicitly model these options in the BPMN. The main advantage that comes with this approach is how potential errors can be handled. Let’s assume the following example:

One activity in the BPMN represents the task to connect to an SAP system and fetch data about a business partner. Due to some reason, the SAP system is not available at the time the process is executed. If the developer has special requirements on error handling and modelled it in the implementation of the process, then this explicit error handling defines what will happen. An example for an explicit handling of the error could be to include a timer event that triggers a reconnect to the SAP system on the next working day. The original process proceeds when the connection can be established.

If, however, there are no specific requirements, and the potential error is not explicitly taken care of – which is mostly the case in practice – then the process transitions automatically in the Error state of the service state machine. This prevents that the process gets stuck in some undefined intermediate state. There are two options now how to proceed from this state: The activity can be repeated manually when the SAP system is available again, or an automatic retry can be configured in which the process engine retries the SAP connection at regular intervals.

In general, this approach allows the developer to identify errors that occur in production, fix the process implementation (including proper error handling if reasonable), redeploy the improved service and restart the process instances from the state where they went into error. All this without losing data or actions that have been processed in the affected instances before the error occurred (-> see Data safety), and without affecting other running instances of the process.

REST Interface for processes

Within the Scheer PAS platform, each implementation of a process is built and deployed as individual service in a Docker container. This has the advantage that each process runs separately and is not affected by errors or maintenance of other services/processes.

To access the process, the Scheer PAS Process engine automatically generates a REST interface for the process state machine. This REST API provides endpoints to gather information on BPMN process instances and their state(s), and to trigger transitions. This approach also provides the option to realize inter-process communication via REST APIs, to build a custom UI layer on top of the process execution or to manage and publish process APIs via the Scheer PAS API Management.

Outlook: Scaling of Processes

With one of the coming Scheer PAS releases we will move to a Kubernetes platform architecture. In addition to separate services for each process, this will allow an easy and automated scaling, i.e. several replicas of the same service / process running in parallel. An individual process can be deployed several times, depending on the required resources, and handle more load in parallel or provide a high availability setup.

The highlight is: Even when there are several replicas of the same service, each executing different instances of the same process, the persisted data and state information will be synchronized between the replicas. This means if one replica of the service crashes, another replica will pick up the started process instances at the point where they were stopped and process them further.

I will explain this unique and powerful concept a bit more in detail in another blog post.