Process Mining is a process management technique. It enables companies to reconstruct and evaluate business processes based on digital traces in IT systems. With the help of process mining techniques, it is possible to discover processes and review event logs to evaluate conformance with defined processes. This provides the necessary insights to manage, control and improve business processes. The use of process mining is therefore essential for the digital transformation of the business world. With process mining, companies gain insights into their complex business processes and thus have the opportunity to take action measures and make decisions as part of transformation and automation projects. For many entrepreneurs, however, this important automation foundation remains a closed book. The following article is therefore intended to provide an introduction to this rather complicated topic and provide a little clearer vision in the thicket of process automation.
Process Mining: Insights into Digital Mysteries
Sometimes the work just doesn't run and the fundamental problem remains a mystery - we certainly all know this feeling all too well. But help is at hand: Instead of searching for hours in the breadcrumbs of the process landscape, the method of so-called process mining offers itself for such cases. Here, real workflows are compared with theory, which should lead to better transparency as well as insight into the processes. But why is this necessary at all? Well, unfortunately, reality rarely corresponds to the optimal conception.
Let's take the classic beach vacation as an example: Mostly influenced by promotional images, we imagine empty and beautiful beaches that are then completely overcrowded when we arrive or with razor-sharp stones under the rather dark brown sea surface. The same applies to corporate processes as in this example. When an employee describes his or her process, it often appears uncomplicated and clear. However, if you look at this process in detail, you will notice that the real process is much more complex.
So where does the difference between a real and an ideal conception of business processes come from?
The Story behind Process Mining
The first attempts to establish process mining as an independent technology for the analysis of business processes originated in the Netherlands. Dr. Wil van der Aalst, considered by many to be the godfather of Process Mining (Podcast episode with Wil van der Aalst), began to explore the possibilities of automated process discovery based on event logs in the late 1990s at Eindhoven University of Technology (TU/e) while studying workflow and business workflow management.
His approach combines the strengths of process- and data-centric analytics. For him, the technology bridges the gap between traditional model-based process analysis and data-centric analysis techniques such as machine learning and data mining. Dr. Wil van der Aalst sees three areas where process mining is useful:
In determining the actual operation of processes using event logs in which historical information is stored.
In conformity, which examines how well the reality of what is recorded in the event log reflects what is happening in the real world.
In improvement, examining how management can improve business processes to eliminate inefficiencies.
Van der Aalst first used the term "process mining" in a research proposal he wrote in 1998. In 2011, van der Aalst published his first book on the subject called "Process Mining: Data Science in Action." But until 2011, when the Munich-based company Celonis was founded, Wil van der Aalst's approach had little practical relevance. Also in the late 1990s, IBM's Almaden Research Center was actively developing algorithms for determining process models from event logs. In December 1997, IBM patented a method for automated process model discovery.
In 2009, the Institute of Electrical and Electronics Engineers (IEEE) established the IEEE Task Force on Process Mining, which is supported by over 75 organizations. In 2011, the IEEE published its Process Mining Manifesto with the goal of "advancing the research, development, education, implementation, evolution, and understanding of process mining."
The biggest value accelerator for the technology has been the collaboration of Process Mining vendors with well-known enterprise applications such as SAP, Oracle and Salesforce. These vendors have heavily promoted process mining to improve the efficiency of these applications. Eventually, process mining has moved into areas other than process discovery, such as compliance testing, productivity improvement, customer interactions, and social networking. It has even spread into areas of the Internet of Things (IoT), manufacturing, and logistics distribution networks, which have demonstrated the sustainable value-added capabilities of process mining. In 2019, Aachen hosted the International Conference on Process Mining 2019 for the first time.
What is Process Mining?
Process mining is a Process Management technique. It aims to discover, monitor and improve process flows by extracting readily available knowledge from information systems event logs. Process mining provides companies with complete visibility into how they really operate. With these insights, companies can then identify opportunities for process optimization.
Process mining includes automated process discovery (extraction of process models from an event log), conformance checking (monitoring of deviations by comparing model and log), organization mining, automated construction of simulation models, model repair, case prediction, and history-based recommendations.
Process mining technology could also be compared to magnetic resonance imaging (MRI) technology, which collects information from the body's cells to create an image - only in a business environment. Doctors then use this MRI image to diagnose health conditions. Process mining works on a similar principle: it collects data from the smallest part of process activities and assembles the parts into a picture that companies can use to diagnose the state of their workflows.
Process mining is changing the way companies operate and manage their business processes. In their quest for process quality, process mining enables companies to truly know their process, evaluate it against the ideal process model, and optimize it as needed. Process mining platforms combine technologies from Data A, Data Mining, Process Analytics and Business Intelligence (BI) into a holistic approach to generate insights into processes that companies can use to continuously optimize the value chain.
What is Task Mining?
However, process mining should not be confused or equated with task mining, which unfortunately happens too often. Task mining is a complementary approach to where useful information is derived from low-level event data available in UI logs. These UI logs describe the individual steps taken by a user (e.g., when using a workstation) based on keystrokes, mouse clicks, and data inputs. Additional mining functions interpret the data by applying NLP (Natural Language Processing) and OCR to correlate data in different ways.
While process mining targets end-to-end business processes or parts of them, where different resources work together to achieve the process outcome, task mining focuses on the single task that consists of different actions or steps, such as a mouse click, a keystroke, a data entry, or a desktop operation like copy and paste.
How does Process Mining work?
It's all about event logs. When humans and software robots work with IT systems, their activities are recorded by those systems. Process Mining reads this data and converts it into an event log, then creates visualizations of the end-to-end process, along with insightful analytics.
An event log should contain each step performed during the process (the activity), the time at which the event occurred (the timestamp), and for which instance of the process (the case ID).
Using this event log, algorithms generate a process model that shows the process as it really is - including the timing of each step and all variations in process flow, process deviations and exceptions - bottlenecks, workarounds and inefficient workflows - rather than as it is perceived to be. Other data science methods can be applied to further enhance this model. The result is then used for process discovery, conformance testing, and process improvement.
The visualization capabilities built into advanced process mining tools help companies focus on what should be optimized, how to do it, and what the return on their efforts will be. Companies are able to immediately understand the impact of proposed process changes or automation - including the cost saved and effort required. They can easily build and prioritize their automation pipeline or process optimization efforts.
Advanced process mining solutions continuously monitor and measure results, so companies immediately know if they are on track or off base. They also get a complete audit trail for compliance purposes.
Mining Algorithms
The main component in process mining is the mining algorithm. It determines how the process models are created. There is a wide variety of mining algorithms. The three most popular categories are.
- Deterministic Algorithms
- Heuristic Algorithms
- Genetic Algorithms
Determinism means that an algorithm delivers only defined and reproducible results. It always delivers the same result for the same input. The deterministic algorithm was one of the first algorithms that is able to handle concurrency. It takes an event log as input and calculates the order relation of the events contained in the log.
Heuristic mining also uses deterministic algorithms. However, they refer to the frequency of events and traces to reconstruct a process model. A common problem in process mining is that real-world processes are very complex and their discovery leads to complex models. This complexity can be reduced by neglecting rare paths in the models.
Genetic mining algorithms use an evolutionary approach that mimics the process of natural evolution. They are not deterministic. Genetic mining algorithms follow four steps: Initialization, Selection, Reproduction, and Termination.
The idea behind these algorithms is to generate a random population of process models and find a satisfactory solution. The initial population of process models is randomly generated and may have little in common with the event log. But because of the large number of models in the population, better fitting models are generated through selection and reproduction in each generation.
What are the advantages of Process Mining?
In the past, process mapping had to be done by teams that met for several days to work it out on a whiteboard or spreadsheet. In other words, it was a lot of hard work. But thankfully, process mapping done by humans is a thing of the past. Today, powerful process mining algorithms and advanced data transformation do the job of discovering and optimizing processes quickly, data-driven, completely, and without much effort. That's because the accumulation of data ensures that process analysis cannot be based on gut feelings. Companies need fact-based numbers and evidence to back up their strategies.
Save time and free up work capacity
Traditionally, managing and optimizing business processes is a very labor-intensive area that requires a lot of time from experts in the organizations. The growing demand for efficiency and the ability to clearly present results requires modern technologies. Classic BPM approaches are often overwhelmed here because the data they use to support their assumptions has already changed by the time their situation analysis is complete, which in turn renders the analysis useless. When optimizing processes, the most time-consuming part is data collection. Process mining gives companies a head start by automating the data collection part. The basis for understanding the operational situation in an organization is to understand what is actually happening, not what is assumed to be happening. That's what you find out with process mining.
Finding process bottlenecks
Process bottlenecks are difficult to uncover through BPM and process mapping workshops. People have a gut feeling about what might be wrong or inefficient, but they lack fact-based evidence. They need data to back up their assumptions, and that's where process mining comes to the rescue.
Replacing opinions with facts
One of the main goals of process mining is to see the big picture of business operations and still be able to track down the root causes of deviations, bottlenecks or process variations.
Process mining can be used,
- to scale optimization efforts across multiple business operations and locations and support process control through the use of Data,
- to capture processes anywhere in the enterprise - on a large scale and with little human effort,
- to identify bottlenecks, deviations and inefficient processes that should be reconsidered or automated,
- to continuously monitor and measure improvements,
- to simplify compliance, with complete audit trails,
- to provide a full context and end-to-end perspective needed to improve processes,
- to identify the most valuable and effective processes for the use of automation.
Important KPIs at a glance
Process Mining uses advanced algorithms to bring visibility into current business processes, helping companies streamline and improve them. It quickly uncovers valuable insights that can improve productivity and ultimately illuminates opportunities in core business processes that will have the greatest impact on customers, management and the bottom line.
Thus, process mining can be used to examine three main types of key performance indicators (KPIs):
- Time KPIs: How long does it take to complete a certain process?
- Cost KPIs: How much does it cost to complete a particular process?
- Quality KPIs: Does the result of the process meet the defined criteria?
Process mining has a key advantage over traditional as-is analysis: it can access real-time event data. Moreover, it also looks at historical data and is able to closely examine a range of event logs to gain a deep understanding of what is happening - a stark contrast to the slow and manual heavy lifting of data infrastructure previously used to perform the same calculations. Rather than relying on traditional data infrastructure to analyze transactions, process mining can surface what is happening. To do this, it leverages the vast amounts of event data from all of their systems to
- Discover the actual behavior of people, organizations, and machines and compare it to existing models,
- Correlate millions of events to show how reality differs from perceptions, opinions, and beliefs,
- Provide a foundation for continuous improvement and building more effective business operations,
- Understand the current state of systems and business processes while providing a faster, more granular way to identify deviations and misalignments - and then course-correct.
For example, let's look at the processing of requests in an HR department: 80% of these requests are processed quickly and smoothly, but in 20% of them the process is disrupted. Through the KPI's we know that something is wrong with 20% of the requests, however, we do not recognize the cause. Process mining allows us to understand exactly where the flow is being disrupted. So we can find out that in 20% of the requests an expertise had to be obtained, which delayed the flow.
Let's imagine a KPI like a thermometer, for example: We get a value, but we don't know how it was determined in detail. Process mining, on the other hand, creates an X-ray image that can be used to work out the exact causes. Process mining enables objective "data driven analytics", with which processes in companies can be controlled, optimized and ultimately automated.
Process mining makes work processes transparent
Sometimes the work just doesn't run and the basic problem remains a mystery - we certainly all know this feeling all too well. But help is at hand: Instead of spending hours searching through the breadcrumbs of the process landscape, the method of so-called process mining can be used for such cases.
Here, real work processes are compared with theory, which should lead to better transparency as well as insight into the processes. But why is this necessary at all? Well, unfortunately, reality rarely corresponds to the optimal conception. Let's take the classic beach vacation as an example: Mostly influenced by the promo pictures, we imagine empty and beautiful beaches, which are then completely overcrowded upon arrival or have razor-sharp stones under the rather dark brown sea surface. It is the same with corporate processes as in this example. When an employee describes his or her process, it often appears uncomplicated and clear. However, if you look at this process in detail, you will notice that the real process is much more complex.
Where does the difference between a real and an ideal conception of business processes come from?
The human factor
As is so often the case, people represent the greatest potential for error - and this is no exception. In the following, we list the main causes for the large difference between the assumed by employees and the actual reality of business processes:
Perception differences
Employees usually do not go through a process as it is optimally intended. They often spend time repeating certain work steps or incorporate additional steps that are not actually intended in the process described.
Subjectivity
Everyone has a subjective picture of a workflow, depending on their role and perspective. This is one of the reasons why it is so difficult to discover the "as-is" process in a classic workshop or interview-based setting.
Partial view
Specifically with processes, there is the added challenge that there is not a single person performing the entire process. Instead, multiple people, often multiple teams, departments or even companies work together to deliver the final product or service to the customer.
Exceptions confirm the rule
Furthermore, exceptions play an important role in certain process flows. Not all processes follow the so-called happy path, in which no special features are found in the process flow that could complicate it.
Abbreviations in the process chain
In order to reach a certain goal more quickly, employees often use surreptitious routes or certain shortcuts. Process steps are therefore not executed in their entirety, but only partially or not at all. The route provided for in the process described is thus frequently abandoned and the process is not mapped in reality.
Individual practices and opinions
As already described, process steps are not always completed by employees in the same order. Likewise, certain sub-functions are weighted differently and different problems are identified. Here, among other things, the so-called brown paper sessions should be mentioned, which try to combine the different opinions and approaches similar to a mind map - not always with success. This is how the difference between the considered process and the reality arises.
Lack of overall view of the process chain
In addition, employees often only have insights into certain sub-processes. They do not know in detail which processes precede or follow their work step. This often leads to a lack of an overall view of the process chain and ultimately to sources of error that are difficult to identify.
Processes change
Processes are not a rigid construct, but change continuously. They are readapted for customers, must comply with new laws, or are reorganized due to reorganizations. These adaptations are occasionally not recorded in the process documentation, which ultimately makes it a potential source of errors.
Lack of transparency
The processes in the digitized world often seem invisible and not really tangible to us. Understanding how they work remains elusive to many employees. IT processes in particular are very difficult to understand, which is why insight into the real process flow is necessary. Only when the current process is clear and transparent, and contains all repetitive work steps, special features and sneak paths, can process improvement be considered. And process mining helps to achieve this insight.
The Process Mining Architect
The Process Mining Architect plays an important role in this. The classical Business Process Management - as we know it - gets effective support. The professional field of the Business Process Manager is extended to the Process Mining Architect. This extension not only simplifies and accelerates the documentation of processes, but also the optimization, controlling and general management of business processes.
The tasks of a Process Mining Architect:
Process recognition
The Process Mining Architect has the task of visualizing processes on the basis of data. This model is usually generated automatically from event logs and represents the processes in real terms.
Conformity check
The actual state of a process is compared with a model of the target state of the identical process. This visualizes omitted or unnecessarily repeated processes. Likewise, a simultaneous compliance check takes place.
Model improvement
The model reveals optimization potential within the overall process. Identified potentials, such as bottlenecks or unplanned process sequences, are analyzed and identified for improvement.
How To Start a Process Mining Project
To get an initiative rolling, the pain points must first be identified and the data determined. The starting point for any Process Mining project is process analysis, which takes a close look at the current state of business processes, highlights shortcomings and identifies opportunities for improvement.
A proven method to elicit the value of Process Mining to the business involves the following steps:
Determine problem
Identify the problem of importance to the business that can realistically be addressed with process mining. Determine the business value of solving the problem and what metrics to use to measure success.
Identify the data
Identify the data sources that need to be fully understood to address the business process issues under consideration. Identify the applications and systems that need to provide feeds of event data for continuous visibility into end-to-end processes.
Setting up a pilot project
Set up a pilot project to prove the potential value of a process mining solution. Ensure that the project can be executed relatively quickly and deliver specific, measurable results that everyone in the organization can understand.
Accept truth
Accepting the results of the analysis, as Process Mining provides, among other things, a clear picture based on facts. Although challenging, process mining can be just the right tool to validate the uncomfortable realities of the business process environment, and also a constructive springboard to ask the right questions that fix the problems.
In addition, it is important to realize that process mining is more than just a tool. It is a paradigm shift that requires skilled administrators to discover and fix problems. The technology creates the basis for a dialogue with the rest of the organization to comprehensively and objectively address ongoing, systemic process problems that have hindered productivity and effectiveness.
Process Mining in Digital Transformation
Process Mining is regularly used as part of larger digital transformation efforts because it can provide objective, data-driven insights into the core of delays and inefficiencies within business processes. At the same time, it gives organizations the clear insights needed to make process improvements so that systems run faster, smoother and leaner.
Thus, Process Mining can help prioritize the most value-added opportunities for digital transformation and evaluate whether transformation efforts have actually delivered the desired benefits. Process Mining becomes an invaluable tool for maximizing the return on investment in digital transformation initiatives.
Process Mining can, among other things.
- Create transparency, speed up process times, and increase productivity, which affects the bottom line.
- Identify bottlenecks in business processes as well as detect undesirable developments and pinpoint the biggest time wasters.
- provide better and more accurate insights that can translate into higher quality results.
- Provide more up-to-date monitoring that allows risks to be drastically reduced by addressing issues as they arise.
What are typical use cases for process mining?
Process mining is beneficial for many situations in large organizations. Areas where process mining can be actively applied include.
- The automation - This is about understanding the actual processes, variations and opportunities to be successful in RPA Projects.
- The reporting of full process KPIs and dashboards for a given process,
- The Digital Transformation to understand the "big picture" - how companies work, which priorities need to be set and which need to be transformed,
- Auditing and compliance to promote adherence to agreed-upon procedures, rules and regulations,
- Process improvement to identify bottlenecks and other symptoms of inefficient processes,
- IT and ERP development to support ERP consolidation, new deployments and major version upgrades.
Use of process mining in specific industries
Process mining can be used for business process management and process improvement in any application in any industry, especially financial services, telecommunications, healthcare, and retail - industries with big data that can be used as a foundation and where deviations of processes from their intended behavior can have costly consequences.
Production
In the manufacturing industry, timely and accurate delivery to a customer is the ultimate goal. When a company has multiple factories in different regions, there are usually differences between the reliability of deliveries. It's fairly easy to see that they exist, but it's more difficult to understand exactly where or why they are happening. Process mining can be used to compare the performance of different locations down to individual process steps, including duration, cost, the person performing the step. All event data available in the systems is suitable for use. In this way, facts can be brought to the table without debate.
Banking and finance
In the financial sector, it is important to comply with rules and regulations and to be able to provide evidence of this. By using the event data from the systems, individual cases can also be visualized as a process flow. It can be shown how often deviations occur and what the reason for this non-compliance was.
Telecommunications
Telecommunications is a highly competitive sector worldwide. The ability to improve operational processes is key to success and profitability. Process mining helps telecom companies gain visibility into geographically dispersed operations, identify bottlenecks, and ensure that customers receive products and services on time.
Sale
The success of retail companies is based on efficient business operations. Logistics, warehousing, forecasting, order management, and supplier management are the foundation for an excellent customer experience throughout the customer lifecycle. Process mining provides visibility into all of these interconnected processes and offers an understanding of bottlenecks and faulty interfaces. Data-driven fact-based process mining results focus development efforts on the areas that matter most and deliver the highest business results.
Services
The business logic of service companies is to achieve higher operational excellence, i.e. lower costs, for the processes outsourced to them than their customers. Process mining is an important tool for improving the efficiency of a service company by ensuring harmonized processes and finding causes for process problems and inefficiencies.
Healthcare
As data about patient experiences and outcomes increases, so do the risks around maintaining population health and individual patient journey outcomes. For healthcare organizations facing an exponential increase in data, process mining helps deliver efficient and high-quality end-to-end patient journeys, from before the first doctor visit, through treatment, to completed treatment cases.
How do you choose the best process mining software?
To select a good Process Mining Software, a solution should be characterized by three features
- Process Detection
- Conformity Test
- Performance Analysis and Improvement
First, a process mining solution should have strong discovery capabilities, i.e., it should be able to search event logs to track what employees are actually doing, and then build an appropriate process model by generating process maps of the entire business flow of the.
In addition, the solution should have robust conformance checking that analyzes event logs to ensure that actions match process models.
And third, a process mining solution needs performance analysis and improvement capabilities that analyze potential inefficiencies within an event log to determine if and how they can be improved, and then make improvements based on real process data.
Which software is ultimately the right one depends on the size of the company, business requirements and goals. Nevertheless, the solution should include key features that provide the opportunity,
- Identify bottlenecks as well as opportunities for process optimization,
- Gain insight into failed process steps,
- Get an end-to-end view of the entire business process,
- Perform continuous business process monitoring in real time,
- Monitor key performance indicators in real time,
- To be able to perform data cleansing,
- Perform compliance review and gap analysis,
- Improve process models,
- Correlate data across heterogeneous systems.
A company's ability to measure, monitor and optimize business processes has a direct impact on revenue and customer satisfaction. That's why companies should proceed with caution when selecting a process mining solution that best meets all of their business objectives, and consult an expert if necessary.
Conclusion
Process Mining is a holistic, bottom-up approach to process excellence that builds on the principles of continuous improvement such as Six Sigma while advancing the technologies used to achieve it, such as AI, automation, and BI analytics.
In process mining, event data in the company's IT systems (such as ERP, CRM, and BPM) is used to gain insights into the company's business processes. Insights are provided by automatically visualizing data with process flow diagrams and creating analytics that provide information on required improvements and deepen the understanding of what is happening in the business processes. That is why Process Mining is now an indispensable component of any Automation and Digitization Strategy.
.png)