Green status, wrong data: The pipeline monitoring gap IT Ops needs to close

Green status, wrong data: The pipeline monitoring gap IT Ops needs to close

It’s 7:14 AM, and Finance has already filed a ticket. The overnight reconciliation report is missing data. You open three tools and work through the list: 

✔ The Apache Airflow DAG completed

✔  The Snowflake load finished on schedule

✔  The ERP batch job ran without error

Every status is green. An hour later, after pulling logs and looping in data engineers and the ERP team, the root cause surfaces: a batch window collision. Three days earlier, someone on the data engineering team rescheduled a Snowflake data transformation job without flagging the downstream dependency. By the time the transformation finished, the ERP ingestion window had already opened and run on whatever data happened to be there.

No tool failed. No alert fired. The business simply didn’t get what it needed.

Here’s what should bother you about this: every tool in the chain did its job correctly. The Airflow DAG ran exactly as designed. Snowflake processed its workload on schedule. The ERP batch process completed without error. The failure didn’t happen inside any of these systems. It happened in the space between them — the business process handoff that none of them was built to own.

Who’s accountable for the execution chain from system to system?

This isn’t a tooling failure, and it isn’t a criticism of any individual platform. Airflow is excellent at orchestrating data engineering workflows. Snowflake is excellent at processing analytical workloads. ERP schedulers are excellent at managing batch execution. Each of these tools does exactly what it was designed to do, within its own domain.

But the metric your leadership actually cares about — Did the business get trusted data on time? — isn’t tracked by any of them. There’s no shared dependency model spanning the end-to-end workflow, no common SLA tying the outputs of one system to the inputs of the next and no unified data pipeline monitoring capability that measures data flow against business deadlines rather than technical job completion. Each tool’s definition of “done” stops at its own boundary.

A 2025 IBM Institute for Business Value study of 1,700 chief data officers found that poor data quality often goes unnoticed precisely because its impact doesn’t surface at the point of failure. It appears downstream as an incorrect decision, lost revenue, process delays and compliance exposure, long after the root cause has propagated.

If that finding describes your Monday morning with uncomfortable precision, will you keep treating these episodes as isolated incidents, or will you recognize them as an architectural gap in how the end-to-end business process is governed?

The postmortem cycle you can’t break

By the time Finance files the ticket, the failure has already propagated. The reconciliation report is wrong, the financial close cycle may have started on incomplete data and your team is in recovery mode, conducting root-cause analysis on a problem that occurred hours earlier. Mean time-to-resolution (MTTR) starts from when the business notices, not when the job failed. The time between those milestones is typically measured in hours.

What makes this pattern so persistent is that it never quite presents as a systems failure. It looks like a process coordination issue, easily slipping between the various team-to-team cracks. And it gets addressed in a postmortem, assigned to a working group and recurs two months later with a marginally different trigger: a different rescheduled job, a different batch window, a different team that lacked awareness of the downstream dependency.

You’ve seen this cycle. The postmortem identifies “improved cross-team communication” as the fix. The action item is a shared calendar or a Slack channel. It holds for six weeks. Then someone new joins the data engineering team, a batch window shifts by 30 minutes or a schema change propagates without notification, and nobody updates the tribal knowledge that was holding the chain together.

The structural cause is that no one owns the end-to-end business process that spans these tools. It’s never truly resolved because it’s rarely named as the problem. Instead, it gets filed under “communication,” which is a diplomatic way of saying “we have no dependency model across the business workflow, and we’re substituting human memory for architecture.”

Two chains, same invisible failure

The handoff problem surfaces differently depending on where it hits. Two scenarios illustrate how much ground it covers.

  1. In a B2B edge-to-core flow, a supplier file arrives at the network edge via a secure file transfer gateway. From there, the file requires transformation, schema validation and ingestion into the ERP as part of a larger supply chain or financial process. Each step runs on a different schedule, owned by a different team, with no shared data lineage connecting receipt to posting. When the supplier file lands two hours late, the failure is silent until inventory counts are off, an invoice doesn’t post or a supply chain decision gets made on data that hasn’t fully arrived. IT owns the latency, even though the failure happened in a gap nobody was monitoring.
  2. In a financial close scenario, period-end close depends on outputs from cloud data platforms like Snowflake, Databricks or a cloud data warehouse service, feeding into the ERP’s record-to-report process. When data engineering reschedules a transformation job without flagging the downstream ERP dependency, the batch window executes, the job shows green and Finance pulls the morning pack to find numbers that don’t reconcile. The data freshness issue doesn’t surface until the business is already operating on compromised figures.

In both cases, every individual tool performed correctly within its own domain. The failure resided in the business process that depended on their outputs arriving in the right sequence, at the right time, for the right downstream system.

The reflex that keeps you stuck

When the same failure recurs, the instinct is to add more observability: more alerts, more status feeds, another real-time dashboard layered on top of existing tooling. More signal gets you to the problem faster, but it doesn’t change the fact that the business found the problem first.

This is where most IT Ops leaders get trapped. You’re optimizing for faster reaction when the real leverage is in eliminating the category of failure entirely. The absence in most hybrid environments isn’t signal, but scope

Each tool produces comprehensive telemetry about its own execution. What’s missing is visibility across the end-to-end business process that depends on those tools. SLA management is tied to business outcomes rather than individual job completion, and dependency mapping spans the full workflow from data platform outputs through ERP ingestion to downstream business action.

That’s the layer a business service orchestration platform like RunMyJobs by Redwood occupies. It doesn’t replace Airflow, Snowflake, ERP schedulers or any other domain-specific tool. Each continues to do what it does best. RunMyJobs integrates their inputs and outputs into the broader business workflow, applying SLA monitoring, compliance, security and dependency governance across the full chain. It’s the technology-agnostic orchestration layer for the business process that currently has no owner.

Visibility starts where the tools stop

The hybrid application and data technology estate won’t consolidate or become less complex on its own. Innovation demands that new applications be built and new platforms be adopted. With this inevitable expansion comes the open-source data pipeline orchestrator, application-native scheduler, cloud service tooling, ERP batch process and trading partner file transfer workflows. This is the environment today — and for the foreseeable future. 

Come back to the 7:14 AM ticket. If you had RunMyJobs running that business service, the ticket would have never happened. Neither would the multi-team incident response, the protracted root-cause analysis and all the finger-pointing and frustration these types of failures create. RunMyJobs would have monitored, identified and automatically remediated the situation, and the business service would have been delivered on time, without error. 

Closing this architectural gap requires a strategic decision on application and data orchestration. It means looking at your automation silos and deciding to consolidate mission-critical business application outcomes onto a platform that can accelerate your transformation at the lowest possible total cost of ownership (TCO). That’s not a monitoring upgrade, an amended cross-team communication process or another dashboard that doesn’t address the underlying problem. It’s a different and vastly superior operating model. 

See how RunMyJobs connects hybrid data pipelines into a single governed execution layer.

Get Peace Of Mind with SmartThings Safe Premium

Get Peace Of Mind with SmartThings Safe Premium

SmartThings Safe Premium is powered by Arlo, a third-party partner that operates the professional monitoring center and coordinates emergency dispatch on our behalf. Feeling safe, whether at home or on the go, shouldn’t take more than a tap. With SmartThings Safe Premium, users can instantly send help requests, from inside the SmartThings app, to Arlo-powered […]

The post Get Peace Of Mind with SmartThings Safe Premium appeared first on SmartThings Blog.

Scheduling in modern SAP landscapes: SAP BTP Job Scheduling Service, Application Jobs and when to orchestrate 

Scheduling in modern SAP landscapes: SAP BTP Job Scheduling Service, Application Jobs and when to orchestrate 

SAP landscapes have become considerably more complex than the tools most teams use to automate and optimize how heavy backend workloads run. 

A decade ago, background job scheduling was a contained problem. SM36 and SM37 handled ERP processing, local schedulers covered the rest and the architecture those tools were built for was largely self-contained.

That architecture is gone for most enterprises. A single business process now routinely spans SAP S/4HANA, SAP Cloud ERP, SAP Business Technology Platform (BTP), SAP Business Data Cloud (BDC), SAP Analytics Cloud, Databricks, Snowflake, AWS … and the list goes on … before producing a result anyone can act on. Each platform executes reliably within its own boundary. Whether the process completes reliably across all of them is a problem local scheduling can’t solve.

To answer the question, “How does this job impact the entire business chain?” SAP architects must differentiate between simple task scheduling and true enterprise orchestration.

When local scheduling remains the right call

Native scheduling tools are legitimate architectural choices, not placeholders until something better comes along. For isolated, self-contained tasks with no upstream dependencies and no downstream impact — and where advanced or variable calendaring handles any scheduling complexity — they’re often exactly what the job requires.

Across a modern SAP estate, that means:

  • SM36 and SM37 for standard background processing in SAP S/4HANA and SAP ECC: batch jobs, reports and ABAP programs that live entirely within a single ERP instance and don’t depend on external system state
  • SAP BTP Job Scheduling Service for Cloud Application Programming (CAP) model applications running on SAP BTP Cloud Foundry: time-based or one-time job execution for CAP services that operate independently within the SAP BTP environment
  • Application Jobs for RESTful ABAP Programming (RAP) model extensions on SAP S/4HANA: the metadata-driven, Fiori-integrated successor to SM36/SM37 for modern ABAP development, appropriate when the job scope stays within the SAP S/4HANA system boundary
  • Native scheduling within SAP Integrated Business Planning (IBP), SAP SuccessFactors and SAP Digital Manufacturing: purpose-built schedulers for localized workloads inside those solutions, where cross-system coordination isn’t required
  • Built-in schedulers in Snowflake and Databricks for internal scripts and transformations that begin and end within those platforms

The common thread is isolation. Each of these tools executes reliably when the task it governs doesn’t depend on the state of another system and doesn’t need to trigger work in one.

Trigger points: When traditional scheduling becomes operational risk

Three conditions reliably signal that native scheduling is no longer sufficient.

  1. The job becomes a link in a cross-system chain. As soon as a task has an upstream dependency outside its own platform or triggers a downstream action in another system, local schedulers start operating on assumptions. For a material requirements planning (MRP) run that depends on an external warehouse feed completing first, a Datasphere refresh that needs to finish before SAP Analytics Cloud models publish or a CAP service on SAP BTP that should only execute after an upstream SAP S/4HANA job confirms success, native schedulers in each of those systems can only confirm that their own job ran. None of them can verify the state of the system that the next step depends on.
  2. You need resource-aware, resilient execution. High-concurrency workloads like SAP Datasphere task chains, Databricks pipeline runs and overnight batch sequences across multiple platforms require load balancing, automated retry logic and intelligent error handling. Without a coordination layer, a single failure can cascade across dependent processes before anyone is notified.
  3. Your roadmap involves eliminating operational blind spots. RISE with SAP and clean core transformations distribute process logic across SAP BTP services, cloud extensions and non-SAP platforms. That distribution improves architectural flexibility and creates a new operational problem: no single view of what’s running, what’s waiting and what failed across the landscape. Each platform reports on itself, but nothing reports on the process.

Governed execution in practice

RunMyJobs by Redwood is designed to take over at the boundary where native scheduling capabilities end. Rather than replacing SM36/SM37, Application Jobs or the SAP BTP Job Scheduling Service, it coordinates them — along with every other system in the landscape — into governed, end-to-end process flows. Workflows execute based on verified event completion and real-time system state, where a downstream process starts because the upstream transformation succeeded, not because a timer reached a predetermined hour. 

The difference a dependency makes

Scheduling Orchestration
Runs isolated tasks Coordinates end-to-end business processes
Time-based execution Event-driven, latency-aware execution
Local system visibility Cross-platform visibility
Static, assumed dependencies Dynamic, verified dependencies
Reactive troubleshooting Centralized operational intelligence

RunMyJobs’ out-of-the-box connectors span SAP S/4HANA, SAP BTP, SAP BDC, SAP Datasphere, SAP Analytics Cloud, SAP Cloud ALM, SAP Build Process Automation, Databricks, Snowflake, AWS and ServiceNow, making cross-platform dependency management possible without custom code. 

For RISE with SAP and SAP Cloud ERP environments, RunMyJobs connects through its Secure Gateway and agentless architecture, consistent with how SAP Cloud Connector operates. It’s the only workload automation and orchestration platform included in the RISE with SAP reference architecture. With no agents inside the ERP and no custom ABAP, RunMyJobs supports clean core principles from the start. 

Coverage extends across the full range of modern SAP development models: Application Jobs, RAP-based extensions, CAP services on SAP BTP, SAP Datasphere pipelines, SAP Integration Suite workflows and AI-driven scenarios involving Joule.

Job requirements decision tree

Blog diagram v4

Build the control layer before you need it

Most teams introduce enterprise orchestration after complexity has already compounded — after the first SLA breach that took half a day to trace, the planning run that produced results on incomplete data or the migration that left three separate scheduling tools running in parallel with no consolidated view across them.

Getting ahead of that inflection point is considerably easier than resolving it afterward. Teams that introduce RunMyJobs early in their RISE with SAP journey retire legacy schedulers, establish governed dependency management and build a control layer that scales with the architecture from the start.

SAP’s native tools still do what they were designed to do, while orchestration governs what happens between them.

Get a demo of RunMyJobs, and explore its full range of SAP connectors to see how enterprise orchestration extends across SAP S/4HANA, SAP BTP, SAP BDC and your broader cloud landscape. 

Care Pathway Automation in Practice:  Lessons from Helsinki University Hospital’s Cancer Care Implementation

Care Pathway Automation in Practice: Lessons from Helsinki University Hospital’s Cancer Care Implementation

4.6.2026  Care Pathway Automation in Practice: Lessons from Helsinki University Hospital’s Cancer Care Implementation.
This article is a reflection on a Presentation by Finland’s Largest Hospital District, HUS: Results and Future Opportunities of Care Pathway Automation. Author Juha Nieminen is Global Head of Healthcare at Digital Workforce and a member of the executive leadership team.


At a recent healthcare IT event in Helsinki, HUS representatives Administrative Chief Physician Meri Utriainen and Planning Specialist Johanna Pakarinen shared their experiences of automating end-to-end care pathways. The case example focused on a breast cancer follow-up solution, for which Digital Workforce serves as the contracted supplier.

When a customer shares two years of production experience, it is worth listening carefully. I was particularly interested in three things: the tangible results achieved through automation, the unexpected effects of implementation, and the extent to which HUS believes this operating model can be applied to other care pathways. In this article, I reflect on the key observations and lessons I took away from the presentation.

The presentation examined HUS’s breast cancer follow-up solution from three perspectives:

• the initial challenge
• measured production outcomes
• scalability and potential use cases

Challenge: The Administrative Burden of Care Pathways

Breast cancer follow-up is HUS’s largest long-term post-treatment monitoring programme. After completing active treatment, thousands of patients remain under specialist follow-up, with monitoring programmes that can extend for up to ten years.

The follow-up pathway consists of recurring activities such as imaging, laboratory tests, outpatient appointments, symptom assessments, and patient communications. Managing these activities across a large patient population requires extensive coordination, creating a significant administrative burden and increasing the risk of delays and backlogs. HUS openly described how the COVID-19 pandemic further highlighted the need for better operational control, forecasting, and resource management.

From the patient perspective, follow-up should be timely and predictable. For clinicians and care teams, however, delivering that experience requires extensive coordination, communication, and administrative effort. Once again, this case highlights that a significant portion of healthcare’s productivity challenge stems not from clinical decision-making, but from orchestrating the fragmented workflows, communications, and administrative activities that enable care delivery.

How The Problem Was Addressed

HUS sought to shift towards a model in which the entire care pathway is designed upfront, with automation orchestrating its execution and escalating to clinicians only when clinical input or decision-making is required.
In addition, there was a clear ambition to give patients greater involvement in their own care planning by enabling self-service appointment booking and allowing them to choose their follow-up approach—either symptom-driven or scheduled routine visits.

Measured Impact and Results

HUS replaced a manual operating model with a single configurable care pathway process that orchestrates patient flow and automates administrative tasks.

The solution has been in production for two years. Here are the key figures shared by Meri and Johanna:

  • 6,909 patients on an automated care pathway
  • 95% of all manual tasks in patient follow-up automated
  • 52% of patients chose symptom-based follow-up, leading to a significant reduction in nursing visit volumes annually
  • 47% reduction in inbound calls
  • Over a three-week measurement period, automation executed 6,768 tasks, with only 1.2% escalated to clinicians

A particularly notable finding relates to the reduction in inbound calls. HUS had expected demand to increase as outpatient visits decreased, based on the assumption that patient uncertainty would grow. However, the opposite occurred. When follow-up is timely and patients have clarity on what will happen next, the need for additional contact is significantly reduced.

For patients, care pathway automation is experienced as timely communication, self-service appointment booking, and selection of follow-up mode. Communication channels remain unchanged—automation executes tasks through HUS-defined channels and can also use traditional channels such as letters where necessary.

The main value of care pathway solutions lies in reduced waiting times and delays, more reliable and timely follow-up, and enabling clinicians to focus more on patients requiring urgent clinical attention.

Scaling The Impact

Although the results presented by Meri and Johanna were impressive, a more interesting question is how widely the same operating model can be applied across other care pathways. In long-term patient monitoring, similar structural patterns tend to repeat, suggesting that HUS’s experience is not limited to a single patient group.

The presentation also identified several emerging use cases, including medication monitoring in dermatology and neurology, imaging-based follow-up in other cancer types, and monitoring of genetic risk carriers and meningioma patients. Further opportunities were highlighted in care coordination between specialist and primary care, such as secondary prevention of coronary artery disease events.

Based on HUS’s experience-based estimates, the scalability potential of the model is significant:

• Over 95% of suitable patient flows can be transitioned to automated pathways
• Over 95% of tasks within these pathways can be handled by automation
• Over 95% of imaging findings are classified as non-actionable and do not require intervention
• Approximately 50% of patients prefer symptom-based contact over scheduled follow-ups

Surprises and Key Learnings

At the end of the session, Johanna and Meri reflected on key lessons from the breast cancer follow-up implementation. Three particularly important insights stood out to me:

“One directive, ten years” framework

Replacing periodic decision-making with a single configurable workflow is key to scalability. The care pathway is implemented as a core template, with variations defined through parameters for different diseases, patient groups, and care plans.

47% reduction in inbound calls as an unexpected outcome

Healthcare automation initiatives are often justified by cost savings and efficiency gains. However, the most significant benefits are frequently those that cannot be predicted in advance. In this case, freed capacity was greater than expected, and more focus on change management could have improved early utilisation of that capacity.

“No rocket if a bicycle is enough”

The breast cancer solution is based on algorithm-driven process automation, not AI. It does not make clinical decisions and is not a medical device; instead, it orchestrates and automates scheduling and administrative tasks, while also managing work coordination in a single seamless flow.

A key lesson is that AI should not be used where simpler automation is sufficient. Instead, it should be applied where it adds real value. HUS identified applicable areas such as document processing, imaging, structured data capture, and referral handling.

HUS is also quite advanced in this area. Digital Workforce has been involved in HUS’s AI-based referral triage solution, which processes and classifies more than 300,000 specialist care referrals annually.

The key principle is simple: first design the process, then select the most appropriate technology for each step. In many cases, the best outcome is achieved through a combination of automation and AI, supported by strong orchestration of the overall system.

Summary

After leaving the session, I reflected on how much of healthcare’s productivity challenge is still driven by fragmented processes, limited coordination, and the burden of communication and administrative work. HUS’s experience shows that these tasks can be extensively automated without shifting clinical decision-making to technology.

As clinicians’ time is freed for clinical work and care pathways become more transparent, predictable, and timely, the benefits extend to patients, professionals, and organisations alike. In my view, transforming care pathways by leveraging automation and advanced process orchestration to create seamless workflows that deliver the core objective—better care and better outcomes at lower cost—is set to become one of the key development directions in healthcare in the coming years.

HUS’s example is compelling: two years in production, 6,909 patients on an automated pathway, and 6,768 tasks in three weeks—only 1.2% of which required manual intervention. While these results are significant for a single patient group, the real impact lies in the scalability of the model across other pathways and organisations.

Key Learnings:

  • Automation delivers the greatest value at the level of orchestrating entire care pathways
  • The most significant benefits are not always predictable in advance
  • Change management is as important as the technology itself
  • Many care pathways share a common underlying process logic, enabling solutions to be scaled
  • AI is not a universal solution; automation and AI each have their strengths and can be used together. The starting point should always be clear process design

Author: Juha Nieminen is Global Head of Healthcare at Digital Workforce and a member of the executive leadership team. He has over two decades of experience in sales leadership and business development across healthcare, IT, and other industries. In his current role, he focuses on healthcare process automation and care pathway solutions. He holds a Master of Science in Engineering (Industrial Engineering and Management).

The post Care Pathway Automation in Practice: Lessons from Helsinki University Hospital’s Cancer Care Implementation appeared first on Digital Workforce.