Executive Summary
DHIS2 is increasingly used by Ministries of Health and their stakeholders to collect, manage, and analyze data for global health programs. While DHIS2 has powerful and flexible analytical capabilities, the system has limitations when it comes to doing complex calculations needed to implement, measure, and support decision-making for health programs. Governments and their partners currently use external tools to implement their performance monitoring and M&E frameworks that cannot be included in DHIS2. Enabling these more advanced computation and analytics functions to be performed from within DHIS2 would help streamline program management processes, utilize outputs generated from those equations within DHIS2, and increase availability and accessibility of that data to countries, donors and other partners.
ORBF is an open-source rules engine developed by Bluesquare to enhance DHIS2’s computational capabilities to support calculations for M&E and the allocation of performance-based incentives. It is interoperable with DHIS2 and currently used in eight countries (specifically Cameroon, Senegal, Malawi, Liberia, Uganda, DRC, Nigeria and Zimbabwe). However, ORBF is a standalone system that connects to DHIS2 via the API. The configuration of the rules and mapping to DHIS2 metadata is completed within the application. Despite ORBF being open source, the awareness and accessibility of the tool to the greater DHIS2 community of users remains limited, since it is a separate platform.
To enable broader community use of the tool, we propose to make the ORBF administration panel a DHIS2 app so that users can download it from the DHIS2 app store and use it to configure and manage rules and computations directly within DHIS2. This will make ORBF accessible from any DHIS2 instance and would actively draw on the DHIS2 community’s feedback to make future improvements. Additionally, digital square’s investment in ORBF will contribute to improving the user interface for designing rules, making program monitoring within DHIS2 easier for managers.
Consortium Team
Bluesquare is a technology company focused on universal health coverage. We build a set of tools that enhance the capabilities of DHIS2 to support governments and their partners to implement programs within DHIS2. Bluesquare has implemented data systems for governments and their partners in 24 countries. Bluesquare first launched ORBF in 2017 to support the calculations for incentive payments for RBF programs, but has since extended the capabilities to support broader M&E frameworks, based on project needs.
In addition, from a pure development perspective, Bluesquare is proud of its strong team of agile developers. They have broad experience in software development both in Bluesquare’s own products and with other custom products (i.e. we have a team working on a sleeping sickness elimination data platform). All of our developers have strong experience in interoperability of global public goods software. And at the heart of it all, the products we work with are all interoperable with DHIS2. From tools that link ODK to DHIS2 to a Data Visualisation tool enriching DHIS2.
Bluesquare will be the Prime. The company will lead the software engineering and create the documentation, as well as manage the project. Bluesquare’s team includes is comprised of software engineers, product managers, and project managers. Our team members have extensive backgrounds in technology and public health, and a long track record of developing, implementing, and delivering technology solutions for health systems strengthening. We provide their full CVs in appendix.
Bluesquare will partner with Abt Associates, an experienced program management partner working with DHIS2 systems across projects and countries. The identified partner will provide program management for the project in coordination with Bluesquare as the software developer, to carry out an agile software development, testing, and validation process, and ensure that the app creates value across a range of projects.
Abt Associates is a mission-driven, global leader in research and program implementation in the fields of health, social and environmental policy, and international development. Abt has more than 30 years of health systems strengthening and private sector experience working in more than 50 countries, tackling critical public health challenges in maternal, newborn and child health, family planning and reproductive health, HIV/AIDS, tuberculosis, malaria, and nutrition. Abt’s health systems strengthening work leverages technology solutions for robust monitoring and evaluation (M&E), and to deliver high-impact outcomes. Their team of programmers, developers, and system architects has deep experience in data systems, applications, usability assessments, implementing off the shelf solutions, web development, cybersecurity, e-learning, user experience (UX), quality assurance (QA) testing, and user acceptance testing (UAT). Abt also brings applied experience implementing and customizing DHIS2 solutions, including roll-outs across individual countries and multi-country M&E solutions through DHIS2.
For the DHIS2 App, Abt has relevant experience in developing, testing, and deploying applications. The team has developed more than 150 websites and applications using an agile development process that includes requirements analysis, design, testing and QA, implementation, and maintenance. This experience includes projects for USAID, NIH, and EPA. Abt has also been contracted to redesign or re-architect existing applications, including CDC’s TravWell and HUD’s HDX 2.0.
Related Work
Bluesquare is the developer of ORBF, and has implemented the tool at scale and trained users on the tool in 8 countries. To date, ORBF was developed in close collaboration with our partners to ensure that it meets their needs. Over time, those partnerships continue to play a key role in the iterative evolution of the the application.
Here is a list of the countries and use cases of where ORBF is currently being implemented.
Bluesquare has extensive experience in enhancing the capabilities of DHIS2 by creating tools that are interoperable with DHIS2. Bluesquare has developed and implemented a set of applications that are interoperable with DHIS2 including Data Collect, D2D, and DataViz. These tools are being used at scale in more than 20 countries. In addition, Bluesquare developed and has implemented OpenRBF, an open source Performance Based Financing Management and Data System that is deployed at scale in over 20 countries.
As a part of these and other projects, Bluesquare has extensive experience in setting up and maintaining DHIS2 systems. As a result, Bluesquare frequently encounters and has to overcome the limitations of DHIS2. To overcome those barriers, Bluesquare has implemented a series of open source tools, as well as built custom DHIS2 applications. Here we provide a selection of the DHIS2 applications we have implemented:
An app to control for data quality on historical data in the UNICEF WASH Data system in DRC, connected to the DHIS2 Tracker.
An app to support patient tracking, connected to the DHIS2 tracker for a Kangaroo Mother Care data system in Cameroon.
A facilities management app in the RBF data system in Cameroon.
A reporting app to enable navigation through linked reports for the PBF program in the HMIS in Liberia.
Abt has been conducting health systems strengthening and international M&E for more than 30 years. Technology and digital solutions, particularly DHIS2, have been critically important in providing cost-effective, usable, and timely monitoring to drive impact. Examples of Abt’s work include:
For USAID’s Health Finance and Governance (HFG) project (2012-2018) in 22 countries, Abt built an enterprise monitoring and evaluation system (MandE). The web-based platform features “out of the box” support for common needs like building work plans and indicators, and provides rapid start-up of high-quality M&E. Built-in tools allow non-programmers to design custom forms and indicators to capture each project’s distinctive data collection and reporting needs. MandE tracked more than 700 work plan elements and 100+ indicators involving more than 75 organizations.
For the President’s Malaria Initiative VectorLink project (2017-2022) in 23 countries, Abt is building out a DHIS2 instance for the spray operations and management to provide comprehensive M&E capabilities. A future phase of the project will also pull other data from available DHIS2 instances maintained by the country governments.
For USAID’s Health Systems 20/20 (2006-2012), Abt provided technical assistance to Kenya’s Ministry of Health for their roll-out of DHIS2.
For USAID’s Malawi-funded Support for Service Delivery Integration Systems (2011-2016) Abt provided technical assistance to Malawi to expand its DHIS2 capability and data reporting.
Number of years in operation
Bluesquare has been in operation since its founding in 2012. We are celebrating our 6th year.
Project Description
Problem Statement
Health Program Managers are stretched for resources and time. With more data available, they are looking for more efficient ways to understand their program’s progress. DHIS2 offers a platform to collect, manage, and analyze data in one place. By adding additional computational and analytic features in DHIS2, program managers would be able to make better use of the system for program monitoring and decision-making. Program managers would be able to better use the platform to manage programs and take action based on the data.
ORBF is a web-based, open-source rules engine that supports the computations needed for programs to implement M&E frameworks and the performance-based allocation of resources, such as in performance-based financing. It is a powerful tool that helps program managers better understand performance and where to target their support. It pulls data from DHIS2, runs the data through the algorithms configured in ORBF and sends the outputs back to DHIS2 as discrete data values of data elements. This enables the computed data to be used for analysis in DHIS2. However, ORBF is currently available as a separate application, which creates a barrier to access. ORBF is currently available on Github. There is an opportunity to enhance the accessibility of this tool to reflect the DHIS2 community’s requests for more advanced analytic features.
To make it easier for program managers to more proactively use their data and manage their performance using DHIS2, Bluesquare and Abt Associates propose to make an ORBF app available in the DHIS2 app store and to simplify the process of configuring rules within the app. As a DHIS2 app, ORBF will be a more relevant and sustainable application. It will also enable the DHIS2 community to provide feedback and help make ORBF an even more powerful resource.
Ways that ORBF will help to support data use in DHIS2 include the following:
Monitoring and Evaluation
Compare data between multiple periods (ex. Calculate the difference in performance of a facility between Q1 of 2016 and Q1 of 2017.
Group facilities based on performance (ex. Identify all facilities with at least a 10% increase in antenatal care between Q1 of 2016 and Q1 of 2017)
Conditional “if-then or AND” calculations (ex. If a health facility collects at least 20 patient satisfaction surveys, they they receive 50% of a score, and then if it also demonstrate that they made changes based on the feedback, they receive 100% of a score.)
Multi-part equations that need to be completed sequentially
Tracking progress towards targets
Understanding the relative performance to target resources
Risk-based algorithms (ex. Identifying which facilities need to be monitored most closely based on reporting rates and validity of self-reported data)
Categorizing facilities into high, medium, and low risk groups based on scores
Compute the relative scores of health facilities to distribute incentives within a budget envelope
Technical Approach
To improve ORBF’s accessibility and use within any DHIS2 instance, Bluesquare and its implementing partner will build a DHIS2 app that that connects to ORBF. The frontend for configuring and managing rules will be moved into the ORBF DHIS2 app.
System enhancements that would improve access and usability include:
Simplifying the process of building rules through the DHIS2 interface.
Create documentation for ORBF use, available from within DHIS2.
Create a forum for users to provide feedback on the tool.
Open the product roadmap.
Build an ORBF DHIS2 app
In this model, the whole front end of ORBF would be moved to the new DHIS2 App, allowing the RBF admin to manage its whole system without ever leaving DHIS2. This would be made possible by the creation of a REST API on top of ORBF2.
Configuring Rules within the App
The app will allow users to build complex rules, from within DHIS2, that are otherwise not supported by DHIS2. The ORBF DHIS2 app will allow the following activities (currently completed in ORBF) to be carried out directly from within DHIS2:
Select the DHIS2 metadata that will be used in the rules.
Configure the rules and computations by defining formulas and dependencies.
Define constants to be used in the computations.
Run simulations of the formulas within the application to test computation integrity before creating new metadata in DHIS2 and sending the results to DHIS2.
Select to which org units and org unit groups the computations should be applied.
Auto-generate metadata in DHIS2 to store the outputs of the equations (data elements, data sets, data element groups).
Track metadata created through ORBF for auditing purposes.
Improving the Rules Building UX
In improving the simplicity of using the rules building interface, we aim to offer the best tradeoff between a full rules engine, which would typically require knowledge in software development or highly specialized tools, and the simplicity and clarity of a well designed interface. We want to provide the user with parameterization and clear instructions for how to build rules, along with tools to validate their work at every step in the rules building process. This will to avoid mistakes early in the process, and give the user visibility into the consequences of each change.
How will each change modify the output?
How many org units, and which ones, will be impacted by the change?
This would enable the user to setup a complex system of interacting rules, without losing visibility to the consequences of each step in the computations.
In-application Rules Building Assistance
Providing DHIS2 administrators with a configurable rules engine provides a lot of autonomy, ORBF provides a lot of autonomy and flexibility, but it is also essential that users have a clear understanding of how the tools works to ensure that they are achieving the intended outputs. This is why, in addition to a standard manual, we propose setting-up the following documentation and support for users:
Open access to a live project for people to discover the software.
A wizard-like step-by-step guide to start and setup a new project.
Simulation at every step, enabling the user to run the engine without generating any real data, using randomly generated inputs if needed. This will show the outputs for validation of the rules at every step (so the user understands the consequences of their computations /edits).
Contextual help provided on each screen.
Description of fields displayed for easy reminders.
A visual representation of the decision-trees - the software will provide a visual representation of the rules in the project and how they are linked to each other.
A debug screen where the user can see all the values for every step of the process to be able to understand the source of possible misconfigurations.
Process
The project will be carried out through an agile methodology so that progress is being tested, validated, and iterated on an ongoing basis. This will ensure that partners are delivering the most robust results and they are able to deliver within the proposed timeline.
Use of Digital Health Technologies
ORBF - DHIS2 Interaction
ORBF is a rules engine that is interoperable with DHIS2. A foundational principle of ORBF is to avoid data duplication at all costs. This means that any data (values or metadata) that exist in the DHIS2 should stay there (and only there), and simply be used by ORBF.
To achieve this, ORBF uses the DHIS2 web API. Here is a (non exhaustive) list of data that is pulled from DHIS2:
Capture the health pyramid (Org Units)
Capture groups of facilities (Org Unit Groups)
Capture data set for the various input packages (Data Sets)
Retrieve input values (DataValueSets)
Once the process is done, ORBF again uses the API to send back the data to DHIS2, so that it is saved as discrete values in DHIS2.
In addition, ORBF provides the user with the ability to create the metadata necessary for using and storing the outputs in DHIS2 from within ORBF, and establish naming conventions for organization of data in DHIS2.
Rules construction and Dependencies between formulas
ORBF2 supports a wide array of rules, all able to use input and output from/to DHIS2. This includes, but is not limited to:
Rules using inputs from different periods
Using the past quarter value to generate a target for this quarter,
Generating a quarterly value based on monthly reported values
Rules using other org units as input. Example:
Generating an efficiency indicator compared to other similar org units
Using the price, weight, population defined at parent levels (country, region, district,...)
Distinguish if the value was set or not but default to 0 to allow smooth expression evaluation
Excel like expressions combining
Aggregation (avg, min, max, sum)
Conditional rules (If, Then, AND, OR, score table, safe_div)
Rules based on decision tables
if the value is in a specific region or group, get a specific input for the rule
If a given activity and org unit group, then apply “x” value
Target at different level - some rules are executed at the level of a single Data Element, some can act on a whole Data Set (example: multiplying number of activities by a price for each of them then summing altogether)
Cross reference between expressions of the same level or from the lower/upper levels example :
weighting the indicator obtained points by the total possible points,
spreading a district budget on all org units based on their performance score
Rules are constructed by:
Defining components
Grouping components that will be calculated together into Activities, and linking them to existing DHIS2 metadata
Grouping related Activities to which the same sets of rules will be applied into Packages and attributing qualities to those sets of components.
Generating rules per package, using the different components
Generaging rules that combine packages
Defining the outputs
Linking the outputs to DHIS2 metadata
Timeline
An anticipated timeline of 6 months is needed to build, test, iterate, and validate the software, as well as to create the documentation. Bluesquare will designate an additional three months to improving awareness of the app and building a community, finalizing documentation, and monitoring use of the application for the use cases over time.
While we propose here a prospective planning, our objective is to work in close collaboration with Abt, building on the use cases on an iterative basis. We’ll build the simplest system possible able to solve one of their use case, deploy it and collect their feedback. Once this element is working, we’ll work our way toward the next one. This will ensure that the system is built meet expectations along the way instead of evaluating it only at the end.
Note: The process that we use to work will be more iterative than is possible to reflect in a Gantt Chart. We work closely with our partners to build, test, and deploy the software one feature at a time at every stage to ensure that we are aligned throughout the process. Given the agile approach that we will take, these activities may shift during the implementation.
See here the full roadmap:
https://docs.google.com/spreadsheets/d/1i43DmkEhqEWbMz6YiBwjFnbicuN_WFLtTGt4SYJKDiE/edit?usp=sharing
Phases and Project Deliverables
Phase 1: Deployment of beta application within DHIS2 including simplified parameterization for building rules | Activities: Software development to create the DHIS2 app Software development to move the user interface for rules configuration into the DHIS2 app Parameterization of the rules interface to simplify setting up rules Improving the in-app wizard Update documentation
Deliverables Timeline: Months 1 - 4 Global Maturity Model: Core Indicator Improved | Sub-indicators Improved | Global Utility | Source Code Accessibility Digital Health Interventions | Community Support | User documentation | Software Maturity | Technical Documentation Software Productization Interoperability and Data Accessibility |
|
Phase 2: Iterative testing and updates | Activities: Setup use cases in test environment using ORBF DHIS2 App Multiple Rounds of testing use cases - Abt will setup and test the use case Open Jira Board to public for posting issues and feature requests
Deliverables: ORBF DHIS2 App available for download in DHIS2 app store (Bluesquare) Updated documentation available for download from within ORBF App (Abt and Bluesquare) Open Jira board for posting issues (Abt and Bluesquare)
Timeline: Month 5 - 6
Global Maturity Model: Core Indicator Improved | Sub-indicators Improved | Global Utility | Country Utilization | Community Support | Developer, Contributor, and Implementer Community Engagement Community Governance User Documentation | Software Maturity | Technical Documentation Software Productization |
|
Phase 3: Expanding Awareness and Increasing Community Engagement | Activities: Update and open the product roadmap on Jira Hold a webinar to introduce the app to the DHIS2 community Make video tutorials available from within the ORBF DHIS2 app and on the Bluesquare website Write a blog and share it through various channels and listservs to introduce the ORBF DHIS2 app to the DHIS2 community Update documentation, including a link to the product roadmap, and level of engagement on GitHub
Deliverables: Timeline: Month 7
Global Maturity Model:
Core Indicator Improved | Sub-indicators Improved | Global Utility | Country Utilization | Community Support | Developer, Contributor, and Implementer Community Engagement Community Governance Software Roadmap User Documentation |
|
Phase 4: Monitoring and Evaluating use of the app over time | Activities Ongoing feedback from Abt on implementation in the field Ongoing review of feedback on the public Jira board, prioritization and incorporation of feedback into the product roadmap Updates to the application based on feedback Reaching out to known users to seek feedback
Deliverables: Updated product roadmap (ongoing) Updates to the app, tool, and community engagement requests based on feedback
Timeline: Months 7 - 9
Global Maturity Model: (Ongoing) Core Indicator Improved | Sub-indicators Improved | Global Utility | Country Utilization | Community Support | Developer, Contributor, and Implementer Community Engagement Community Governance Software Roadmap User Documentation | | Technical Documentation Software Productization |
|
Digital Health Atlas:
The following projects have been registered in the Digital Health Atlas:
Cameroon: http://digitalhealthatlas.org/project/CM4ed06188
Zimbabwe: http://digitalhealthatlas.org/project/ZWba1310dc
Sénégal: http://digitalhealthatlas.org/project/SN7c8f44e2
Description for a non-technical audience
The ORBF DHIS2 app will enhance the computational capabilities of DHIS2 to enable strategic purchasers, program managers and M&E staff to better utilize data in DHIS2 for program management, performance tracking, and decision-making.
What this investment from Digital Square will specifically fund?
The Digital Square investment will fund making ORBF easily accessible to a wider audience, and improving the ease of the use of the application. In addition, it will help to improve the awareness of the application by the DHIS2 community, and the ability of all to contribute into the ongoing development of the product to ensure that it evolves in line with the evolving needs of the community.
Community engagement
We recognize the power of the open source community to enhance applications to make them as relevant and usable as possible. To that end, ORBF will seek feedback from the DHIS2 community to evaluate whether the DHIS2 app is meeting the needs of the users, and adapt and build on the tool accordingly.
The consortium will foster and engage with a community o promote the use of the ORBF DHIS2 app, and receive feedback on the use of the app, as well as to raise awareness of the app and its use cases within the community through a multi-pronged approach.
Raising awareness of ORBF
Hold a webinar for the DHIS2 community and existing end users. The webinar will serve to introduce the tool, the use cases, and provide an explanation of where additional learning and implementation resources are available.
Create a blog with the Use Cases. The blog will be distributed to the registered community, as well as distributed through various digital and HIS channels including the DHIS2 listserv, global digital health network, ICTworks, The Digital Health & Interoperability Working group
Building and Engaging with the Community
Active engagement with developers on GitHub. Direct the developers oto the GitHub page for access to the code and to engage with Bluesquare developers to promote the use of the tool, as well as track ongoing updates to the code. This includes updating the technical documentation.
Creating a public Jira board. The Jira board will provide public access to the product roadmap, as well as provide a forum for users to report issues and product features requests.
Request community-wide feedback quarterly. Bluesquare will reach out to the community of existing ORBF users to request feedback on a quarterly basis on the architecture and design, user experience, existing use cases, and limitations of the software from the development community and implementer perspectives.
Hands-on engagement on specific projects. Bluesquare will identify and request ongoing feedback from specific projects to gain deeper insights into how ORBF is being used and what are the needs of the community.
Use Cases
Use Case 1: Abt is leading the Integrated Health Program in DRC (DRC IHP). It has demanding M&E requirements. Because of the complex calculations required to carry out these requirements and the limited computational capabilities of DHIS2 the program managers and M&E team need to manage the M&E in a tool external to DHIS2. By implementing the DHIS2 App we are proposing the M&E team will be able to set up dashboards within the DHIS2. It will also centralize the performance measurements. In addition, it provides the MoH with easy and transparent access to the DHIS2 dashboards on the program.
Use Case 2: Abt is leading the President’s Malaria Initiative VectorLink Project in 23 countries from 2017 to 2022. One component of the project is building out a DHIS2 instance for the spray operations and management. The M&E team is currently rolling out the first phase of DHIS2. Future phases will require linking indicators across the entire VectorLink portfolio of countries and generating dashboards for reporting metrics. The DHIS2 App would provide computational capabilities that are not currently available in DHIS2 and would greatly enhance the project’s M&E capabilities.
Use Case 3: The Ministry of Health in Liberia is updating their Result-Based Financing program indicators to match the evolutions in the program.t They currently distribute incentives based on the relative performance of health facilities. They calculate this based on the individual performance scores for a set of weighted indications. The MoH needs to be able to change not only these indicators but also the weighting of the indicators, and the set of additional parameters that are factored into calculating the relative performance, as well as the incentive distribution based on that performance.
Currently, the Ministry of Health uses ORBF, integrated with the HMIS DHIS2, to calculate the performance,the relative performance, and the incentive payments. In order to make the necessary changes, the HMIS will need to use the independent ORBF to update the rules. In the current format building rules leaves is not as detailed in the parameters leaving room for error, especially given the complex tree of rules to manage the set of equations. The Ministry of Health needs easy access to the tool, as well as a controlled setting for use that prevents changes that will break the existing rules to be able to independently maintain their PBF Data System. The new interface proposed directly within DHIS2 would be able to respond to both of these needs.
Self-Assessment on the Global Goods Maturity Model
The self-assessment on the Global Goods Maturity Model can be found here.
Tagging
Interoperability, M&E, Health Information Systems, DHIS2, Performance calculations, Performance-based incentives, Analytics