Research Activities
This dairy and beef research project worked in the provinces of Punjab and Sindh (Pakistan) to improve the profitability of farm and animal production. It did this through developing market and value chain opportunities. Building on a 10-year long engagement with the dairy sector through the Australia Pakistan Agricultural Sector Linkages Program (ASLP), the project aimed to scale out its extension model by enlisting a broad range of local collaborating organisations.
More information about the overall project can be found on the ACIAR website http://aciar.gov.au/project/lps/2016/011
Initial Perceptions
The project leader Dr David McGill had previously been a lead researcher on the MAD Bali Pilot (MAD 1). He therefore had the most experience with CommCare of any project leader in the MAD 2 series. The project staff did not have previous experience with CommCare but were excited by the prospect of MAD adoption. Dr McGill had concerns around the data security and ethics, as well as the possibility the MAD implementation would lead to excessive data collection (See table). The in-country project leader was most concerned about hardware issues, while the app manager also felt concerned about data security and excessive collection, as well as potential problems with internet connectivity.
MAD Activities
In contrast to the adoption approach favoured by other projects in the MAD series, this team were able to build MAD capacity internally through trial and error and minimal external support. This was largely due to Dr David McGill’s existing experience with MAD allowing him to lead the project staff in mostly self-directed training without AgImpact support. The project also received additional separate funding from ACIAR for a scoping study into CommCare implementation (C2016/043). As a result, AgImpact support was less focused on training and much more so on planning and broader architecture of MAD implementation.
"After the masterclass, I realized that we can use the software as a training tool. We developed and transferred our extension material into the Commcare and now we are using this Commcare extension material in the field." - Abdul Aziz, Pakistan Dairy App Manager
Within the Pakistan Dairy project, the capacity for the CommCare platform to be used as both a data collection system and as an extension tool was trialled. The project built two application(s): one for mapping out value chain actors in the beef-value-chain (deployed during the SRA) and one for tracking research and extension activities by partner organisations (built but not deployed during SRA). This second application was designed to be used during a scale-out phase to monitor activities of each actor in the value chain and their teaching, learnings and adoption. In addition, a final activity was the conversion of one printed extension factsheet into a digital/electronic format in CommCare. This factsheet represented one module in a 10-module extension program, the remainder of which remained to be digitised at the completion of the MAD 2 SRA. This initial digitised factsheet was deployed as a static digital sheet and as an interactive application, along with traditional printed sheets and evaluated for its ability to engage farmers. During the development of these applications, the Pakistan Dairy team evaluated the automatic CommCare translation service for Urdu. Finding it unreliable, the team then devised a new method for translating CommCare applications into Urdu through trial and error with several other services (e.g. Google translate). This was undertaken by the in-country team in a self-directed manner.
Designing and building the app
Three planning sessions were held between the project lead and Oikoi staff to assist in designing the two apps. During the second planning meeting, it became evident that the project staff already had ample capacity in basic CommCare application building, but had not considered how to best design their application. Allocated training days were therefore spent assisting refining the Value-Chain Survey MAD application (which had already been partially built). This consisted of considering best design practices and overall architecture for effective MAD implementation. Training in more advanced features such as ‘case management’ was also provided. After this process, the in-country project team completed the value-chain survey application. The in-country team were able to develop the app well, though encountered a significant challenge during the translation from English to Urdu. The translation of English extension materials into Urdu was completed on CommCare by project staff in Pakistan without assistance from the project lead or AgImpact. This was achieved through a trial and error approach which began by using CommCare's inbuilt translation function before attempting various other approaches and finally settling on a Unicode software that could translate Urdu into machine-friendly symbols. This task was particularly time consuming, with a total of 15 project staff days spent on it.
Testing the app
Testing of the initial value chain survey forms built by the project staff was performed with the presence of Oikoi. Forms were tested with farmers in Pattoki, and challenges or issues with the application questions were noted for discussion and / or revision. This testing process allowed for more sophisticated planning by the in-country team for the remaining forms in the value chain survey which had not yet been built.
Deploying and monitoring
Deployment of the value chain survey application was managed in-country by the app-building team with little assistance from Dr McGill. Field staff collected basic data using the application and supplemented these with more detailed data which were recorded on paper. No application issues were reported to the project lead during deployment, though Dr McGill noted that the forms being deployed used only the most basic aspects of CommCare so their success may be attributed to the simplicity of the app.
Deployment of the digitised extension factsheet (both in static digital form and an interactive form) along with the original printed sheet was conducted in the form of a field study in extension delivery. Three extension teams each consisting of five individuals each (a mix of researchers, experienced field staff and new field staff) were sent to deliver extension materials. Each group was given one of the three formats (traditional printout, static digital app, interactive digital app) and delivered the information to farmers. All enumerators were then surveyed to measure their perceptions of farmer engagement. While the value chain survey application was being used on a daily basis, the project lead was able to check the data and monitor their quality. When any data were unclear, Dr McGill called the team directly to clarify any issues. At the completion of the SRA, the project team had plans to develop more monitoring and evaluation of data and app usage.
Benefits & Challenges
Benefits
"Going out and carrying out the survey is easier. Building the survey and changing it is easier. Analyzing data is easier" - Dr David McGill, Project Lead
The main benefit of MAD adoption in this project was that it gave the team a platform which could be used to discuss the future of data being collected. The project was able to visualise what the data would look like and then consider how it would be used during analysis. Conversations around the depth of questioning, the enumerator comprehension and the scale of the collection (i.e. the time it takes to complete a survey) all took place between staff. These led to revisions in the survey instrument as well as valuable lessons in research design for all researchers involved. The project lead Dr McGill stated that surveys on paper simply don’t provide the same catalyst for such discussion.
Another key benefit was the sense of ownership among in-country staff of the research process. The ‘light touch’ approach by Dr McGill in training and application building allowed the in-country staff to have more confidence in tackling data analysis. This was due to their time investment into planning the analysis during application building. Having access to the data through CommCare HQ also meant the team did not need to get the data files directly from the project lead, saving time.
Challenges
While the Pakistan Dairy app builders were able to master the basic CommCare functions, they had less opportunity and success working with more advanced features. The main application for value chain surveys consisted of very simple forms. Dr McGill suggested that a renewed time investment from the team would be needed to master these more advanced features. This would not be possible through purely self-directed learning and would require a more hands-on management approach from either the project lead or Oikoi.
Responses from project staff showed they were most likely to find all phases of MAD implementation to be a ‘Normal’ level of difficulty compared with their usual work (see Figure below). However, a substantial number of responses indicated training, building and testing to be ‘Difficult’. Deployment and Data Monitoring & Evaluation were the only phases that any staff nominated as ‘Very Difficult’.
Financial & Time Costs
A total of $2,200 was reported by the Pakistan Dairy project. These operational costs were used to purchase tablets for the team to collect MAD in the field.
This project differed from most in that little support time was provided by AgImpact whereas a great deal of amount of time was spent by project staff on MAD implementation. A total of 5 support days were provided to the Pakistan Dairy project by Oikoi. While three of these days were logged as ‘training’, due to the high CommCare capacity among the project staff, training was mostly focused on high-level planning. The Pakistan Dairy project staff logged a total of 210 days on MAD implementation. Most of this time (79 days) was spent on planning, with training (39 days) application building (53 days) also using a lot of staff hours (See table).
