Today, UNICEF receives an average of 170,000 text messages per month. Approximately 20,000 of these are unsolicited messages and initial analysis suggested that seven percent of these require immediate action from community leaders or the government.
As a result of U-report’s growth, the volume of diverse data was becoming a challenge for the existing system, making it difficult to identify what emerging issues were trending and which were the most urgent messages. Since February 2013, U-report has been using text analytics and machine learning technologies from IBM Research to help deal with the flood of information by automating the classification of messages. UNICEF Uganda and IBM Research deployed A-Class, a text classification system trained to understand the content of the text messages and analyzes the data much faster, and with much more accuracy.
IBM analytics, working in combination with UNICEF Uganda’s existing classification process are being used by UNICEF, partners and community leaders to make more informed decisions about where to place, and how to prioritize, development and relief work efforts.
“This project has the potential to change the lives of young people in Uganda simply by giving them a platform to communicate and be heard,” said Dr. Sharad Sapra, Representative, UNICEF Uganda.
“Today we’ve created a richer U-report that makes sense of streams of data in real-time,” he said. “This technology helps us understand the real impact of policies and development programs, the pulse of the nation at any given time and it even provides an early warning system on disease outbreaks or where we need to focus relief work.”
“We are working toward removing the guesswork out of knowing where to put resources in areas where they are most needed.”
Since its deployment, the A-Class system has contributed to a number of efforts. For example, in May, messages on what was confirmed to be flooding in the Kasese district of Uganda were identified in real time, allowing accurate and official information to be sent to relief workers and other partners who were able to respond to the disaster quickly. UNICEF provided a wide range of emergency supplies to help thousands of vulnerable people affected by the flooding that included emergency non-food items, temporary shelter and cholera kits to help prevent disease outbreaks within the camps.
As part of its further adoption, A-Class has the potential to help UNICEF Uganda pinpoint issues that might not immediately get picked up among the thousands of incoming messages, as well as automate the classification of messages that require immediate action. For example, if a disease outbreak is initially restricted to a village, even a small number of text messages could be picked out by the system and brought to the attention of necessary partners for action to be taken before the disease spreads to other areas.
Today, all 386 Members of Parliament in Uganda are now subscribed to U-report and through this system they receive SMS updates on reports pertaining to key issues in their districts. This enables the MPs to stay abreast of the latest issues affecting their constituents, and to take parliamentary actions to address these issues. By elevating the voice of youth to political levels, the U-report program has already resulted in notable policy changes in Uganda. For instance, the requirements for accessing a youth loan scheme, called the Youth Fund, were changed after U-reporters complained of the O-levels (high-school diploma) requirement for eligibility.
“This collaboration is a powerful demonstration of the value of analytics. We are very proud that text-analytics algorithms developed by IBM Research have made U-report a smarter and even more innovative tool that allows UNICEF to more accurately identify what matters most to the youth in Uganda,” said Dr Rick Lawrence, Manager of Machine Learning, IBM Research.
The A-Class text classification model was developed by IBM Research using a combination of supervised machine learning, the latest advances in text mining and keyword matching. The objective of the model is to classify each incoming text message into one of 12 UNICEF message categories — this classification task is particularly challenging given the brevity and frequent misspellings in SMS text messages. Much like some of the machine learning technologies found in IBM Watson, the A-Class model was also trained against 54,000 messages labeled by UNICEF volunteers so that it understood the full lexicon of words to classify and rank messages appropriately. Evaluation of the model has shown a classification accuracy of approximately 85%.
In August, a joint IBM — UNICEF paper, “Amplifying the Voice of Youth in Africa via Text Analytics” received the Best Paper Award in the Industry and Government Track at the 19th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, the leading international applied data mining conference.
Based on the success of the U-report program in Uganda, UNICEF has launched it in Zambia, Burundi and South Sudan, with a roll out expected in the Democratic Republic of Congo. The technology collaboration with IBM will also continue to explore how to automatically assign action status to messages, making the system portable in other countries and understanding how the system could be implemented in non-English environments.
U-report relies on RapidSMS, an open source software framework that works with any cell phone on the market, enabling UNICEF to connect with communities using a low-cost and popular device. Results from U-report surveys are publicized via the Polls Results section of the web site, and messages classified by A-Class are captured in real-time and displayed on the National Pulse page, allowing the public to visualize what issues are trending within each district in Uganda. Reports are also shared with government officials via newsletters and national media channels to ensure decision makers have access to information regarding their districts.
This initiative is supportive of IBM’s mobile computing vision, MobileFirst, which is rooted in the notion that innovative organizations connect with mobile users, build insights through more powerful analytics, deliver more convenient and relevant engagements, and improve management and customer service with seamless integration of front-end functionality and back-end data.
For more information on how the collaboration started, please visit A Smarter Planet blog post about IBM research scientist, Dr. Solomon Assefa.