A ' Human-inthe-Loop ' Mobile Image Recognition Application for Rapid Scanning of Water Quality Test Results

This paper describes an interactive system for drinking water quality testing in small community supplies, particularly in the developing world. The system combines a lowcost field test (the Aquatest field kit), a mobile phone for data processing and communications, and a human operator who is able to react immediately to a test result. Once a water sample has been collected and incubated, the mobile phone camera is used to 'scan' the test and obtain the result, which is displayed to the user along with information about the health implications of the water quality. Initial prototypes, while not yet sufficiently robust for real-world use, demonstrate that the system is technically feasible. This opens up interesting possibilities for wider use of 'human-in-the-loop' sensor systems in environmental monitoring.


INTRODUCTION
An effective water quality monitoring programme is essential to maintaining water supplies that are adequate for basic needs, safe to drink and environmentally sustainable.Inadequate water, sanitation and hygiene services responsible for 5.7% of the total burden of disease worldwide [1], and account for 1.6 million preventable child deaths each year [2], the majority of which are in developing countries.
Small community water supplies are the most prone to water quality problems [3], but also the hardest to monitor.The usual method of measuring microbiological contamination in a water source involves taking a sample at the source and then transporting it to the laboratory, where the sample is tested by trained technicians under carefully controlled conditions.Unfortunately, this approach becomes infeasible in areas with many remote supplies and poor transport infrastructureparticularly in the developing world, where monitoring programmes operate under severe resource constraints.A site manager at one of our field sites in South Africa describes the challenges they face: "this municipality serves an area roughly 200Km x 100Km, with more that 450 000 people spread over a deep rural area crossed by very poor roads.Travelling 100Km here can easily take 3 times longer than elsewhere.We have about 100 rudimentary stand alone rural water schemes and water supply systems to 5 towns.Just collecting representative water samples on a frequent basis, getting it to a laboratory (closest is about 200Km away), interpreting the results and reporting it is a huge and very costly exercise" Municipal Water Services Manager, South Africa In this paper, we describe an alternative strategy for water quality monitoring, based on a low-cost field test (the Aquatest field kit) that is interpreted by 'scanning' the completed test with the camera of a mobile phone.Software on the phone processes an image of the test and displays the result to the user.The combination of a field test and supporting phone software allows the location of the person doing the test to be prioritised over the skill level required.Providing appropriate information at a local level can empower small communities by improving their ability to manage their water resources in a sustainable, decentralised way.Regardless of the availability of communications infrastructure, the test result is immediately known to someone on-site, and remedial action can be initiated as soon as a test shows contamination (after 24 hours of incubation).The person doing the test can also be prompted to identify potential hazards that may be causing contamination.The ability to act on the test result information is a major advantage of a 'human-in-the-loop' interaction over an automated sensor.

THE CONCEPT: A MOBILE PHONE READER FOR AQUATEST FIELD KIT
The Aquatest field kit includes an Aquatest device and a field incubator [4].The Aquatest device (Figure 1(a)) measures microbiological contamination of drinking water using a multiple tube method.Inside the device, the water sample is divided into a number of chambers of various sizes and a reagent is added that detects the presence of Escherichia coli (E. coli is a standard indicator of faecal contamination).Any chamber of the device that contains E. coli will change colour after an incubation period of approximately 24 hours, with a pattern of positive and negative chambers becoming visible on the base of the device (Figure 1(b)).The pattern of results can be used to estimate the number of E. coli in the sample -this estimate is often called the most probable number (MPN).
The mobile reader application asks the user to take a picture of the base of the device, and then decodes it to give water contamination information to the user.The image processing is done entirely on the phone, meaning that network coverage is not require to obtain the test result.The development of a mobile phone based reader for Aquatest device also reduces the potential for human error in reading the test result, especially when the test is used by non-specialists.
The data storage, processing and communication capabilities of the mobile phone are also exploited to streamline the process of obtaining the MPN, extracting from it an understanding of its implications for water safety and entering the result into an information management system.Secondary functions of the mobile reader include incubation timing (recording incubation start, then sounding an alarm and launching the reader program at the end of the incubation period) and tracking of tests devices by means of cameraphone-scannable barcodes, scanned at discrete points throughout the testing and incubation process.The Aquatest field kit is intended for areas with limited resources, particularly in the developing world.The mobile reader concept was developed following Marsden's "pragmatic design" approach [5].This meant working with technologies that were already widely available in the intended context.Although the literature on mobile image processing [6,7] reveals that most similar applications are built on smartphone platforms, smartphone ownership in the developing world is limited, especially outside the major urban centres.Network coverage in these areas is also variable, leading us to reject an alternative approach that uses a low-end mobile phone for image capture but relies on a remote serve to process the image and return a result.

PRAGMATIC DESIGN (a) (b) (c)
The mobile phone reader therefore targets J2ME (Java Mobile Edition) devices, specifically those implementing the Mobile Media API (MMAPI).Apart from a few very low-end phones, this platform is available on almost all cameraphones sold today.It is also the platform of choice for Microsoft's recently released flagship mobile application for emerging markets, Microsoft OneApp, as well as South Africa's very popular Mxit instant messaging client.Developing for J2ME has significant limitations (some described in this paper), but phone availability renders all other platforms infeasible in the short term.

PROTOTYPE APPLICATION
The current application version is divided into four major sub-sections: setting up a timer application to prompt the user to start application, registering/recognizing the device, decoding the colour information and finally converting the chamber count into useful information for the user.

Timer and barcode scanning
This section of program manages the test registration and times the incubation process.It registers the test and incubator number by scanning a barcode or by entering a number.The barcode could be used for additional tracking functionality, such as checking the expiry date of the test and keeping record of the stock of devices.The timer application reminds the user with an alarm to read the test after a predefined incubation period, and launches the reader application.Figure 2 shows the use of the mobile application to manage the test registration and incubation process.

Figure 2: Barcode scanning and an incubation alarm help the user to manage the testing process
After the picture of test device is taken, the first step in the image processing algorithm is to recognize and locate the device in the image.A red marker line is printed around the circumference of the device (itself circular) for this purpose.The Hough transform [8] is used here for circle detection, and helps to locate the red circle.The Hough transform is a very effective but computationally intensive technique and the time taken to complete the operation depends on the number of pixels in the circle.To reduce the computation time, the red circle is thinned out.First a red filter is used to separate the red pixels, then it is ANDed with a Sobel edge detection version of the image.Hough transform is then applied on the ANDed pixels to identify the circle and locate its centre.

Reading / filtering the colour
Locating the outer circle also defines the location of the two rings of cells inside the device (Figure 3).The outer ring of cells is sampled at a radius of about 75% of the circle radius.The pixels picked up around the sampling circle are filtered by applying a colour filter.Coloured pixels obtained as a result are divided by the total number of cells (in the ring) to obtain a count.A similar count is obtained for the inner ring, except that the sampling radius is now 34% of the circle radius.

Converting the chamber count into MPN and health implications message
The chamber count is converted to an MPN count by using a look up table already stored on the phone.According to the MPN count obtained, a simple health message is displayed to the user.The health message could be something like 'Water is contaminated, DO NOT DRINK', 'Water is safe to drink' or 'Boil water for children, elderly or sick people'.The relation of message and ranges would be governed by the water regulatory practices in the concerned country.At this stage, the software also offers data upload options to the user.

CHALLENGES
Development of the mobile reader application is still in progress.In its current state, an experienced user operating the reader in good light is able to read the test correctly most of the time.However, the reader still falls short of meeting the acceptance criteria for error rates, effort required compared to manual reading, and correct operation in variable light conditions.The image recognition algorithm is likely to be improved by applying adaptive thresholding in the colour filtering stage.
The choice of J2ME as a development platform has presented some significant challenges.The 'write once run anywhere' promise of Java is far from the reality of J2ME development, making it difficult to determine which phone models are able to run the application without examining each of them individually.On some Java phones with a camera, camera access through MMAPI is not implemented.Similarly, having a phone with Java and an alarm clock does not guarantee the availability of the Personal Information Management (PIM) API, which is used by Java applications to set the alarm; where PIM is available it does not necessarily implement alarm setting functionality; and where it is possible to set the alarm from Java it may not be possible to specify the alarm type or tone.On Nokia Series 40 phones, which include most of our test phones, the only available alarm is a silent alarm, limiting the usefulness of the alarm functionality for incubation timing.In short, inconsistent implementation of J2ME makes it difficult to build applications with same functionality on different phones.This in turn complicates the user experience.
A related problem arises out of the limitations of camera access with MMAPI, combined with the fixed focal length of low-to-mid-range mobile phone cameras.Fixed focal length mobile cameras are designed to work best for photos of people or large objects, 1-2m away.As a result, images of an object held in the user's hand (or of a small barcode such as would be printed and stuck to the device) are always out of focus.Even on phones with autofocus or focus-adjustable cameras, J2ME image capture through MMAPI does not provide access to any methods to set the focal length, or to the full resolution of the phone camera.As a result, the image processing step is forced to operate with imperfect data -a blurry image taken at relatively low resolution.
In fact, image capture restrictions turned out to be less significant for the core mobile reader image processing than for the recognition of the small barcodes used as identifiers on the test and the incubator.The limited flat space (1.5x1.5cm) on the test proves a real challenge for reading the barcodes.Perhaps for this reason, literature on cameraphone-scanable barcodes [9,10] has emphasized new applications for barcoding that use larger codes.Applications that require close-up image recognition, such as business card scanning, would suffer similar problems if they were to be implemented in J2ME.In practice, most applications of this nature leverage smartphone platforms, with better processing power and better access to camera controls.

CONCLUSION
This paper has described the development of water quality testing system combining a low-cost field test, data processing and communications using a mobile phone, and a human operator who is provided with information to react to a test result.Lack of standardization and poor application access to hardware capabilities on our chosen platform proved challenging.Nevertheless, the system is technically feasible, and development is ongoing.This demonstrates the ever-present tension between the promise of new expressive interactions on the one hand, and the constraints in resources and environment that we are required to work within according to the pragmatic design approach.
Looking beyond the immediate details of the implementation, the concept of a three-part, human-inthe-loop sensor system linked to a communications network has wide applicability in environmental management, particularly in the developing world.Supported by the processing, information storage and communication capabilities of mobile phones, specially designed field tests for environmental parameters can be distributed to people located at testing sites, with the results feeding into a central monitoring database.This has great potential to expand and densify monitoring networks, while also providing appropriate information to affected communities and empowering local-level environmental management.

Figure 1 :
Figure 1: (a) The Aquatest device.(b) picture of the base of the test, which is scanned to determine the result and (c) scanning the base of device with mobile phone.

Figure 3 :
Figure 3: Test device showing sampling circles Information flow to a central water quality database is maintained by uploading test results either immediately or when network coverage is next available (store-andforward functionality).
If the test result indicates contamination, the software will recommend appropriate remedial action, such as additional treatment.Both the test device and the phone software are designed to be used by people who are based near the water supply, but do not necessarily have specialist training in water testing.