Low Resolution Spectroscopy Applications to Smart Home Systems : An Exploratory Study

A common challenge in smart environments is tracking individuals throughout different environments. Reliable solutions to this problem often involve cameras, which pose significant privacy issues, or trackable tags such as RFID which require that individuals be ‘prepared’ for the environment. In this paper an exploratory study is presented that investigates the utility of portable visible range spectroscopy hardware for the purposes identifying individuals based on the spectral pattern of their clothing. This is done by assessing the accuracy of a data-driven machine learning models when differentiating clothing. These sensors have the potential to achieve similar success to using the colour histogram from camera tracking without the privacy issues, and without the need to pre-tag individuals as with RFID’s.


INTRODUCTION
Smart environments (SE) provide a unique case of human computer interaction (HCI).By monitoring interactions within the smart home, they can adapt to the inhabitants and assist with daily activities or recognise important and meaningful changes in behaviour (Vega-Barbas, et al., 2015) such as deterioration in patients with dementia.
Current research into smart environments (SE) shares a common difficulty, the detection and tracking of individuals in either multi-tenant or multienvironment settings (Zouba, et al., 2007) (Boulos, et al., 2007) .

It is necessary to track individuals in situations
where what is normal for one individual is concerning for others.In assisted living situations a common task is to identify a fall (Wang, et al., 2016).It is important however to distinguish between the primary inhabitant and their carer(s).An elderly inhabitant falling is significant, a young carer tripping is less so.Solving this issue requires not only identifying everyone in the smart environment, but also tracking them throughout from room to room.The easiest method for doing so is cameras and technology such as facial recognition, but this is complicated by the ethics and privacy implications posed by cameras and their invasive nature, especially when monitoring users for healthcare and Ambient Assistive Living (AAL) applications involving vulnerable people (Chen, et al., 2012) (Michalis, et al., 2015).Various approaches have been taken to address this challenge, such as using RFID tags to identify individuals (Alsinglawi, et al., 2017).However, this requires ensuring every person in the environment is tagged and provides conflicting data should a non-tagged person enter the environment.
In this paper we present an exploratory study into using portable spectrometers for this purpose.This does not require everyone to be tagged and does not pose the privacy concerns associated with cameras.Other work with cameras has utilised colour histograms to differentiate people in multitenant scenarios (Krumm, et al., 2000), indicating that colour can be a reliable indicator.Conceptually, differentiating colours with a spectrometer is trivial, as they can provide more granular colour information than a camera, which is often restricted to just red, green, and blue colour channels.This is however an unusual environment for a spectrometer, which are often lab-bound and used in carefully controlled conditions.
In this study we look at the ability of a portable spectrometer to differentiate items of clothing in these conditions that they are not accustomed to operating in, using machine learning to differentiate and classify the items of clothing.
The remainder of the of the paper is organised as follows: Section 2 gives a brief description of 2 spectrometers.Section 3 thoroughly describes the methodology.Section 4 discusses the results of the experiments, followed by Section 5 which summarizes this work in a conclusion with future work.

SPECTROMETERS
Spectrometers break the incoming light into its constituent wavelengths and provide the intensity of light at that wavelength.This differs from the three channel RGB data provided by cameras.The spectrometer does not provide two-dimensional spatial information (i.e. a picture).It provides a breakdown of all incident light as a onedimensional spectra of wavelengths.Also, while red and green together are perceived as yellow by our eyes and by the RGB model, yellow is, from a physics perspective, the wavelengths of light in the range 570-590nm.
Spectrometers operate in different ranges, from ultra-violet through visible and into the infra-red region depending on the task they are being used for.The focus of this work is the visible range, as the task is to identify items of clothing.

METHOD
This section details the data collection apparatus, sampling procedure and testing used, and the preprocessing that was applied to the data.

Apparatus
For this study the spectrometer was suspended 2 meters away from the items of clothing.The clothing was suspended vertically so it was presented with the same profile as if someone was wearing it.To keep a similar environment to a smart-home while still controlled the windows were covered with regular blackout curtains (that you would find in a home) and the room's main light was on to keep a consistent background profile.The eight items of clothing used, and the setup of the system can be seen in Figure 1.The room was illuminated with a standard 50W household bulb as the only light source.
The spectrometer used was the SPARK from OceanOptics.It has a wavelength range of 380-700nm and a spectral resolution of 4.5-9.0nm.

Sampling procedure and testing
For each item of clothing 50 scans were taken before moving onto the next item.After all items had been scanned they were rescanned and another 50 samples taken, resulting in 100 scans total per item.This was done to account for any possible changes in the background over the duration of the recording to ensure any changes did not produce erroneous classification boundaries.Additionally, 50 scans were collected before and after of the background profile of the room with no items of clothing.The samples were collected sequentially with an integration time of 750ms, resulting in a total of 75 seconds to scan each item.
The data was separated into a training set (75%) and test set (25%).The dataset was split using random sampling within each class.To test the system models were built comparing each item to every other, including the background, and the classification accuracy used to assess performance.
Partial Least Squares Discriminant Analysis (PLS-DA) was used to classify the data.PLS-DA is an algorithm commonly used on spectral data in the visible and near-infrared ranges (Santos, et al., 2013).PLS and derivatives such as PLS-DA are often used due to PLS's ability to 'analyze data with many, noisy, collinear, and even incomplete variables' (Wold, et al., 2001) Spectrometer Clothing 2m

Pre-processing
The spectral data output by the spectrometer was pre-processed before being used in PLS-DA.For this study pre-processing consisted of baseline removal to account for global changes in the light baseline, normalizing the spectra to the same range, smoothing with a Savitzky-Golay filter, and finally binning the spectra to the approximate spectral resolution of the device.The bins were aggregated using the maximum value within each bin.The average spectra for each class can be seen in Figure 2.

RESULTS
All items of clothing could be differentiated from the background successfully; the majority of items could also be differentiated from each other with high (>90%) accuracy.With the exception of F vs G at 46% accuracy all other models achieved >80%.The results of these models are presented in Table 2.
Figure 2 Average spectra for each class after preprocessing

Conclusion
In this paper we examined the feasibility of utilising a portable spectrometer to differentiate different items of clothing in a setting appropriate for a smart environment.The purpose was to provide an initial assessment as to whether this type of data has potential when used with machine learning to track inhabitants across rooms and identify individuals.
The results show that most models have performed highly at over 90% with only one dropping below 80%.They can do so without presenting the privacy issues that can be inherent in other solutions such as cameras/facial recognition and without requiring each inhabitant to be tagged such as with RFID's.

Future work
Several items of future work have been identified.Primarily a study with a larger sample set and a greater distance from the spectrometer are recommended.As well as this the following items are identified: (i) Assessment of the long-term stability of these models and those like them as the background drifts over time (hours to days).(ii) Smart homes would require optics to alleviate the need for the spectrometer to be pointed in the vicinity of the inhabitants -Investigate how this affects the signal to noise ratio/accuracy of the system (iii) Movement of clothing in the spectrometer FOV (i.e.changing distance and angle from sensor) would produce differing spectral intensity changes.Are (linear) machine learning models capable of dealing with this well?

Table 1
Items of clothing used in the study