Feeding a Finch with social media: A bird that responds to tweets!

This paper presents the design and implementation of a system to allow the control of robots or other systems using natural language and social networks


INTRODUCTION
Social networks have permeated many aspects of our daily lives and by using them to communicate with each other, they have made the Internet a more accessible place, allowing anyone to publish content for others to see without needing specific skills.Robots, although more complex and powerful than they once were, are still relatively inaccessible to the general population, commonly requiring the knowledge of programming languages or training for complex applications.In this paper we bring social networks and robotics together which would allow any naive user to control a robot using simple natural language commands.

BACKGROUND
Twitter (2012) is one of the most popular social networking services on the internet with on average 340 million "Tweets" or messages sent per day .Commonly used for one-to-many communication, it also provides one-to-one communication using private messages.Because Twitter has a simple interface and is basic in its design Tagtmeier, C (2010) it should be easy for an inexperienced user to swiftly gain an understanding of its use.In this paper Twitter is used to send commands and queries to the system controlling the robot.A potential user only begins their message with @robotTweeter42 followed by their command or question.Natural Language Processing (NLP) covers a wide range of methods used for analysing, creating and manipulating natural or human language.It was used to convert natural language commands from Twitter into code to control the robot.The Finch Robot was developed to assist students learning how to write code to control basic robots.It has light, temperature, obstacle and accelerometer sensors, and has control over two motors, an RGB LED and a buzzer Lauwers, T. (2010).It has been used to show the outcome of the translated commands by moving, taking photos and obtaining sensor readings.

ARCHITECTURE
The process (Figure 1) shows a Tweet received from a user over Twitter; the next step is to translate the message of the Tweet into code that can control the Finch robot.Once the code has been generated from the message, it is processed and controls the robot by moving it and/or collecting data from its sensors and camera.The results of the processed code are then summarised in an HTML document that shows the original Tweet, converted code and the results.This is then uploaded to an HTTP server and the URL is saved in a response Tweet file.This is then sent back to the original user.The system has been divided up into four main components: the Twitter processor; the translator; the Finch controller; and the response processor.Translation.This Python script translates from Tweet to code.The message is assigned Part Of Speech (POS) tags to each word (or token) in the message.Once the POS tags have been assigned, the toolkit then chunks the command, by grouping consecutive words/tags together that match certain pre-defined patterns.
Once all words in the command have been chunked together, the list of chunks is processed.For simple actions, the chunk is passed to a keyword search that identifies specific keywords e.g.move, show, tell, wait.Depending on the keyword identified, the chunk is passed to a specific function that converts the chunk into a line of code.The function initially assigns default behaviours in case the command is vague e.g. the turn command will turn left at speed 2 by default.The code then searches the processed commands for keywords that allow it to refine the behaviour.For example, if the chunk was turn 180 degrees left, the function would assign the default values, then refine them until it had iterated over the entire chunk.By searching for specific terms, keywords or integers, the system can create a loop or conditional structure.It then processes the actions within it to create the whole code structure.Once all chunks have been processed, the lines of code are saved to a Javascript file.
Finch Control.This Java application processes .jsJavaScript files to control the Finch robot using as few commands as possible.It handles all aspects of the connection to the robot and allows the system to capture still images via a webcam.
Response Processing.This Python script summarises all data from the completed jobs directory.The Django foundation framework Django (2013) is used to generate an HTML document containing the collected information.The HTML file is uploaded to a webserver using FTP.In order to ensure that the response Tweet is under 140 characters long, the URL to the file is shortened using a free URL shortener service such as tinyurl.comTinyUrl (2013).This is then placed in a response message and saved to a .Tweet file in the outgoing Tweets directory.

EVALUATION AND CONCLUSION
Currently, this system is designed to run purely in the context of controlling the Finch robot.However there is great potential to look at different systems that could be controlled using Social networks, including: more complex robots, database queries, simple machines e.g. a coffee machine, question/answer systems etc.As well as designing the system to be specifically compatible with a defined piece of technology, the system could be rebuilt so that it is easier to connect other devices and then teach the natural language system to translate commands to code for them.During development, it became apparent that although it was easy for anyone to control the robot, it was still important to explain to the user the type of basic commands/actions that could be used to control it.Moreover, in order to improve the quality of NLP, the system could be modified to learn from its mistakes.If a user sent a specific command and the system failed to translate it correctly, the user could send the system a line of code, or a revised command for the system to use in future.Over time this would improve the system as it would evolve to improve the accuracy of translated commands.
In future, different social networks could be examined for their potential when working with robots.New communication paradigms may occur that will lead to new ways to control robots.Although Twitter is currently one of the most popular methods, the system could be tied into other services to allow greater access to users.

Figure 1 :
Figure 1: High level processes of the system

Figure 2 :
Figure 2: An example of the HTML page produced.