Art for Computer Scientists: Processing as an open-source art medium for computer science undergraduates

Art undergraduates increasingly engage in the design and coding of digital media and generative art, for example, using the open-source Processing library and Integrated Development Environment (IDE) created for visual artists. An equivalent cross-over, of art and creativity into traditional computer science programmes, is not so evident despite a shortfall of skilled graduates in the digital media sector. This paper considers art for computer scientists and outlines a Keele University Animation and Multimedia module that uses media programming not only as a vehicle for first-year undergraduates to practice new Java programming skills, but also as an opportunity for artistic exploration and creative expression.


INTRODUCTION
Beyond combined honours programmes and specialisms like computer games and music technology there is not much art in traditional computer science and engineering programmes, outside the arts-oriented institutions such as Goldsmiths.
Modules on art using computers were once a common part of computer science programmes (Hertlein, 1977;Scala, 1976).However, current practice is to focus on the use of high-level digital arts applications for creativity rather than on lowerlevel computer programming, and, perhaps, there is a tendency to assume that computer science students are uncomfortable with creativity and artistry (Eber and Wolfe, 2000).
In this paper we outline a first-year undergraduate Computer Science module: CSC-10026: Animation and Multimedia at Keele University, where the students are encouraged to consider themselves as artists and creative designers, and to explore creative opportunities whilst also developing their Java programming skills and learning the fundamentals of 2D and 3D computer graphics and interactive multimedia.
Most students studying the module are single honours Computer Science students studying for BCS-accredited BSc (hons) degrees.However, some of the students are on combined honours programmes, and a few are non-Computer Science students taking the module as an elective (based on some prior prerequisite programming experience).The cohort therefore encompasses a wide variety of skills from quite experienced programmers who also may have A-levels in Computer Science to relative newcomers with, as yet, little experience of coding.
Pablo Picasso is quoted as saying that "All children are born artists, the problem is to remain an artist as we grow up".Students studying the Computer Animation and Multimedia module are encouraged to take the opportunity to be an artist once more and to create their own visual assets for all the assessed animation components using methods of their preference, e.g., using any combinations of graphical primitives, hand or computer drawings or photographs.Additionally, students can elect to invest more effort in artistic creativity or in programming, or a balance of both.
Students studying the module use Processing and other open-source media tools to create interactive multimedia and animations.

Processing
The original open-source Java Processing library and IDE was developed by Casey Reas and Ben Fry for non-programmers and artists to create visual art (Reas & Fry 2015).Together with Daniel Shiffman and the support of a large community of contributors, the Processing Foundation (Fry and Reas, 2021) now also supports Python and JavaScript (P5.JS) languages as well the original Java.Processing is also supported for Android and Raspberry Pi platforms and there are many libraries (created by both the Processing Foundation and by third parties), for example, libraries for creating graphical user interface components, producing and controlling sound, plotting graphs, simulating physics, and enabling serial and network communications.
Although Processing is a lower-level visual art medium compared to some of the more sophisticated 3D graphical tools, professional creative suites and games engines, it does afford students an opportunity to implement and explore many fundamental graphical and animation concepts whilst still supporting the creation of functional and multi-faceted visual art.

Other open-source media tools
In addition to Processing, there are a large number of other open-source media creation and editing tools, several of which are used by the students.For example, GIMP for image editing, Inkscape for scalable vector graphics, OBS for video capture, Blender for 3D graphics and Audacity for audio editing and recording.With many students currently working remotely due to COVID-19 restrictions, the availability of such open-source tools has been especially beneficial.

ANIMATION AND MULTIMEDIA
A module on Animation and Multimedia provides an excellent opportunity to encompass both technical and creative components.The core themes and practical activities of the module are summarised in Table 1 and examples of animations used to explore and exemplify these themes are listed in Table 2. Screenshot examples from these sketches are shown in Figure 1.There are also, of course, many other animations available via the Processing community and also a large number of example animations are included in the Processing IDE download and organised into topic folders in 'Examples'.

Scientific theory and concepts
The Animation and Multimedia module introduces and reinforces useful concepts for computer science students, for example, trigonometry, geometry, transformations, data structures, number systems (binary and hexadecimal numbers), modular arithmetic, audio synthesis and effects, image processing, laws of motion, collision detection, particle systems and the simulation of physics.

Art and design theory and concepts
Animation and Multimedia also provides opportunities to introduce art theory (at least in terms of formalism), and composition and design concepts.For example, the elements and principles of design are introduced and also illustrated using Processing as shown in Figure 2 with simple examples generated with a sketch using PShape forms.

The principles of animation
Students are introduced to the 12 Principles of Animation defined by Disney's animators [Thomas, 1995] and these are reflected on at intervals through the module.For example, students are encouraged to identify specific principles of animation in Pixar's Luxor Jr animation and to identify any uses of the principles (as well as other concepts and sources) in the rolling credits of their submitted animations.

Visual effects and generative art
The module content also touches on generative art and visual effects, and these are observed with a variety of sources including simple Processing examples and selected Khan Academy 'Pixar in a Box' tutorials on Animation, Rigging and Rendering as well as inspirational ACM SIGGRAPH conference trailers and presentations, and Oscarwinning VFX trailers.

Processing in undergraduate projects and research
As academics, we are often encouraged to inform teaching with research.However, there are also opportunities for teaching to inform projects and research.In the Computer Animation and Multimedia module we use our 3D research to demonstrate photogrammetric Structure from Motion acquisition and the anatomy of 3D models.
In addition to using Processing in first-year teaching, we have found the Processing IDE to be useful for both undergraduate student projects and research projects when real-time user interactivity and graphical feedback are required without excessive coding requirements.For example, finalyear project students at Keele have used Processing for interfacing with Arduinos and for developing mobile apps.Additionally, prototyping an Augmented Reality (AR) app for Android smartphones can be a daunting undertaking when using the Android Studio IDE, but a very simple AR app can be achieved with just 20 lines of code using Processing for Android.For this reason, we used Processing to create the 'Augmented Reality Museum App' (Woolley et al. 2020) shown in Figure 3 that was developed as part of the Virtual Cuneiform Tablet Reconstruction Project (VCTR 2021).

CONCLUSIONS AND FURTHER WORK
Overall, students respond very positively to the Computer Animation and Multimedia module content and the opportunity for creativity.The quality of coursework submissions is impressive, as is the creativity, effort and care that students invest in their work.
There is scope for further work to evidence and quantify student views and to develop and evolve approaches and identify best practice.

Figure 1 :
Figure 1: Screenshot examples of module Processing sketches.Top to Bottom: Bezier curve grasses and fractal tress with randomness, flowers with transformations and randomness, circular motion and solar systems; sprite animation and 2.5D effects, binary and analogue and Android app clocks and particle system fireworks, visual effects (Dolly Zoom, Back Projection and Bullet Time; and musical instrument.

Figure 2 :
Figure 2: Elements and Principles of Design created with PShapes by a Processing sketch.

Figure 3 :
Figure 3: Processing for Android Augmented Reality

Table 1 :
Key themes of Keele's first year Computer Science Animation and Multimedia module

Table 2 :
Examples of animations used in the Animation and Multimedia module.