A Study of Incidental Notetaking to Inform Digital Pen and Paper Solutions

With the aid of digital pen and paper technologies, information written on paper can be made available digitally without an intermediary transcription step. This creates opportunities to harness paper notes in ways that are only possible with digital systems. We report on the outcomes of a user study on incidental paper-based notetaking that examined, not only the forms of notes that users take, but also if and how these are later used. Our aim was to establish how useful existing digital pen and paper solutions would be in such settings as well as informing the design of new solutions.


INTRODUCTION
Information workers often rely on paper-based notetaking to record information in mobile settings as well as in situations where a certain degree of spontaneity is required [2].On the one hand, the notetaking process improves the memorisation of information [5] and, on the other hand, the unique affordances of pen and paper support the information capture process.However, the benefits of paper are often limited to the recording phase and do not integrate well with digital information management tools to be used in later lifecycle phases.
Existing notetaking systems based on Anoto's digital pen and paper technology1 are addressing this duality in paperbased notetaking.Entire pages or parts of paper notes are captured as digital ink data and optionally transformed into digital text based on handwriting recognition.In these solutions, the focus is on what Lin et al. [8] have identified as the temporary storage role of paper notes.In addition, they have described two more lifecycle paths of notes: prospective memory aid and immediate use.Specific guidelines in the notetaking process are used to infer the organisation of notes in digital space.These guidelines can either be inspired by certain observed notetaking habits or be driven by features of the technology used, such as handwriting and gesture recognition accuracy.
Much of the previous research on paper-digital solutions for notetaking has focussed on particular information tasks such as the collection of field data [20] or the composition of music [17].In these settings, where notetaking is a compulsory part of a user's daily work, it can be expected that users are more likely to accept certain notetaking guidelines if these will ensure that these notes can be captured and processed digitally in support of the overall information task.Furthermore, it may be easier to identify particular notetaking practices for specific tasks.Brandl et al. [3] have observed that digital pen and paper-based solutions are not widely used for natural notetaking.It is our hypothesis that the lack of support for natural notetaking results from different work practices and requirements in settings with incidental rather than necessary notetaking.
In order to verify our hypothesis and to gain some insights on incidental notetaking practices, we conducted a user study on paper-based notetaking as well as analysing results from existing related work.First, we wanted to understand how different forms of paper notes support activities in relation to the situations in which they are taken.Next, we planned to investigate to what extent specific formatting patterns can be identified in the notetaking process.We wanted to understand whether specific guidelines and rules in the notetaking process influence the acceptance of a particular solution.In addition, we aimed for new insights on how users might react to digital pen and paper related issues.Based on our own experiences of developing a wide variety of tools and applications for digital pen and paper solutions, our goal was to identify aspects in paper-based notetaking that might require special attention to enable the seamless transition from paper notes to digital information.
We start in Section 2 with a discussion of related work.In Section 3, we describe the user study method that we employed.The general findings of our user study are presented in Section 4 and digital pen and paper related issues are discussed in Section 5. A number of design implications for digital pen and paper-based notetaking are presented in Section 6. Concluding remarks are given in Section 7.

RELATED WORK
Previous studies on the use of paper notebooks [6,12,19] provide a detailed description of the nature of the notetaking process and identify multiple aspects that characterise the quality of paper notes.Paper notes are a schematic materialisation of user generated content in working or living environments.The notetaking process is characterised by a tension between the desire to capture information as quickly as possible and the rich information representation and structure [2].Notes often consist of highly subjective keywords and there is little organisation other than a chronological one.Notebooks are an amalgam of notes destined for multiple tasks, activities and purposes.The long term management of information written as paper notes has been identified as a major problem in paper-based notetaking.
The management of paper notes is further affected by general paper document management issues [2,18].Given the large amount of information that users have to deal with on a daily basis and their limited resources to process that information, information workers refer to a filing or piling strategy in maintaining paper documents.Whether extracted and filed into various kinds of special purpose archives or kept in notebooks, there are high chances that notes are no longer used.Due to their schematic nature, notes may also be affected by crises of intelligibility in time [10].Furthermore, written notes that are perceived as adequate memory cues at a given moment may lose this property over time.The efficiency and accuracy of pen and paper notes has been shown to be fairly high in the first days and week, but significantly lower a month after their capture [5].
Digital pen and paper, along with other technologies that support the capture of handwritten information, solves a major problem of paper notes by making them digitally available without an intermediary transcription step.Solutions that exploit these technologies have tended to focus on specific communities of information workers and their issues in managing paper notes [3,5,6,16,19,20].Paper notes are captured and integrated with digital tools in a post processing phase.These solutions rely on certain notetaking rules such as tagging, marking gestures or special purpose capture areas.Based on these notetaking guidelines and rules, the digital processing can be automated to a certain degree and different properties and classes of notes can be inferred.The properties are further used for the digital representation, organisation and retrieval of notes.Many of these solutions have been motivated by the observation that users tend to make use of various marks and labels in their notes [3,19].Similar approaches have been adopted by commercial solutions including Livescribe's Desktop2 , Oxford's Easybook3 , Capturx 4 and solutions integrated with Logitech's io Pen 5 .
Solutions that rely on specific notetaking guidelines may have limited success in settings where the notetaking process does not form a compulsory part of a user's work tasks for a number of reasons.One reason might be based on the difficulty of categorising information when it is captured [9].Even if the categorisation of notes is performed in a subsequent post-processing step, the definition of suitable categories for that processing can already be a challenging task [12].Another explanation might be the error rate in recognising special purpose marks or the highly unstructured nature of incidental notes which complicates the mapping of content to the corresponding marks [3].Moreover, notetaking with pen and paper is preferred exactly for the freedom and unselfconscious engagement that it supports [10].Proposed sets of rules for notetaking may be biased by technological limitations and, therefore, differ from the natural formatting users would normally use.It has also been reported that users are not always consistent in the choice of their marks and that the meaning of the used marks may become unclear over time [11].Besides altering the notetaking behaviour, these solutions may also increase the amount of time users spend to execute familiar notetaking activities.For example, Yeh et al. [20] reported a certain reluctance in the acceptance of systems that required users to spend more time in the field.
Existing studies emphasise the mixed content of paper notebooks and propose to extract and transfer notes into various digital information management and processing solutions.A question to be addressed is whether the requirement to digitally manage content applies to all types of notes in the same manner.Qualitative aspects of notes and the previously mentioned unselfconscious engagement may limit the use of simply providing a digital copy of the notes without a filtering mechanism [11].Another question is what the lifespan of paper notes in terms of their usage is.Sellen and Harper [15] describe the transition of paper documents through a series of hot, warm and cold states.A similar idea was introduced by Barreau and Nardi [1] for digital files that can be ephemeral, working and archived.Is the same evolutionary process applicable to paper notes and how should solutions for paper-based note processing support it?How much of this information is actually needed in the long term or loses its value once a current activity has been accomplished?Is it a fact that users rarely look back at paper notes [18]?To what extent is it true that information is not looked for because it is no longer needed [12]?
It has been argued that notable information loses a lot of its value for the information worker once it has been filed [7].Furthermore, Lin et al. [8] emphasise the role of paper notes in refreshing memory and argue that the way in which notes are maintained will have a great impact on the ease of referral.Khan et al. [6] also catalogue recall as the main reason for taking notes.One of the motivations behind our study was to investigate the role of recall in notetaking with a view to considering how this might be supported by digital pen and paper solutions.In particular, we wanted to get insights on how reminders might be extracted from incidental notes with a view to storing and visualising them to support recall.Proposals in this direction have also been made in the case of information scraps [2].

METHOD
Our aim was to document the design of paper notetaking solutions for information workers who are not focusing on a specific information task.In the recruiting process of our study, we targeted participants whose activities consist of multiple daily and weekly tasks that do not adhere to a well defined specification of processes.We chose to focus on meeting situations motivated by the fact that notetaking in meetings often consists of an agglomeration of notes with different utilities and does not give participants a lot of time to think about the formatting of their notes [6].We expected that meeting situations would reveal a high number of potential issues in paper-based notetaking in the time frame of our study [12].Furthermore, we were interested in a setup that would result in a high amount of variability in terms of notetaking situations.
We chose to investigate paper-based notetaking practices in meetings forming part of the professional activities of researchers in a computer science department.These users are typically not forced to take notes in meetings.Furthermore, the researchers are often involved in a range of activities such as teaching, research projects and administrative duties.We recruited 11 computer science PhD students (7 male, 4 female).We wanted to achieve an appropriate mix of participants in terms of the set of meetings and activities that involved paper-based notetaking.Therefore, we selected 4 participants who are in the first year of their PhD, 3 users who are in the last year of their PhD and 4 participants who are at an intermediate stage.In the prestudy phase, all of the participants declared that they make use of paper notes in meetings and none of them had used digital pen and paper notetaking solutions.
Each participant was given a high-quality lined paper notebook and asked to use it for a period of two weeks.The users were instructed to use the notebook for meeting-related activities and that they should write down information as they would normally do outside the scope of the user study.After the two week study, the notebook was given as a present to the participants to ensure that they would not lose their data and to compensate for their efforts.
After the participants had used the notebooks for two weeks, semi-structured interviews with each of the users were carried out.Analogous to the auto confrontation methodology described in [13], we asked the participants to explain their notes.The discussion was guided through a prepared set of questions that referred to topics such as the context in which the notes were taken, the purpose of originally writing the notes and the ways in which notes were used afterwards.In each particular case, the discussion evolved based on a participant's original experiences and further questions were asked.All interviews were scheduled at least two weeks after the initial two week time frame of using the notebook.The interviews lasted between 1 and 2 hours and were recorded.We also scanned each participant's notes with their permission.Subsequently, the recorded interviews and scanned notebooks supported us in performing a qualitative and quantitative analysis of their notes.

STUDY RESULTS
During the two week interval, participants took paper notes for a total of 133 meetings with an average of 12.1 meetings per person (min: 4, max: 23, median: 11, SD: 4.99).The notes contained a total of 2693 lines (min: 41, max: 590, median: 201, SD: 163.03) on 162 pages (min: 3, max: 29, median: 15, SD: 7.39).Furthermore, we counted a total of 5955 words (min: 99, max: 1067, median: 490, SD: 263.82).We first introduce a taxonomy of notes based on their support for different activities and highlight patterns in the use of notes for each identified category.Existing solutions for the digital capture of paper notes often make use of specific formatting patterns in writing.We present quantitative measures for a series of these observed formatting strategies to support our discussion in Section 6.

Categories of Paper Notes
We asked participants how they use their notes as part of their professional activities and identified 7 categories of notes.In the following, we describe the types of notes for each of these categories.
Support for work in progress.In this category, we classified notes taken as part of discussions that were related to producing or updating papers and reports, preparing presentations or designing software code and architectures.We also considered notes taken as part of the responsibility to update shared web resources (e.g. in a Wiki).Examples of notes in this category include, but are not limited to, feedback from others about deliverables, answers of other meeting participants for questions that were prepared before the meeting, as well as excerpts of programming code, sketches or diagrams.Notes that were taken to update shared web content included outcomes of meetings in terms of agreements, decisions, conclusions, tasks of team members and open questions.As highlighted in Figure 1, this category contains the largest number of notes, including 37% of the total number of words.Information to be communicated to others.As part of this category, we classified notes that support some kind of collective responsibilities.This information is not relevant for the users themselves, but has a certain relevancy for other parties.Examples include ideas, feedback and recommendations related to a colleague's work that are meant to be communicated after the meeting.Other notes in this category include information that has been written down before a meeting to be communicated to meeting participants.This type of information includes room numbers, URLs, instructions, reminders and questions.
Support for work organisation.This category includes notes that represent a user's own tasks, such as todos, as well as reminders for next meetings, presentations or conference deadlines.
Metadata.A separate category is formed by the notes documenting a meeting's content.In most cases, metadata notes were represented by a single or two line text block at the beginning of a meeting entry.As shown in Figure 2, 112 out of the 133 meetings made use of this pattern and only notes from 11 meetings contained no metadata at all.We further classified notes as metadata if they marked the transition to a different topic during a meeting as occurred in 10 cases.As a separate category we considered notes that were originally written under the assumption that they might turn out to be useful in the future.This category is further subdivided into notes that were considered potentially relevant or irrelevant at the time of the interviews.
Potentially relevant information.This category contains two types of notes.First, there is the set of notes with an undefined future use.Examples for this category include cue notes for open questions, references to papers, people, research groups and conferences, instructions and best practices, as well as technical terms such as names of technologies or frameworks.A second set of notes supports post meeting tasks and includes information about phone numbers, passwords, as well as shop or hotel names.These types of notes were generally used within a short interval after the meeting took place.Afterwards, they were treated similarly to the first set of notes mentioned in this category with an undefined future use.
Irrelevant information.This category mostly contains notes that document the meeting and were written as a manifestation of a type of unselfconscious engagement mentioned in [10].Users declared that the importance or the lack of importance of these notes was not obvious at the moment of taking the notes.The material is captured "just in case".Examples include points of discussion or outcomes of a meeting that represent general information, organisational details, interesting facts, details or numerical values and details extracted from larger documents, such as budgets or statistics.Often a designated user is responsible for updating a shared webspace (e.g. a Wiki) with such details.Participants reported that they would not check their personal notebook but rather have a look at a digital version of this type of information.In three cases, users declared that they write their own version of the notes in case disagreements arise in the future.
Diverting attention.This category refers to those notes which at the moment of the interview were declared to be useless information and primarily included doodles.Another example are unspoken opinions or reactions to presented content or discussed aspects.A special case of notes that we considered as part of this category were notes that are not related to professional activities including, for example, shopping lists.Users mentioned that these types of notes were only serving the purpose of fixing the information in their memory and would most likely not be of any use in the future: "writing helps me remember".

Post-Meeting Processing of Notes
Notes that form part of the support for work in progress category are dealing with tasks and activities that are characterised by a certain urgency or importance.For example, the work on some deliverable that the notes refer to is most likely executed in a relatively short time after a meeting took place.A user's attention is focussed around that activity and they are likely to rely extensively on their memories while creating the deliverable.In some cases, participants declared that they only referred to the notes in the final step to verify that nothing had been forgotten.
"Most of this I remember anyway.I wrote them down so that I don't forget them, but in general I kind of push them in the front of my brain." Analogous to the hot and warm documents mentioned by Sellen and Harper [15], paper notes that have been taken to support work in progress are a type of hot or warm information.However, paper notes show a faster and more definite transition into cold information than general paper documents.The reason for this behaviour lies in the different quality of the information represented by paper notes which is often incomplete and imperfect information: "I write enough details so that I can hopefully reconstruct [. . .] I rely on myself to remember the general stuff and keep the details in the notebook." The forms in which information appears in the final document may be completely different from the original notes.Notes are integrated into the final deliverables through a process of modifying, restructuring, rephrasing, filtering and completing with additional details.For example, one of the participants describes the process of incorporating feedback for a paper that they were working on: "[. . .] these are more to-dos, like I should work on this part, make it more clear [. . .] If I would have gotten any sentences or content directly I would have, maybe, copied it, but I did not get this kind of feedback [. . .] I would write feedback from all the sources, but some of the feedback is actually not usable, especially if comments from different people contradict.In this case, when I write on the document, I ignore some of the comments." Users declared that they would most likely not consult their notes once these had been processed and incorporated into the target deliverable.While three users mentioned that they are still keeping such notes for sentimental reasons or because they generally tend to keep everything, the rest declared that they would throw these notes away if they were not written in a notebook.
We asked the users whether having a digital representation of the notes in the form in which they have been written on paper, possibly transformed into digital text via handwriting recognition, would help them in processing their deliverables.Users were not convinced by the value of such a service.While some users declared that in certain situations it might be useful to have this as a starting point for their documents, a clear preference for starting fresh was observed.However, the idea of having a digital representation of their notes was found suitable for to-dos by those users who use a digital tool to manage their to-do lists: "To some extent, yes [. . .] If you already have the text there and it is something that would end up as your task, like a to-do, then yes.I think for to-dos it would work well." The idea of having digital versions of diagrams drawn in meetings was perceived as "possibly useful".However, participants were not able to describe in which way they would use the information: "It would be good to have it there to look at it".The usability of such a service for editing deliverables seemed to be limited: To-dos and reminders were marked on paper in a particular manner in only a few cases.Reminders typically contained time related information.Therefore, the meaning of the notes was easier to infer.The fact that certain notes represented a to-do was often only apparent to the participants themselves: "It is a task.It is just not written down so."The relevancy, importance and urgency of to-dos was not explicitly marked on paper, except for a few cases where the participants used circling, underlining or exclamation marks.Only part of the notes that were interpreted by the users as having the meaning of a to-do or reminder were still considered as something that they needed or wanted to do at the time of the interview.In some cases, the explanation was that the corresponding actions had already been performed: "I just went back to my desk and did it."In other cases, they reported that the to-dos were not something that they "really needed to do".Some users reported that they copied tasks on post-its.They kept the post-its on their desks in such a way that these were visible.For the same visibility reason, two users reported sending emails to themselves.For future tasks that were considered important, the tendency was to add entries into various digital tools, such as calendars or task managers.Users reported that they would typically set automatic reminders for this type of task.Often to-dos and reminders were not sufficiently specified on paper and the participants explained that they would add more details when entering the data into a digital tool.
Users had no concrete strategy in processing to-dos and reminders.They did not actively go back to their notebooks to look for something they needed to do.In some cases, the to-dos were processed after noticing them while consulting notes for other activities.For example, while processing the notes for updating a Wiki, a user mentioned: "And if there is something else I need to do, like reserve a room, I would probably also do it.I think it is useful that I have to go through these notes, because otherwise I would probably not look at them and forget that I had to do something." There were also multiple situations in which users remembered about certain tasks only while going through their notes together with the researcher.There was a quasi unanimous statement that "I just do it when I remember".Especially in the case of less important tasks, users declared that there were high chances that they would not do them.
Users mentioned multiple variations in managing information that was considered as potentially relevant for the future, but that they did not need at the particular moment when notes were taken.Examples include special purpose tools such as Outlook Notes, plain text files or browser bookmarks.However, none of the participants were consistent in extracting this type of notes.As in the case of to-dos, users extracted parts of the information while performing other more urgent activities.One user motivated the situation with the statement that "transcribing is boring".A more general reason could be the fact that users were not particularly fond of their information management approach and declared that they were still in search of a better way to keep track of their data.In the case of to-dos, users showed a certain ignorance towards the fact that these might be overlooked: "If something is really important, it will appear again."However, issues in managing the latter category of information was associated with a kind of frustration since there were high chances that information in the notebook was no longer remembered or found.
As shown in Figure 2, users did not put extensive efforts into an indexing mechanism based on metadata for their notes.Similarly to related work [12,19], users reported that the approach in looking for information in their notebooks consisted of browsing through pages and looking at titles.At the same time, they reported that they rarely looked back at notes.Therefore, the use of notes representing metadata is limited.
Given the fact that information that is meant to be communicated to others is not necessarily directly related to a user's interests, notes are written only "not to be forgotten".The time until they are used is typically very short and, after the information has been communicated, they lose their relevancy and are not consulted again.Often this information is remembered anyway and does not have to be consulted on paper.

Notetaking Strategies
Notes in the different categories described previously were not marked in any differentiating manner.Marks were used very scarcely by our study participants.Furthermore, the existing marks were not consistent in any particular way with the categories of notes.Therefore, identifying what the different parts of the notes represented could be done only with the help of the users.On the other hand, notes had a visual representation that presented a few distinguishable elements.These were typically separated by empty spaces.We identified three major types of blocks of notes based on their visual formatting: bullet lists, paragraphs and sketches.Figure 3 shows the average number of words per user for each of the formatting approaches and the average number of words per block of notes in general.
Bullet lists were the most commonly used approach for structuring notes.In some cases, entire meeting entries appeared as a single bullet list with the exception of the meeting description which was typically emphasised through underlining and separated from the rest of the content as shown  in Figure 4. Blocks of notes representing different categories are mixed inside a meeting entry.Furthermore, long bullet lists typically contained several categories of notes.Not all the points in a bullet list represented information that was used.In Figure 4, the first point was considered irrelevant, the following two represented to-dos and the last two points were reminders.From the two reminders, the first one was declared by the user as not really important for themselves.

Figure 4: Meeting entry in a notebook
As mentioned earlier, the use of various marks was rather limited.The marks that we encountered included underlining, deleting, circling, check marks, question marks, exclamation marks, arrows, parenthesis and "To-do".Figure 5 shows a histogram with the number of underlined words per meeting entry that includes all the notes written within a single meeting.73 out of a total of 133 meetings contained no such marks and there was only a limited use of them in the rest of the meetings.Similar results were found in the case of cross-out and strike-through marks used for deleting content and in the case of marks used to encircle content.98 meetings had no deleting marks at all, whereas 126 meetings contained no circling marks.Low values were also encountered in the case of check marks, used by 7 participants on average in 2 meetings.Exclamation and question marks were used by 6 users in an average of 4.5 meetings each.Arrows to annotate content were used only by 2 users and in both cases only once.Parenthesis were used by 4 users on average in 2.5 meetings.The marking of parts of the notes with the label "To-do" was performed by 7 users on average in 1.15 meetings.To get a better understanding of how well specified each user's notes were, we classified the blocks into three categories.The sentences class contains notes with complete or almost complete sentences and has an average length of 23.9 words.Notes mainly consisting of keywords are grouped in the keywords class with an average length of 5.1 word per note.Last but not least, notes between these two extremes are classified as fragments with an average length of 11.1 words.The resulting classifications are shown in Figure 6.

DIGITAL PEN AND PAPER ISSUES
Our assumption was that participants might be reserved in using digital pen and paper technologies to take notes if that implied changes in their work practices.We briefly explained the basic technology to each user and asked if they would be willing to use special sets of marks and conventions in notetaking.In exchange, they would get special benefits such as being able to digitally search and consult notes based on categories or be automatically reminded of  things that they are required to do.In each interview, the list of benefits was adapted to the user's specific problems in notetaking that they had mentioned earlier in the interview.Overall, users were reluctant about the idea of adapting to work practices that would require major changes or additional time for the notetaking process.Some users even explained that they would most likely not be able to adapt: "I would probably not be able to fulfil a lot of requirements." "I don't think I am that structured [. . .] I am a wild note taker." We also asked whether the marking of notes in a postprocessing phase would be preferable, but their responses were similarly reluctant: "If I have to do it as post processing, it doesn't make sense, because I could just type it, if I do that anyway."Furthermore, users expressed a certain reticence towards a mechanism that would hypothetically extract and digitally organise their notes by processing their natural marks such as crossing out, underlining or circling words: "I don't like and I don't trust that a machine takes decisions for me, what is important and what is not [. . .] If I say I don't need this, then ok, but I don't like that a machine does it for me." Users were further asked whether they would like their notes to be transformed into digital text.We mentioned that there might be potential handwriting recognition errors.Most of the users mentioned that they would probably discard a tool that would distort their notes: "If there are errors, I would have the feeling that something is wrong." Users acknowledged that paper-based notetaking is not optimally integrated with their activities that relied on the notes.At the same time, they reported that they preferred paper to digital tools for notetaking: "I really like to take notes on paper.I have a tablet PC, but somehow it is not the same feeling.The notebook is small.I really can take it with me, but a laptop I don't take it every time."

DESIGN IMPLICATIONS
On one hand, our study has shown that not all notes are used.Notes classified as not particularly useful to be extracted and made available digitally represented 34% of the total number of notes: irrelevant (20%), metadata (6%), information for others (6%) and diverting attention (2%).On the other hand, notes used to create some sort of digital representation are typically not used in the form in which they appear on paper.Notes for work in progress (37%) were integrated into deliverables through a process of updating, restructuring and modifying their content.Notes used for work organisation (21%) were often not sufficiently specified and users provided additional details to create entries within digital calendars or similar tools.This suggests that a solution for notetaking and usage that merely extracts and integrates notes with digital tools might not be directly applicable for 92% of the notes.Extraction and digital management could potentially be useful for the remaining 8% of the notes representing information classified as potentially relevant, especially since it is mixed with 92% of notes that lose their usefulness over time.However, the mentioned crisis of inteligibility [10] could render the information unusable, unless users further process the digitally imported notes.Furthermore, it is not very likely that notes in the latter category can be identified without user intervention.Potentially relevant information was rarely explicitly differentiated on paper in terms of formatting and users were not particularly fond of accepting rules and guidelines for notetaking.
Given these facts, proposing a solution for notetaking and usage can become challenging.Results in Section 5 show that such a solution should not depend on the use of notetaking rules and guidelines.Furthermore, the approach has to apply to all notes in the same way and still account for differences in the types of notes and how they are used.
Two aspects have drawn our attention.First, users attributed notes a main role in reminding and backup of details.Second, users were particularly fond of the flexibility provided by paper in notetaking, as well as the processing of notes.Unless associated with a certain urgency, notes were mostly processed when randomly encountered.Information that was "in the way" was most likely to be processed.Therefore, we think that solutions that focus on reminding users about notes that they have taken could potentially better support note usage.
As an initial step in this direction, we are currently experimenting with a system that passively informs and reminds users about the content of their notes based on ambient information visualisation channels.For this purpose, the user may select between either a ticker or slideshow application integrated into the desktop or an ambient display in their office.When a user returns to their office, paper notes can be extracted and integrated with the continuous information stream displayed by the application.We are investigating approaches based on the automatic extraction of note items and ones that rely on the user manually selecting reminder items from the notebook using gestures.Further, it is possible to specify whether the notes are only relevant for that day or for a longer period.Another functionality that could be provided is sharing information with colleagues by sending a note intended for them to their ambient display.
In a first experiment, we have integrated notes into a general ambient news service used within our research group [4].
The service for capturing notes to be integrated into the news service has been implemented using the iPaper platform for interactive paper solutions [14].
By extracting items from notebooks and making them visible in a non-disruptive way, users can be reminded of tasks to be performed or interesting facts or references that they may wish to look up at a convenient time.However, it is possible that taking notes out of their context could be problematic.We therefore plan to investigate a number of issues related to the types of items to be displayed, the granularity of these items and also the form of visualisation.At the moment, notes are displayed as an uninterpreted image of the handwritten note.However, we also want to experiment with the use of animation of the handwriting as well as the use of handwriting recognition software to display them as text.We will also investigate different ways of allowing users to have finer control over the display of items, including the ability to delete items from the stream.

CONCLUSIONS
In contrast to previous investigations of notetaking solutions, we have studied incidental forms of notetaking practices.Our study was based on notetaking in a variety of meeting situations involving university assistants.We identified different categories of notes and found that the majority of notes were either used shortly after the meeting or never looked at again.In the cases where notes were used as the basis for digital content such as updating a web page, authoring a paper or developing software, the information was reworked and users felt that the direct digital capture of the notes would be of little value.
Based on this study, we conclude that one of the main purposes of incidental notetaking is to support the recall of information and current digital pen and paper solutions for notetaking are unlikely to provide significant benefits.Instead, we believe that tools need to be developed that support the recall of valuable information items contained within these notes and are investigating ways of extracting and integrating them visually into a user's environment.

Figure 1 :
Figure 1: Percentage of words per category

Figure 2 :
Figure 2: Metadata text blocks per meeting

"
A picture, I wouldn't just put it on a Wiki [. . .] I would transform it somehow digitally [. . .] If it is a class diagram, I would do a class diagram"

Figure 3 :
Figure 3: Average number of words per user and formatting element

Figure 5 :
Figure 5: Underlined words per meeting