AR-RA

An Ultimate Personal Research Assistant

Team

Bonnie Jiang

Shenhao Xie

Mentor: Michael Nebeling

Software

Adobe Aero, Aframe, Figma, Premiere

Duration

Sep 2020 - Dec 2020

My Responsibilities

Problem Identification, Design, Video Editing

01 Overview - What is AR-RA?

AR-RA stands for Augmented Reality Research Assistant. It is a proof of concept we designed and prototype to represent our interpretation of an ultimate personal assistant. As young researchers, we want to take a different approach using AR technology and redefine personal assistant in academic research. By using AR to organize, synthesize, and search their workspace and references more efficiently, researchers can concentrate on their topics and speed up the process more efficiently. The purpose of this design is not to utilize AR to complete the work for the users but to provide them a research environment that is guided and encouraging.

Domain

Education/Academic Research

Context

The users will use the product with their other devices and software. The product will make the reference, writing, and data organization more efficient.

“How might we utilize AR technology to design a personal assistant to make the early-stage process more efficient and motivated?

01 Sort by Keywords

Users can use the sort function to automatically sort existing literature into groups. They can also manually sort the sources by simply dragging the articles into the augmented reality space. The users are able to get visualized categories in less time.

02 Manage Organization

Using AR, the assistant provides more workspace for the researchers. The users can use horizontal, vertical, and physical space to organize their work and collaborate with the team. They do not need to switch back and forth from browsers. The dispersion of the windows is also ergonomically designed.

03 Reference Check

The assistant helps researchers to identify low relevance articles. The researchers will save more time reading irrelevant work. The assistant also helps to identify highlighted references within the article, searching them from various databases.

04 Eye-tracking Search

Utilizing eye-tracking, the assistant allows a more efficient search experience. Researchers save time by switching back to the search bar in the database.

02 Understand - How Can AR Support Research?

Problem Statement

The early research process (exploratory research) can be challenging to one’s mental and physical capacity, and sometimes could be less motivated. In their early career, many graduate research assistants feel that they are doing repetitive tasks without making any major contribution. Although the research process includes meaningful repetitive exploration, it could also diminish researchers’ confidence and passion in the topic. Therefore, less experienced young researchers like me need a personal assistant to make the early-stage process more efficient and motivated.

Target Users

Researchers
Office Workers

What they are looking for

Reduce inefficient reading and repetitive tasks
Using the algorithm to aid early-stage review
Utilizing AR technology to achieve better organizations.

Existing Solutions

There is a lack of AR solutions for research aid. But there is citation software such as Zotero and Mendeley, writing aid like Grammarly, and visual aid such as highlighter. But they are currently limited to the screen.

Constraints & Ethics

There are several constraints for this project. First, we need to be careful about the level of assistance the product provides. The assistance needs to excel current citation machines, keyword search, and file organizer. But it should also focus on educating the researchers and help them grow rather than contributing to intellectual work. We should also consider how the AR environment can enhance this process. What unique aspects can AR achieve in research assistance?

Why AR?

Allow Multiple Screens
According to the research and experiences, researchers prefer working with multiple screens to organize their work, especially with early exploratory work. For various reasons such as financial constraints and frequent travel, it is often limited for them to have access to multiple monitors. Augmented reality comes in handy in this situation.
Diverse Interactions and Organization
AR allows various interactions involving the human body, which opens up the possibility of how researchers can manage their resources.

Storyboard

Before we diverge to brainstorm solutions, we use the storyboard to explore possible functions and in-use scenarios. Storyboarding helps us to better understand what challenges uses may encounter during research.

Because the product is a personal assistant, I use myself to construct the user case.

03 Prototype: Diverge & Converge

Sketch - Diverge

We first created different sketches exploring designs that would address the scenarios. We focused on organizing the resources function first because researchers identified it as most used.

User Flows - Converge

Because there are so many gestures researchers can use to interact with the assistant, we discussed together and draw a user flow on how researchers would be interacting with the assistant on three different surfaces— the physical surface, the horizontal and the vertical surface of AR.

Physical Prototype -Diverge

Based on the user flow, we first draw a sketch to outline the interactions and prototype with paper.

Design Divergence

My partner and I had different design decisions about where we should place the sort and collaboration buttons. We came up with two solutions.

📌The buttons should float in the vertical space on the side of the laptop because it can be the center of attention.

📌The buttons should rest on the physical horizontal space, next to the non-dominant hand to be more ergonomic.

In an ideal world, I would have to conduct an A/B test to gather user preferences and consult an AR specialist to consider the feasibility. Due to time constraints, we did a heuristic evaluation and analysis with a human body sketch.

From the sketch, I was able to convince my partner to go with the horizontal space next to the non-dominant hand. Because “Sort“ & “Collaborate” is secondary functions. Putting the buttons on a flat surface requires less angle change for the user’s wrist. On the contrary, a vertical surface would require the users to lift up their upper arms, which could be more inconvenient.

Final Deliverables

We each took on 1-2 main functions and prototyped the UI elements in Figma.

After we build the UI elements, we make the digital prototype using Adobe Aero. More detailed interactions are described in the video.

04 Validate

Reflection

💡More Ethical Discussions

This project inspired me to think more about the use of an assistant in research. To what extent is the assistant helpful, and to what extent is the assistant hindering critical thinking? Is it still entirely the intellectual work if the assistant helps them to decide which low-reference work they should skip? These are questions worth researching.

💡AR vs. VR

I can’t help thinking about if it is necessary to have a physical laptop in the middle. Can the environment just be a flat surface, or it is feasible if the assistant is in a virtual space? Our preliminary test showed that people have different preferences. Some enjoy typing on a physical keyboard, making it crucial to be AR. Some think VR may be more convenient and accessible for people who do not own a laptop. We need to conduct further research to see if XR affects productivity.