Crit Thinking

Title: voice of the worlds beyond

MR with Headset
Users can see themselves and the environment around them through the pass-through mode, but the sky will turn into a starry sky, which is interactable.
Users will experience the entire work as if they were on a tour.

Art style:

• Reality
• 2D or 3D model for the stars
• models can move and rotate a little bit, make some animation.

Voice: sound of message or noise.

Interaction:

• user can straight their hand to touch, then they will heard the noise from the star or the sound of message.
• Some items (ex. blackhole) will scale up when the user interacts with them.

Engine Version: Unity 6000.3.4f1

Items:

blackhole

3D model link: https://sketchfab.com/search?q=black+hole&type=models

galaxy

Nebula (as creature)

Pillars

J1407b

Wormhole

Movie Reference:

Dune

The Three-Body Problem

Arrival

We went to Richmond Park and used a Insta 360 X3 camera to film the deer. However, it rained that day, the weather was very cold, and the ground was full of mud, so the final footage didn’t turn out very well.

We borrowed the Insta360 Pro 2 camera and consulted our teacher about its functions, as well as thoroughly tested its settings and the entire workflow from recording to exporting.

We also tested the effect of moving shots (I acted as a human tripod here since we didn’t bring one).

We also gained some knowledge about equipment for outdoor audio recording, including which gear to borrow and how to connect the equipment.

We revisited how the project should be filmed. This included continuing to use the deer footage, with us performing as the deer and standing in the same position, intercutting the shots. Through this, we discussed the relationship between reality and fiction, as well as the feeling of presence and absence.

However, after further discussion with the group, we decided to emphasize the effect of movement. We therefore changed the approach to filming a ball-passing game: when someone receives the ball, they introduce themselves. During the process, we want to experiment with guiding the audience and using spatial sound, and finally throw the ball toward the camera to give the audience a sense of being present.

Last week, while discussing with classmates, we talked about wanting to include some fantastical scenes. Based on that, we came up with the idea of placing a 360 camera inside a transparent plastic bottle.

[Cut 1]
At the beginning, the bottle is lying on a table. Then someone enters the frame, stands it upright, and throws it into a black trash can. The person ties up the garbage bag, and the image turns completely black.

[Cut 2]
Once the garbage bag is sealed, we can experiment with light, shadow, and sound to create a fantastical effect.

We lay out the detailed concept elements, discuss them thoroughly, and then decide to make the sceㄇne after the object is thrown into the trash more fantastical. At the same time, we also want to explore some styles related to festivals.

Moving 360 camera

• put insta 3X camera in the bottle
• Slow motion when the bottle fall down into the bin.
• After in the bin can show something unrealistic through light and sound.
• the view in the scene can be show in the beginning, like the poster on the wall.

After turn into dark

We are considering whether rain at the outdoor venue of the film festival will affect the display of our work. Additionally, we want to use interactive posters to describe the design details of the project, such as the size data of NASA’s planets, etc.

Festival

Outdoor or Indoor?

Indoor view
Use the shipping container space

Poster
show the design and information

We’ve also decided to output a web version so that while the audience is waiting in line, they can experience parts of the work (with only the functionality available, but the screen will be entirely black). This will give them a sense of a preview and surprise.

Web

• Black space. But can receive the signal.
• With the interactive/moving poster.

We started implementing the idea from last week by placing the 360 camera inside the bottle. However, we encountered an issue because the image is created by stitching together two lenses, and the thickness of the camera caused a gap in the middle, resulting in a severe division in the image. The bottle also didn’t appear round. To solve this, we decided to lay the camera down, which fixed the problem.

Test of putting the camera inside the bottle

We did our first test of the video effect. We filmed at the Kiki Muimui store, starting by taking something out of the fridge, passing through a park, putting it into a backpack, and finally throwing it into the kitchen trash can at the Highline building.

To guide the user to click the special star (which will export when pinched), I created a particle effect to give a wave-like visual. Here, I used vertical locking to ensure the textures always face the camera, so that when generating stars in the distance, we don’t need to individually adjust their angles.

Here, I added a twinkling effect to the stars. In addition to a particle system that scales them up and down, I also created a material that allows them to flicker.

I implemented two trigger methods. The first one is hovering over a point and then pinching, which plays a sound. The second uses the line between the midpoint of the eyes and the fingertip, which also plays a sound when pinched. I also received some suggestions from my teacher, such as snapping to the nearest point to make selection easier.

In class, we made adjustments to the laser beams, including their color, length, and a fade-out effect when they are close to the camera.

Night Time without half-black skybox

Day Time with half-black skybox

I also added adjustable parameters, making it easier and faster for users to test and tweak.

The hand-ray laser effect has been completed. Visual feedback and sound effects have been added when triggering the stars, and we are also considering incorporating the stars’ twinkling characteristic into the effect.

For the sky environment, I completed the implementation of a semi-transparent material and replaced the traditional skybox with a sphere to ensure compatibility with the passthrough mode. I also implemented smooth sky transitions based on sunrise and sunset times. In addition, I took the headset outdoors at night to evaluate the effect in a real-world environment.

During testing, I noticed that all stars currently share the same size and distance, which creates the impression of standing inside a sphere rather than being immersed in outer space.

We conducted user testing based on the experience flow diagram developed by our group member, Zijun Wu. To better communicate certain interactions and scenarios, we supplemented the testing process with videos captured during the previous week.

Feedback from participants highlighted a potential queueing and waiting-time challenge that had not been previously identified. Additionally, participants noted that an excessive amount of information presented during the experience may contribute to cognitive overload and user fatigue, suggesting a need to further refine the pacing and information density of the experience

Presentation PPT

At this stage, we transitioned from randomly generated stars to real astronomical data. The star dataset is sourced from the European Space Agency’s Hipparcos Catalogue, using the hyg_v42.csv dataset as the basis for star generation and visualization.

By retrieving each star’s RA (Right Ascension) and distance data during the app’s initial loading process, I was able to calculate and represent the relative distances of the stars. I also retrieved each star’s magnitude (mag) value, which is used to control its visual size based on brightness.

In addition, the star’s corresponding HR, proper (Prop), or GL designation is now displayed when the star is selected and its associated content is played.

After further discussion, we revised the UI behavior so that it only appears when the user hovers over a star. The UI remains visible until the hover ends, or, if the star is selected, until the associated playback has finished. This change helps reduce visual clutter and improves the overall user experience.

We also redesigned the interaction effects that occur when a star is triggered, making them more consistent with the project’s comic-inspired visual style.

In addition to using stellar magnitude (mag) to control star size, we now use the spectral classification (spec) data to determine star colors, allowing the visualization to more accurately reflect the characteristics of real stars.

In addition to recalculating and repositioning the entire star field based on the Sun’s location (as the HYG v4.2 catalog uses the Sun as its reference point), I also integrated the NASA API to accurately position the planets within the Solar System.

Besides handling the models and visualization of all eight planets, I developed a reusable script that allows different planets to retrieve their positions relative to Earth simply by changing an identification parameter. This modular approach significantly improved scalability and reduced repetitive implementation work, making it easier to expand and maintain the planetary system in the future.

Through this process, I gained valuable experience in astronomical data processing, API integration, and building reusable systems within Unity.

I added an introductory logo screen that appears when the app is first launched, followed by a tutorial to ensure users understand how to interact with the experience.

To prepare for next week’s mock exhibition, we have been developing a set of questions for user feedback. Although there are many individual questions, they can generally be grouped into three main categories.

User Testing Questions

In addition, feedback received during our tutorial with the instructor suggested that we could replace the “select” interaction with a “hover” interaction, as it would be more beginner-friendly. It was also recommended to support a single-hand ray interaction. Based on this feedback, we have implemented these changes.

During the testing period, Ashwin Fulfagar and I took turns serving as welcome staff for the two mock exhibition sessions. When one of us was occupied, the other acted as the project presenter, introducing our work, assisting participants, and communicating directly with users throughout the experience.

Timeslip Radio XR experience walkthrough and testing

We received feedback from visitors and found that, although switching to hover improved accessibility for beginners, it was still unintuitive for experienced VR users, who tended to expect a pinch gesture instead. In addition, the hover interaction made it too easy for users to accidentally trigger stars, resulting in excessive overlapping audio playback.

We added a tutorial animation at the beginning and combined the title with the tutorial sequence. This ensures that users still have a chance to see the title after we help them launch the app, preventing them from missing it while putting on the headset.

At the same time, we added dialogue audio to guide and invite users into the experience.

Based on feedback from last week, we improved the special stars by increasing their size. After shrinking back, they remain at their original position and can be triggered repeatedly.

We also changed the interaction cue from the previous star-plus-wave indicator to displaying the original 3D star model instead, making the visual feedback more consistent with the rest of the system.

We also added dialogue at the ending to provide a clear conclusion to the experience.

I built the website for our group project: Voice of the Worlds Beyond website. It is used to introduce our project and team members.

We also tried printing the poster designed by Zijun Wu, but the design turned out to be too dark, making the QR code difficult for cameras to detect.