FMP

I experimented with Object Detection, which was able to detect smaller objects. However, it could only detect them while they were visible to the camera, and tracking became less reliable once they moved out of view. Therefore, I decided to use MRUK Scene Setup as the main approach for detecting and understanding the environment.

quest object detection

Object Detection could still be introduced later for movable objects, such as hot-air balloons, landmarks, or other interactive elements that may change position within the environment.

I initially created my own character model and tested it with animations. However, the result had a lot of mesh clipping and unnatural movements, so I decided to use a free 3D character model instead. I then used Blender to re-rig the model with a Mixamo skeleton and retarget the animations for use in the project.

The original character model was designed to have a soft, tentacle-like body, but the result did not work as well as expected. There were also noticeable clipping issues around the chest and shoulders when the arms moved. Because of these problems, I decided to put this character design aside for now and explore other 3D character models instead.

The new character model focuses more on a cute and playful appearance, which is closer to how I originally imagined the Zashiki-warashi in this project. Since the animations were retargeted from Mixamo, there are still some clipping issues, particularly because the character has a relatively large head. Some of the animations will therefore need further adjustment and refinement.

I refining the material rendering by developing a gradient outline effect. Instead of using a standard outline, I created the effect by slightly offsetting and overlapping the geometry, allowing multiple semi-transparent layers to appear around the surface.

By combining these displaced layers with a gradient, the outline gradually fades rather than appearing as a single solid line.

It was my first time attending Develop:Brighton, and it was a really valuable experience. Beyond the conference itself, I had the chance to meet many experienced professionals in the industry, as well as people who, like me, are hoping to start their careers in the field. Through conversations and talks, I gained useful insights into both technical topics and career development, while also getting a clearer picture of the current state of the industry.

There were also plenty of fun activities outside the conference sessions, from grabbing free ice cream to joining the 8am morning swim event! It was a great mix of learning, meeting new people, and enjoying the experience.~~~

At this stage, I focused on developing the material and rendering effect for the buildings. I wanted the building to gradually break apart and disappear, creating the feeling that the little creatures are digging into the furniture and revealing the hidden world inside.

Since the buildings are generated based on furniture with different shapes and sizes, the starting point of the dissolve effect also needs to change dynamically. Considering both flexibility and performance, I decided to create this effect using a Shader rather than controlling individual pieces of geometry.

In the first version, I used a noise texture to create the fragmented appearance and controlled the dissolve amount using a single Control Value. However, this meant that the dissolve value was applied across the entire model at the same time, so the building disappeared relatively evenly rather than breaking apart from a specific location.

In the second version, I replaced the Control Value with a radius-based dissolve system. The Building script uses the creatures’ digging position as the dissolve centre, and then gradually increases the radius over time. By combining this expanding area with the noise pattern, the building progressively breaks apart from the digging point outwards, creating the appearance of a hole gradually expanding through the structure.

I created four different types of buildings — Home, Restaurant, Entertainment, and Work — to support the different needs of the little creatures.

Each creature has its own set of values managed by the ShadowManager, including Hunger, Energy, and Fun. These values gradually decrease over time, creating different needs for each creature. Based on which value needs attention, the system automatically selects the corresponding type of building and sets it as the final destination for the creature’s NavMesh Agent.

This allows each creature to make simple decisions independently and move between different buildings based on its current needs, creating a more dynamic miniature world rather than following a fixed movement path.

I also implemented a capacity and queue system for the buildings. Each building has a limited number of positions for the creatures to use. If all available positions are occupied, newly arriving creatures will wait in a queue beside the building instead of overlapping with the creatures already inside. Once a position becomes available, the next creature in the queue can enter and use the building.

This helps make the movement and behaviour of the little creatures feel more organised and creates more natural interactions within the miniature world.

Since the little creatures need to climb onto buildings and travel between buildings placed at different heights, I needed to develop a ladder system to connect these separate NavMesh surfaces.

I also refined the climbing direction of the little creatures. The movement was adjusted so that they climb up from the outside of the building and also climb down from the outside, rather than turning towards the inside and colliding with the building geometry.

However, the ladder’s orientation and positioning still require further refinement. I have tried to keep the bottom and top connection points aligned on the same side of the building, but some ladders can still appear tilted, misaligned, or intersect with the building. This will need further adjustment to make the generated ladders more consistent across different building shapes and positions.

At this stage, I combined the different systems developed during the previous experiments into a more complete prototype. This included the ladder system, NavMesh navigation, and the building dissolve effect, allowing the different mechanics to work together as part of the same experience.

The blue areas shown in the video represent the generated NavMesh surfaces, which visualise the areas where the characters are able to walk and navigate.

I then built the project onto the Meta Quest and tested it with MRUK, using Scene Understanding to detect real furniture in the physical environment. This allowed me to test how the generated buildings, navigation, and interactions work when they are placed directly onto real-world furniture.

At this stage, I revisited and refined the overall visual style of the project. I made several adjustments to the colour palette and visual layering, moving away from the previous combination of a black background and blue shapes. Instead, I introduced a brighter and softer colour scheme, which feels more playful and better reflects the visual direction I originally envisioned for the project.

I experimented with the building materials by combining three visual layers: an outer noise-based surface, a gradient layer, and an outline effect. The outline is created by slightly offsetting and overlapping the geometry inward, producing a semi-transparent layered line effect around the structure.

After building and testing the project on the Meta Quest, I noticed a shaking/jittering issue with the layered materials. It appears similar to a performance delay or an issue caused by multiple overlapping surfaces being rendered at the same time. This still needs further testing and optimisation to identify the exact cause and improve the visual stability on the headset.

I’m currently exploring ways to transform the buildings by randomly generating connected elements such as windows, balconies, and other architectural structures.

However, I’m planning to develop this further by generating different miniature cities within the furniture itself. This will help strengthen the feeling that each piece of furniture contains its own hidden world, with a completely different environment waiting to be discovered inside.

I arranged windows of different shapes and sizes across all four sides of the Box Collider, using a random generation system to fill each surface. The system divides the available wall space and selects different window prefabs to place within it, creating variations in their size, position, and overall composition while avoiding excessive overlap. This allows the same building shape to generate different façades each time, rather than repeating a fixed pattern.

I also experimented with architectural elements that extend beyond the surface, such as balconies, to add more depth and variation to the generated buildings and make the façades feel less flat.

At this stage, I reconsidered my overall vision for the project and began to rethink how I wanted the experience, world, and different systems to work together.

Concept

As a child, when I pressed my head into my pillow, I could hear my own heartbeat clearly. I would imagine that the sound came from someone underneath the bed or behind the wall, digging through it with a shovel, as if there were another unknown space hidden just beyond what I could see.

This experience developed into a fascination with the invisible spaces within everyday environments — behind walls, inside furniture, underneath beds, or deep inside cabinets. These spaces exist alongside the places we live in, but remain hidden from us.

This project aims to visualise and give physical form to these unseen spaces, imagining that they are not simply empty gaps or solid structures, but secret environments inhabited by small creatures. Different pieces of furniture and architectural structures can be

Reference

The concept is inspired by traditional stories of spirits and creatures that secretly share domestic spaces with humans

1. Yōkai — Zashiki-warashi

These child-like spirits are believed to inhabit houses and usually remain unseen, occasionally revealing their presence through playful behaviour or subtle traces.

This idea inspired the concept of creatures that already exist within our living spaces but normally remain unnoticed.

2. Brownies / The Borrowers / Tatsuya Tanaka

The Borrowers

Tatsuya Tanaka

Brownies and The Borrowers explore the idea of small beings secretly living alongside humans and making use of everyday objects and hidden domestic spaces as part of their own lives.

Similarly, Tatsuya Tanaka’s miniature works reinterpret familiar everyday objects from the perspective of tiny figures, transforming them into landscapes, architecture, and living environments.

These references inspired me to imagine how ordinary furniture and household objects could take on completely different meanings and functions when experienced by much smaller creatures, becoming part of their own miniature world.

3. Gulliver’s Travels

Gulliver’s Travels inspired me through the idea of different worlds, each with its own inhabitants, environments, scales, and ways of living. Rather than focusing only on the difference in size, I am interested in how each world can have its own identity and logic.

This influenced the idea that the hidden spaces within a room could contain different miniature worlds, each shaped by the furniture and architectural space in which it exists.

Design Settings for the Little Creatures’ Life System

One Day: 3600sec (1 hour)

Animation & Model

I also started thinking about how the little creatures could physically construct their environment piece by piece, rather than having buildings simply appear.

For example, they could carry individual parts and place them one at a time, such as extending a ladder section by section or adding different windows onto the walls. This could make the construction process feel more connected to the creatures themselves and allow players to see how their miniature world gradually develops.

Another possibility is that the hidden world already exists inside the furniture, and the little creatures do not build it from scratch. Instead, they gradually dig or carve through the outer structure of the furniture, revealing the world that has always been hidden inside.

For some narrow or elongated pieces of furniture, such as chairs, the approach could be different. Rather than revealing an existing interior space, the little creatures could construct new structures piece by piece, allowing them to gradually grow and extend from the original furniture.

Story (if have time)

The scene begins in complete darkness. A crack slowly appears on the wall, accompanied by the sound of a chisel coming from behind it. The camera moves closer towards the crack. As it reaches the wall, a section of the surface suddenly breaks away, revealing a hidden space behind it. A small creature then peeks its head out through the opening.

The little creature climbs out through the opening and begins using a shovel to expand the hole. It then wanders around the space, guiding the viewer as the camera gradually pulls back to reveal more of the hidden world. The creature begins placing ladder sections one by one, building upwards until the ladder reaches a higher point in front of the viewer.

Adjusted the rendering order of transparent materials so that objects are displayed correctly based on their position and depth.

I also implemented spawning for different types of furniture and refined the digging/dissolve effect, including its duration. Next, I’ll continue developing the buildings and environments that appear inside each piece of furniture.

這邊處理了一開始進去的第一個場景，劇情設定在