COMS W4172: 3D UIs and AR—Spring 2019
Prof. Steven Feiner
Date out: February 9, 2019 Date due: February 26, 2019
Assignment 1: Kaiten-Zushi 3D Introduction
This assignment will be the first Unity project you develop on your own, after having completed Assignment 0 and Assignment 0.5. You will be using Unity 2018.3 and C# to design a 3D scene containing objects with which the user can interact using your mobile device’s touchscreen. Your scene will be a kaiten-zushi (conveyor belt sushi) fast-food restaurant. Unlike restaurants of this type that use a single conveyor belt whose technology is essentially a scaled-down version of airline baggage-claim conveyor belts, your restaurant will use four straight conveyor belts, as described below. (Yes, this is a poor design, but it will help turn your program into an entertaining game.) In addition, there will be a chef, plates of food, tables, and trays.
Since we do not want you to have to create your own 3D models, you can download models (free ones only, please) for all objects in your scene from the Unity Asset Store, or any other source (e.g., the ones listed on our IA page), providing you have permission and cite each source properly in your documentation. Alternatively, you can load a model (.fbx file, .obj file, or other supported formats) obtained elsewhere by dragging its file into the Project View in the Unity Editor Window. Any associated textures should be added to a “Textures” folder placed next to the loaded model in the Project View. You can also import an entire directory at once. While you are also welcome to create any of your objects directly from Unity primitives, you should use at least one downloaded model. Again, any model that you use must be free.
Conveyor Belts, Trays, Chef, and Tables
There should be four conveyor belts arranged in a rectangle, as shown in Figure 1. All four belts are identically sized cuboids whose top surfaces are parallel to the world xz-plane. The conveyor belts do not need to be textured and do not need to move. That is, they do not have to actually behave like real moving belts: Only the plates of food will need to move! The conveyor belts are not connected to each other. Instead, there are four stationary trays at the corners of the rectangle bordered by the belts. The trays should be the same height as the belts.
Within the space surrounded by the belts there should be a chef. Outside the conveyor belts, there should be at least four tables. On the conveyor belts there will be sushi plates, special plates, and dessert plates that move along the paths defined by the belts.
1
Figure 1. The relative positions of the four conveyor belts, chef, tables, trays, and sushi/special/dessert plates, as seen from above. Each red arrow indicates the direction in which plates on a conveyor belt move.
Sushi Plates and Special Plates
The restaurant serves sushi plates and special (non-sushi) plates. Thus, special plates can contain whatever food you’d like, which will make finding a free model easier. Each plate should move in the direction and speed associated with its belt after being placed on it, as described below, and should be in contact with its belt (i.e., not floating) as long as it remains on it.
Sauce Plates
We’re not done with the special plates. This restaurant is famous for the automated plates it uses for its special plates, as shown in Figure 2. Each special plate has two sauce plates hovering over it. Each sauce plate should orbit in a circle about a vertical axis through the center of its special plate at a radius, height, and speed of your choosing. Each sauce plate should have a different orbital radius and should not collide with the other one. By default, each sauce place should maintain the same orientation relative to its special plate while the sauce plate orbits the special plate. (See Transform.RotateAround and Transform.Rotate for more information.) Each sauce plate can have any shape as long as its orientation is clearly visible. (This will help you and us verify visually that your sauce plates are moving correctly.)
2
Figure 2. A schematic illustration of how sauce plates move relative to a special plate.
Dessert Plates
The restaurant also serves ice cream for dessert, using a (famous, of course) automated plate shown in Figure 3. Each dessert plate has one ice cream scoop. The ice cream scoop orbits in a circle above the plate around a point whose height is up to you and at a speed of your choosing. The ice cream scoop begins its orbit within the plate’s local yz- plane (around an axis parallel to the plate’s local x-axis). As the scoop orbits in its plane, the orbital plane should rotate on an axis that passes through the orbit’s center and is parallel to the plate’s local z- axis at a speed of your choosing. The scoop itself should rotate on an axis that passes through the center of the scoop and is parallel to the plate’s local y- axis. You will need to ensure that the single rotation described in the previous sentence is the only change in the orientation of the scoop relative to the orientation of the dessert plate. The restaurant is quite proud of the way its ice cream scoops move; so, to show that off, please choose an asymmetric scoop shape. For example, if the scoop is shaped like a chair (“chairy” ice cream?), its legs should always point down, parallel to the dessert plate’s local y-axis.
3
Figure 3. A schematic illustration of how the ice cream scoop orbits above the dessert plate. The orange axis is parallel to the plate’s local z-axis. The scoop, shown here as a ball, should have an asymmetric shape that clearly shows its orientation in your program.
Lights
There should be at least two light sources in the scene: One is a stationary directional light that illuminates the whole restaurant and the other is a spot light the chef wears to inspect a plate. The spot light is worn on the chef’s head and looks toward a specific plate, as described below. You can make the spot light change direction rather than the chef. You do not need to make any scene geometry that corresponds to either light.
Selection
The user should be able to select the conveyor belts, sushi plates, special plates, sauce plates, and dessert plates, using the mobile device touchscreen. (Direct selection using the touchscreen can be achieved using the Ray object. See the Physics.Raycast and Physics.RaycastAll functions and the ray casting tutorial.) The ice cream scoops, trays, and tables do not need to be selectable.
Once an object has been selected, you should change it visually in some way to indicate this. (If you’d like, you can also indicate selection through audio or vibration, in addition to visually.) For example, the object’s texture could change to a texture that marks it as selected. Please think about which approach(es) would be most effective.
Initially, no object should be selected. At most one object should be in the selected state at a time. Touching a different selectable object than the currently selected object should deselect the currently selected object and select the newly touched object. (Note that touching
4
non-selectable items should not deselect the currently selected object.) You should also provide some way to deselect the currently selected object without selecting another object.
Control Panels
Using the Unity UI System, create a partially transparent control panel (see the Canvas documentation and the Canvas Manual) for each selectable object. These panels should be placed in a position of your own choosing. Each panel should be visible only w hile the object to which it belongs is selected. Thus, none of the object control panels should be visible when your application is initialized. The panels will contain controls (made with Interaction Components and Visual Components) for the parameters of the actions you assign to your objects (see next section for details).
There should also be a camera control panel to control the camera mode (see the discussion of the Camera below). The camera control panel should be initially visible and your user should be able to hide and unhide it.
Object Actions
Conveyor Belts
The conveyor belts act like a single object in terms of selection. When the user touches a conveyor belt, all four conveyor belts should be selected at the same time and the sushi, special, and dessert plates on them should stop moving relative to the belts. (However, sauce plates and ice cream scoops should continue to move relative to their plates.) The panel for the conveyor belts should appear, allowing the user to change the speed of the conveyor belts. All four conveyor belts should share the same speed (i.e., all plates on them should move at the same speed).
Sushi Plates, Special Plates, and Dessert Plates
When a special plate or a dessert plate i s selected, its sauce plates or ice cream scoop should
stop moving, pausing at their current poses until the plate has been deselected.
The user can drag a sushi plate, special plate, or dessert plate that is on a belt or tray, to another belt, tray, or table. Dragging a plate also causes it to be selected. Dragging a plate to a table will serve the (invisible) customers at that table. If the user drags a plate, the plate should follow the user’s finger as long as the finger stays in contact with the screen. The plate should stop moving when the user removes their finger from the screen.
When the user starts dragging the plate, a partially transparent, stationary sphere should be displayed, centered about the original position of the plate. The sphere indicates to the user the volume in which the plate is allowed to move and the plate should be prevented from moving outside the sphere. If after the user lifts their finger, the last position of the plate is directly on a table, belt, or tray, the plate should remain there. Otherwise, it should be destroyed, as described below, because the user dropped it or caused it to collide with another plate and it
5
broke. (To detect collision, please refer to the Unity documentation for Collider, including its messages such as Collider.OnTriggerEnter, and the Colliders as Triggers video tutorial.)
Recall that in Figure 1 the conveyor belts are not connected. Consequently, a plate that reaches the end of its belt should stop on the tray at that end. The user must manually drag that plate to the next conveyor belt before another plate arrives at that tray. If a second plate arrives at the tray before the first plate is moved off it, then both plates should be destroyed!
At most four plates (sushi, special, or dessert) can be on a table at the same time. If the user tries to drag a fifth plate onto that table, that plate should be destroyed (since the table is full and it is impolite to put the plate back on the belt). Each plate should remain on a table for no more than some specified amount of time (set by you), after which it should silently disappear, to indicate that it has been finished. This will allow new plates to be served. Use Time.deltaTime to implement a timer. You are welcome to display a countdown for each table.
If a plate travels all around the four belts back to the chef, it is considered stale and should also be destroyed.
If a plate is destroyed by being dropped, crashing into another plate, or staying on the belts too long (but not by being finished at a table), some visual and audio effects should be used beyond simply having the plate disappear. You should have some way to display the running totals of plates that were destroyed and plates that were finished since your application was invoked.
Chef
The chef should make new sushi, special, and dessert plates automatically at a rate set by the user, one plate at a time. When a new plate is made, it should spawn on the belt next to the chef. But please make sure the chef doesn’t put a new plate on an existing one! Use Prefabs for the sushi, special, and dessert plates.
The chef’s spot light should initially shine on the belt in front of them, where each plate is spawned. However, the chef is very proud of their creations. So, the first time the chef creates a plate, the chef’s spot light should shine on that plate, no matter where it is, until the plate disappears (has been eaten or destroyed). In general, the spot light should shine on the chef’s oldest plate. If at any time there is no plate, the spot light should shine on the belt in front of the chef. See Transform.LookAt to learn how to direct the spot light.
When the chef is selected, the user should be able to set the rate at which plates are spawned. However you do this, by default the chef should alternate between creating sushi, special, and dessert plates.
Sauce Plates
When a sauce plate is selected, it and the other sauce plate hovering over the same special plate should stop moving relative to the special plate, pausing at its current pose relative to the special plate until it has been deselected. Note that when a special plate itself is selected (and
6
its hovering sauce plates have paused), the sauce plates should still remain selectable, and if one of the sauce plates is selected, its special plate should be deselected and start moving again if it is on a belt.
The control panel for a sauce plate should allow the user to adjust the speed at which it orbits its special plate. By default, a sauce plate maintains the same orientation as its special plate. The control panel should also allow the user to set a speed and direction at which the sauce plate rotates on an axis passing through the center of the sauce plate and which is parallel to the local y-axis of the special plate.
Whenever an object resumes moving after it has been paused, it (and any descendants) should resume from its current position and orientation. That is, it should not make a discontinuous jump in position or orientation.
Difficulty Levels
There are two game difficulty levels, which the user should be able to switch between: easy and hard. In easy ( the default), plates should be moved to the next conveyor belt automatically when they arrive at the tray at the end of their belt, but the user still needs to drag plates onto tables. In hard, the user needs to drag plates from one conveyor belt or tray to the next belt.
Camera
Your user should be able to use the camera control panel to switch the camera between the following two modes:
Restaurant mode (the default): The camera should overlook the whole scene, and should be located and oriented such that everything in the scene is within its frustum. No explicit translation or rotation of the camera by the user is possible in this mode.
Player mode: In this mode, the user should be able to move and rotate the camera, so they can inspect any desired part of the scene. You should allow the user to dynamically determine camera yaw and pitch and the speed with which the camera translates in the direction in which it is looking. The user should be able to change yaw and pitch when the camera position is fixed.
Note that objects that are outside the bounds established by the camera’s near and far clipping planes will not be visible; so, please bear this in mind when setting the clipping planes.
Ten Usability Heuristics for User Interface Design
Your user interface should follow the Nielsen usability heuristics. We will not grade your work based on the aesthetics of its models (since we do not want you to buy any assets and do not assume you have any modeling skills) or on physical realism. However, your UI should still be easy to use and user-friendly.
7
(Optional) Mult-Touch
The user might want to use multiple fingers on one or both hands. To make this possible, you are optionally welcome to support multi-touch interaction. See Multi Touch Input for more information.
(Optional) Test Your Project on an Acer Windows Mixed Reality Headset
If you finish the assignment early, you are welcome to try to deploy your project on an Acer Windows Mixed Reality headset (for Windows only). Before doing this, please come to TA office hours, so we can first check the completeness of your project. If everything looks ok, we will let you sign out a headset, for which you will need to install SteamVR and the Windows Mixed Reality for SteamVR patch. Here is a high-level description of how you will then need to modify your project (we’ll provide more details later):
Camera
To enable the user to move and explore the environment in the scene, you will need to make the camera move with the headset. To do so, you will need to add a SteamVR CameraRig to the scene and make sure the Virtual Reality Supported option is enabled in Edit→Project Settings→Player→XR Settings.
Controllers
When wearing the headset, the user will no longer have a touchscreen to use for selection. Instead, they will use the two accompanying 6DoF controllers.
We will ask that you make a separate scene for use with the headset as part of your submission.
Hints
Before starting this assignment, please note that there is an extensive collection of Unity Tutorials. We strongly suggest that you review the Roll-a-Ball tutorial. Additionally, please look through the Unity Manual, to get a better feel for the Editor. Please see the Unity reference page on Input for a comprehensive overview of its functionality.
To do this assignment well, you should think carefully about how you structure your scene graph hierarchy. Which nodes should you use and how should they be arranged in the hierarchy relative to each other? (The relationship between a parent and its children is important, while the order in which siblings are listed should not matter for this assignment.) How should the transformations that you apply to your objects be composed to achieve the required effects? Begin with just one conveyor belt, one table, and the sushi plates to experimentally verify that your approach works, so you can modify it early on if necessary. Then add the rest of the belts (running at first in easy mode), selection, dragging, special plates and dessert plates (with simplified orbits and rotations of sauce plates and ice cream scoops at first).
8
Regarding hierarchy: Understanding rotation is crucial here. Note that when an object is rotated, its descendants will also rotate. Therefore, if you want object B to act as if it were a descendant of object A, but not be affected by A’ s rotation (e.g., to have A rotate at a different rate than B), the easiest way to do this is to create empty GameObject A′, make A a nd B both children of A′, where A i s centered at A′ and B i s offset from A′, and then rotate A and B individually. If you do this, transforming A′ w ill transform all its descendants, but A and B c an each have its own independent rotation.
Regarding model files: Each model file you find will, when you find it, most likely not be of an appropriate scale relative to the other objects you’re using. Therefore, be prepared to apply a scale transform to one or more of your models to bring them up or down to a reasonable size. In addition, note that some models may contain too many polygons for your mobile device to render your scene at a reasonable frame rate. Before you get too enamored of any model, please try it out on your device in context with the rest of your scene to make sure that it will work well. See also How do I fix the rotation of an imported model?
Regarding ray casting: When using the Physics.Raycast function, you will be returned a RaycastHit object. The RaycastHit object contains a reference to a Collider. The Collider contains a reference to the GameObject to which it is attached. You can use that reference to determine the object with which a ray cast through the screen collided.
Regarding rotation and orbiting: In Unity, the Transform.Rotate method specifies a relative orientation change that will be composed with the current orientation. Transform.Rotate needs to be given the amounts to rotate as Euler angles about x-, y-, and z-axes, either as three separate floats or as a Vector3. It will apply a rotation to the object about the z-a xis, x- axis, and y-axis (in that specific order, as discussed in class). You should also read about and use Transform.RotateAround. Please be sure that you understand the relativeTo parameter of Transform.Rotate and how Transform.{right,up,down} differ from Vector3.{right,up,down}. In strong contrast to the Transform.Rotate method, the Transform.{rotation,localRotation,eulerAngles,localEulerAngles} properties set the absolute orientation of a Transform (i.e., will override and ignore its current orientation) and do this in ways that offer many possibilities for you to do the wrong thing: none of the elements of Transform.{rotation,localRotation} is an angle (they are the components of a quaternion) and none o f the angles of Transform.{eulerAngles,localEulerAngles} should be set individually or incremented!
Regarding Play Mode: While Play Mode is a very useful way of debugging your app, it is not fully indicative of how it will run on your mobile device. Make sure you test fully on your device whenever you introduce a new imported asset. While debugging, you will want to include secondary controls that are guaranteed to work on desktop (or laptop) in Play Mode, such as Input.getMouseButton(). Since these controls may affect performance slightly, you will want to disable them when running on your mobile device. (Or if you feel comfortable with your app’s performance on your device, you can choose to support both modes.)
9
Regarding textures: When you load in a model and associated texture in two separate load steps, the texture might not connect to the model automatically. In order to connect the texture to the model, drag the texture onto the Texture box in the Material component of your model in the Inspector View.
Regarding transparency: It might be tempting to make some objects semitransparent. But please note that objects that have any portions that are not opaque are treated quite differently from opaque objects in how and when they are rendered, because of the hardware rendering algorithm used in current interactive graphics systems. This should not be a problem if you have only a single planar object that is not opaque. However, this can cause multiple nonopaque objects to render incorrectly and inconsistently as their positions change relative to each other and the camera. Since you will be using a transparent sphere to show the bounds within which a plate can move, we strongly suggest that you do not make any other objects transparent.
What to submit
Your submission should include:
● ●
The entire Unity project folder compressed.
○ Do not include the app executable (or the XCode project for iOS).
A README.txt file with:
○ Your name & UNI
○ Date of submission
○ Computer Platform
○ Mobile Platform & OS version & Device name
○ Project title
○ Project directory overview
○ Special instructions, if any, for deploying app
○ Instructions for using app
○ Missing features
○ Explanation of bugs in your code and Unity
○ Asset sources (cite everything you didn’t create yourself)
A brief v ideo demonstrating your application’s features. Please practice before recording it and do some simple editing to remove unnecessary pauses. Submit this as a link in your README file and in the CourseWorks File Upload comment to an unlisted v ideo on YouTube or Google Drive. The upload time of your video will be the time at which we will consider it to have been submitted.
●
10
How to submit
Please compress all files in your submission into a single zip file, remembering to include any needed data files. Please follow the naming convention “YOURUNI_Assignment1.zip” for your submission. Name your video “YOURUNI_Assignment1,” upload it as an unlisted video on YouTube and include the URL in your submission, as described below. Submission should be done through CourseWorks, following these steps:
1. Log into CourseWorks.
2. Select Assignments from the left-hand navigation pane.
3. Click the Submit Assignment button in the top right corner.
4. The Submit Assignments page will load. Choose your zipped project using the browse
dialog window that appears after pressing “Choose File.”
5. After choosing your project, copy the URL of your unlisted v ideo upload into the
Comments field beneath the File Upload section.
6. Press “Submit.”
Please try to submit well before the deadline, since CourseWorks can sometimes become busy and slow. You can resubmit multiple times. (Note: CourseWorks will save your previous comments, so you don’t need to re-enter your URL if it has not changed, but CourseWorks will clear your previous upload from the File Upload section.) You can add a file you previously uploaded by clicking “Click here to find a file you’ve already uploaded,” expanding the Unfiled folder and selecting your file, then pressing “Submit.”
Immediately after uploading your submission, please check it by downloading it, creating a new project with which to test it, and reading through its README.txt file. We will not accept an excuse that you accidentally uploaded the wrong file.
Remember, you can use only a single late day on this assignment, so start early! And, have fun! Grading
Object motion
Selection
Plate dragging
User interface (Usability heuristics) Cameras and lights Documentation (including video) Total
25 points 20 points 15 points 25 points 10 points 5 points 100 points
11