Interaction

RedGPU provides an intuitive Picking system that handles mouse and touch events for both 3D and 2D objects, as well as a Keyboard Buffer (keyboardKeyBuffer) feature that manages keyboard states in real-time. This allows you to easily handle various user inputs, from simple mouse clicks to complex character controls.

1. Mouse and Touch Interaction (Picking)

RedGPU's picking system enables user input reception and reaction for most display objects, such as Mesh, Sprite3D, and Sprite2D.

1.1 Registering Event Listeners

To register events on an object, use the addListener method. When an event occurs, the registered callback function is executed, and an event information object (e) is passed to it.

import * as RedGPU from "https://redcamel.github.io/RedGPU/dist/index.js";

// Reference event type constants
const { PICKING_EVENT_TYPE } = RedGPU.Picking;

mesh.addListener(PICKING_EVENT_TYPE.CLICK, (e) => {
    console.log('Clicked object:', e.target);
    // Rotate upon click
    e.target.rotationY += 45;
});

1.2 Supported Event Types

The system supports the 6 basic events defined in RedGPU.Picking.PICKING_EVENT_TYPE.

Event Constant	String Value	Description
CLICK	'click'	Occurs on a mouse click or touch tap
DOWN	'down'	Occurs when a mouse button is pressed or touch starts
UP	'up'	Occurs when a mouse button is released or touch ends
OVER	'over'	Occurs when the mouse cursor enters the object's area (Hover In)
OUT	'out'	Occurs when the mouse cursor leaves the object's area (Hover Out)
MOVE	'move'	Occurs continuously as the mouse pointer moves over the object

1.3 Detailed Event Information (PickingEvent)

The object (e) passed to the event callback contains various pieces of information about the event at the moment it occurred. This allows for the implementation of precise interaction logic.

Property Name	Type	Description
target	Mesh	The target object where the event occurred.
type	string	The type of the event that occurred.
pickingId	number	A unique ID used for picking.
mouseX, mouseY	number	Mouse or touch coordinates within the canvas.
movementX, movementY	number	The amount of mouse movement compared to the previous event.
point	vec3	Precise intersection point coordinates in world space.
localPoint	vec3	Intersection point coordinates in the object's local space.
localX, localY, localZ	number	Individual coordinate values in local space.
uv	vec2	Texture coordinates (UV) at the intersection point.
distance	number	The distance between the camera and the intersection point.
faceIndex	number	The index of the intersected triangle (polygon). (-1 if not available)
time	number	The time when the event occurred (in ms).
altKey, ctrlKey, shiftKey	boolean	The state of modifier keys at the time of the event.

1.4 Practical Example: Interactive Cube

This is an interactive example where the size changes on mouse-over, the color changes upon clicking, and coordinates can be checked while moving the mouse.

2. Keyboard Interaction (keyboardKeyBuffer)

In addition to object picking, you can check the real-time state of the keyboard via redGPUContext.keyboardKeyBuffer. This is very useful for logic that needs to check key states every frame, such as for character movement or camera control.

2.1 Basic Usage

keyboardKeyBuffer is an object that uses the name of the currently pressed key as its key and its pressed state as its value.

RedGPU.init(canvas, (redGPUContext) => {
    // Check state within the render loop
    const render = (time) => {
        const { keyboardKeyBuffer } = redGPUContext;

        if (keyboardKeyBuffer['w'] || keyboardKeyBuffer['W']) {
            console.log('Moving forward...');
        }
        if (keyboardKeyBuffer[' ']) {
            console.log('Jump!');
        }
    };
    
    const renderer = new RedGPU.Renderer();
    renderer.start(redGPUContext, render);
});

2.2 Key Features

• Real-time State Management: Integrated management is provided within redGPUContext, eliminating the need to directly manage separate keydown and keyup listeners.
• Case Sensitivity: Since it uses the e.key value directly, it is case-sensitive. For universal input, it is recommended to check both 'w' and 'W'.
• Modifier Key Support: The state of special keys such as Shift, Control, and Alt can be checked in the same way.

3. Live Examples

You can directly explore various interaction behaviors provided by RedGPU through the examples below.

3.1 Mouse and Touch Examples

• Basic Mesh Interaction
• Sprite Interaction
• Text Field Interaction
• High-Precision Raycasting (Raycasting)

3.2 Keyboard Interaction Examples

• Keyboard Interaction 3D Example
• Keyboard Interaction 2D Example
• Character Controller (WASD)

Key Summary

• Independent event processing for each object is possible through addListener.
• Precise, frame-by-frame keyboard state control is possible through keyboardKeyBuffer.
• A rich user experience (UX) can be provided by using these alongside the web-standard DOM API.

Next Learning Recommendation

Apply post-processing effects that enhance the visual completion of scenes enriched with interaction.

• Post-Effect