
# Image Filter Control

The Babylon.js Filter Control is a web control built on top of Babylon.js in order to apply filter to pictures in web pages with blazing fast speed.

## Introduction

Filtering images to apply effect might be tedious and slow on the CPU. The best place to do so (for a wide variety of effects) is on the GPU. But setting up an entire WebGL pipeline simply to process images might be tricky even more if you wish to benefit from WebGL 2 to 1 fallback and workaround famous platform issues.

To greatly simplify this task we introduced the `ImageFilter` Control.

![ImageFilter](/img/features/controls/imageFilter.png)

## How to use

### Installation

To begin with the image filter control, you first need to install the controls npm package.

```javascript
npm install @babylonjs/controls
```

To reduce the size of your web page, the controls library is based on the es6 version of `@babylonjs/core` used as a peer dependency. Therefore if you are not relying on it so far in you project, you also need to install core:

```javascript
npm install @babylonjs/core
```

### Instantiation

Once done, you can now import the control in your code:

```javascript
import { ImageFilter } from "@babylonjs/controls/imageFilter";
```

And simply instantiate it in your page:

```javascript
const imageFilter = new ImageFilter(filterCanvas);
```

You simply need to provide a canvas on which we will be able to use a WebGL context. You could as well provide another Babylon.js control in order to share the WebGL context.

Also you, would you need more specific engine configurations, or if you want to use post processes, you should manually use your own engine:

```javascript
const engine = new Engine(filterCanvas);
const imageFilter = new ImageFilter(engine);
```

By default the controls relies on ThinEngine in order to optimize your bundle but it might have some limitations you do not want to have as part of your experiences.

## Using Post Process

In order to apply a custom shader as the image filter, you can use the following code:

```javascript
const blackAndWhitePostProcess = new BlackAndWhitePostProcess("bw", 1, null, undefined, engine);
backAndWhiteFilter.filter(imageToProcess, blackAndWhitePostProcess);
```

Where imageToProcess could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This will apply the post process to the provided input and display it in the canvas.

## Using Custom shader

In order to apply an existing post process as the image filter, you can use the following code:

```javascript
const customEffectWrapper = new EffectWrapper({
    name: "Custom",
    engine: customFilter.engine,
    fragmentShader: `
        // Samplers
        varying vec2 vUV;
        uniform sampler2D textureSampler;
        
        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
        }
    `,
    samplerNames: ["textureSampler"]
});
customFilter.filter(imageToProcess, customEffectWrapper);
```

Where imageToProcess could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This will apply the custom shader in parameter to the picture. By default, `vUV` is available as a varying defining the full ouptput as texture coordinates. `textureSampler` needs to be present and defines the texture corresponding to the input parameter.

Please note that if you need to add custom unifoms or samplers, they should be defined in the effect wrapper list:

```javascript
const customEffectWrapper = new EffectWrapper({
    name: "Custom",
    engine: customFilter.engine,
    fragmentShader: `
        varying vec2 vUV;
        
        // Default Sampler
        uniform sampler2D textureSampler;

        // Custom uniforms
        uniform sampler2D otherTexture;
        uniform vec3 colorOffset;

        const vec2 scale = vec2(0.25, 1.);

        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
            gl_FragColor.rgb += clamp(colorOffset, 0., 1.);

            gl_FragColor.rgb *= texture2D(otherTexture, vUV * scale).rgb;
        }
    `,
    // Defines the list of existing samplers (default + customs).
    samplerNames: ["textureSampler", "otherTexture"],
    // Defines the list of existing uniform to be bound.
    uniformNames: ["colorOffset"],
});
```

Then you can simply bind them either in `onApply`:

```javascript
customEffectWrapper.onApplyObservable.add(() => {
    // Sets the custom values.
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", 0.2, 0, 0.2);
})
```

Or during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
    // Only render if the custom texture is ready (the default one is 
    // checked for you by the render function)
    if (!otherTexture.isReady()) {
        return;
    }

    // Sets the custom values.
    time += engine.getDeltaTime() / 1000;
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

    // Render. Please note we are using render instead of filter to improve 
    // performances of real time filter. filter is creating a promise and will therefore
    // generate some lags and garbage.
    customFilter.render(mainTexture, customEffectWrapper);
});
```

Finally, if you are relying on new textures, you need to wait for them to be ready before rendering.

## Process to the canvas

This is by far the simplest, if you have a canvas in your page. You simply need to use the following code to fit the provided element to the canvas size:

```javascript
imageFilter.filter(imageToProcess, filter);
```

On the previous line, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This is the default behavior.

## Process to a Babylon Texture

Instead of filtering directly to a canvas, you could prefer to only create a Babylon.js texture on the GPU. For this, you can use the following function:

```javascript
const texture = imageFilter.getFilteredTexture(imageToResize, { width: 128, height: 100 }, filter);
```

Like before, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used).
* another canvas element (the current visible state will be used).

You also need to provide the size you want your texture to have on the GPU.

Now you are free to use this texture with any other controls.

## Real Time filtering

Instead of filtering only one time, you might want to create dynamic real time effects. For this, you can simply render the effect during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
    // Only render if the custom texture is ready (the default one is 
    // checked for you by the render function)
    if (!otherTexture.isReady()) {
        return;
    }

    // Sets the custom values.
    time += engine.getDeltaTime() / 1000;
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

    // Render. Please note we are using render instead of filter to improve 
    // performances of real time filter. filter is creating a promise and will therefore
    // generate some lags and garbage.
    customFilter.render(mainTexture, customEffectWrapper);
});
```

This requires to use the render function instead of the filter one to enhance your experience performances.

## Full Code Sample

You can find the integrallity of the code sample above on [Github](https://github.com/BabylonJS/Controls/blob/master/www/imageFilter/index.ts) if you want to see it in action and better see how some of the functionnalities could be used.

## Live Demo

Please, have a look at the [Live Image Filter Demo](https://controls.babylonjs.com/imageFilter) to better appreciate how it works.



# Resizer Control

The Babylon.js Resizer Control is a web control built on top of Babylon.js in order to accelerate the resizing of pictures in web pages with blazing fast speed.

## Introduction

It regularly happens on the web that one needs to resize a picture on demand. It is easy enough to do with static pictures and css, but what about capturing a frame from a video or a snapshot from a canvas ?

Also this component can come in as a pair with the [Timeline](./timeline) and helps resizing thumbnails before uploading them to the GPU without relying on a separate canvas element and thus preventing the extra copies required.

![Resizer](/img/features/controls/resizer.png)

## How to use

### Installation

To begin with the timeline, you first need to install the controls npm package.

```javascript
npm install @babylonjs/controls
```

To reduce the size of your web page, the controls library is based on the es6 version of `@babylonjs/core` used as a peer dependency. Therefore if you are not relying on it so far in you project, you also need to install core:

```javascript
npm install @babylonjs/core
```

Please note, that while the controls are still in preview, it might be preferable to use their latest update with:

```javascript
npm install @babylonjs/controls@preview
npm install @babylonjs/core@preview
```

### Instantiation

Once done, you can now import the resizer control in your code:

```javascript
import { Resizer } from "@babylonjs/controls/resizer";
```

And simply instantiate it in your page:

```javascript
const resizer = new Resizer(resizerCanvas);
```

You simply need to provide a canvas on which we will be able to use a WebGL context. You could as well provide another Babylon.js control in order to share the WebGL context.

## Resizing to the canvas

This is by far the simplest, if you have a canvas in your page. You simply need to use the following code to fit the provided element to the canvas size:

```javascript
resizer.resize(imageToResize);
```

On the previous line, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

The source image will be stretch in both direction to fit in the target canvas.

## Resizing to a Babylon Texture

Instead of resizing directly to a canvas, you could prefer to only create a Babylon.js texture on the GPU. For this, you can use the following function:

```javascript
const texture = resizer.getResizedTexture(imageToResize, { width: 128, height: 100 });
```

Like before, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used).
* another canvas element (the current visible state will be used).

You also need to provide the size you want your texture to have on the GPU.

Now you are free to use this texture with any other controls.

## How to use with the timeline

Creating the texture is one thing but being able to use it is even better.

For instance you can simply pair the use of the resizer with the timeline:

```javascript
const resizer = new Resizer(canvas);
const timeline = new Timeline(resizer, {
    [...]
    getThumbnailCallback: (time: number) => {
        hiddenVideo.currentTime = time;
        return resizer.getResizedTexture(hiddenVideo, { width: 128, height: 100 });
    }
});
```

The 2 important points are:

* Use a shared control context by passing one control to the constructor of the other one.
* Returning a resized texture from the thumbnail callback.

That is it, both controls are linked together making an efficient use of the GPU without wasting resources with bigger thumbnails than needed.

This is actually the technique we are relying on in the demo. The source can be seen on [Github](https://github.com/BabylonJS/Controls/blob/master/www/timeline/index.ts)

## Full Code Sample

You can find the integrallity of the code sample above on [Github](https://github.com/BabylonJS/Controls/blob/master/www/resizer/index.ts) if you want to see it in action and better see how some of the functionalities could be used.

## Live Demo

Please, have a look at the [Live Resizer Demo](https://controls.babylonjs.com/resizer) to better appreciate how it works.



# Video Timeline Control

The Babylon.js Timeline Control is a web control built on top of Babylon.js that accelerates some of the typical features associated with video timelines.

## Introduction
One of the inherent issue encountered while scrolling in a video timeline built from IMG tags is perfomance caveats. The bigger the number of pictures to scroll through, the slower and less smooth the interactions with the web page would be.

In order to increase the reactivity of the web page, we rely here on WebGL through Babylon.js to create an easy to reuse control displaying a video timeline.

![Timeline](/img/features/controls/timeline.png)

## How to use

### Installation
To begin with the timeline, you first need to install the controls npm package.

```
npm install @babylonjs/controls
```

To reduce the size of your web page, the controls library is based on the es6 version of `@babylonjs/core` used as a peer dependency. Therefore if you are not relying on it so far in you project, you also need to install core:

```
npm install @babylonjs/core
```

### Instantiation
Once done, you can now import the timeline in your code:

```
import { Timeline } from "@babylonjs/controls/timeline";
```

And simply instantiate it in your page:

```
const timeline = new Timeline(timelineCanvas, {
        totalDuration: 60,
        thumbnailWidth: 128,
        thumbnailHeight: 120,
        loadingTextureURI: "./assets/loading.png",
        getThumbnailCallback: (time: number, done: (input: any) => void) => {
            // This is strictly for demo purpose and should not be used in prod as it creates as many videos
            // as there are thumbnails all over the timeline.
            const hiddenVideo = document.createElement("video");
            document.body.append(hiddenVideo);
            hiddenVideo.style.display = "none";

            hiddenVideo.setAttribute("playsinline", "");
            hiddenVideo.muted = true;
            hiddenVideo.autoplay = navigator.userAgent.indexOf("Edge") > 0 ? false : true;
            hiddenVideo.loop = false;

            hiddenVideo.onloadeddata = () => {
                if (time === 0) {
                    done(hiddenVideo);
                }
                else {
                    hiddenVideo.onseeked = () => {
                        done(hiddenVideo);
                    }
                    hiddenVideo.currentTime = time;
                }
            }

            hiddenVideo.src = "./assets/test.mp4?" + time;
            hiddenVideo.load();
        }
    });
```

The main element to provide is a canvas on which we will be able to use a WebGL context to render the timeline. You could as well provide another Babylon.js control in order to share the WebGL context. For instance you could share the context this way with a Resizer in order to allow directly using a resized image as a thumbnail without extra copy.

As we do not want to be opiniated about the UX you prefer or about the thumbnail generation itself it is all left to your implementation. The only information you need to provide are:

* *totalDuration*: the full duration of the video to cover (use to compute when all the thumbnails should be generated).
* *thumbnailWidth*: define the width of your generated thumbnails (use during the rendering to know how much space they should take in the timeline)
* *thumbnailHeight*: define the height of your generated thumbnails (use to keep the ratio intact with the width previously defined)
* *loadingTextureURI*: define the url of an image used as a temporary replacement for not fully loaded thumbnail (this will help handling network latency gracefully)
* *getThumbnailCallback*: last, but definitely not the least, a callback where you will be able to provide the thumbnail fitting with the requested time in parameter. You can return through the done function either a video element set on the right frame (as done in the previous code sample), a canvas element containing for instance some pre processed data or some image URL. More choices will probably be added here to for instance support texture atlas as it might be a nice transport format.

You can also provide some none mandatory options:
* *useClosestThumbnailAsLoadingTexture*: it is set to true by default and if enabled, it will try to use the closest loaded thumbnail (in time) instead of the loading one as soon as some of them are ready.

### Render Loop

Finally, you can start rendering the thumbnails:

```
    timeline.runRenderLoop();
```

It is also possible to pass a custom callback to this method to ensure that some custom code would run every frame before rendering the timeline:

```
    timeline.runRenderLoop(() => {
        // Sync the timeline with a playing video
        if (!mainVideo.paused) {
            timeline.setCurrentTime(mainVideo.currentTime);
        }
    });
```

### Changing the current zoom level

By default, the timeline is not zoomed at all, it shows as many thumbnails as fit in the canvas distributed evenly along the entire video.

You can zoom until you see one thumbnail per chosen time unit of the video (so if the total duration is 60 and you are zoomed at 100%, you will be able to see 60 thumbnails in total when scrolling from left to right).

To zoom in/out, you can call the following code:

```
    // value is the percentage of zoom desired 0 for the entire video 100 for one thumbnail per unit
    timeline.setVisibleDurationZoom(value);
```

### Changing the current time

As changing the zoom level does not change the number of visible thumbnails, you need a way to control where you currently are in the list of thumbnails. The following code can be use to do so:

```
    // Sets the left side of the canvas to the current chosen time
    // The rest of the visible thumbnails are evenly distributed according to the chosen
    // zoom level.
    timeline.setCurrentTime(value);
```

The current time will be automatically bound between 0 and the max available position.

### Cache Warm Up

In case you have some of the thumbnails ready before waiting for the callback, you can submit them into the cache like this:

```
    timeline.addThumbnail(textureData, time);
```

The textureData type are the same allowed to be returned in the callback: a texture, a video, a canvas or an image url.

This can help if you generate the thumbnails client side to warm up the cache and thus reduce the thumbnails load time.

## Full Code Sample

You can find the integrallity of the code sample above on [Github](https://github.com/BabylonJS/Controls/blob/master/www/timeline/index.ts) if you want to see it in action and better see how some of the functionnalities could be used.

## Live Demo

Please, have a look at the [Live Timeline Demo](https://controls.babylonjs.com/timeline) to better appreciate how it works.



# Controls

Babylon.js Controls are a set of regular web controls that are super-charged with hardware accelerated rendering through Babylon.js to provide blazing fast dedicated controls.

Babylon.js provides an unified API on top of **WebGL**, **WebGL2** and **WebGPU** that controls can leverage to unleash the raw power of your GPU.

![Resizer](/img/features/controls/resizer.png)


Sample https://github.com/sebavan/BabylonjsVideoProcessingSample


# How to use the WebVR experience helper

## WebVR vs WebXR

While the WebVR experience helper will continue to work, **it is strongly recommended that new projects use the [WebXR experience helper](/features/featuresDeepDive/webXR/webXRExperienceHelpers)**. For more information, check out our [introduction to WebXR](/features/featuresDeepDive/webXR/introToWebXR).

## Introduction

The WebVR Experience Helper provides a quick way to add WebVR support to a Babylon scene.

Features include:

1. WebVR camera and non-WebVR camera initialization
1. Enter WebVR button
1. Teleportation and rotation in the world
1. Gaze tracking with mesh selection from HMD and controllers

## Setup

A VRExperienceHelper can be created directly from the scene.

```javascript
const scene = new BABYLON.Scene(engine);
const vrHelper = scene.createDefaultVRExperience();
```

This will initialize a WebVR camera and a non-WebVR camera in the scene. It will also create an enterVR button at the bottom right of the screen which will start rendering to the HMD on click.

### Options

- createDeviceOrientationCamera(default: true): If the non-WebVR camera should be created. To use an existing camera, create it and then initialize the helper with this set to false in the constructor.
- createFallbackVRDeviceOrientationFreeCamera(default: true): When no HMD is connected, this flag specifies if the VR camera should fallback to a VRDeviceOrientationFreeCamera which will render each eye on the screen. This can be set to false to only enable entering VR if an HMD is connected.

## Detect if fallback orientation camera is supported

If a webVR capable device is not detected Babylon will fallback to using a vrDeviceOrientationCamera however device orientation will only be available if the device has an orientation sensor available. In the latest version of Safari, the orientation sensor is disabled by default and it does not prompt users to enable it in settings so currently this must be done by the app. See https://www.applemust.com/how-and-why-to-use-motion-orientation-settings-in-ios/

```javascript
vrHelper.onAfterEnteringVRObservable.add(() => {
  if (scene.activeCamera === vrHelper.vrDeviceOrientationCamera) {
    BABYLON.FreeCameraDeviceOrientationInput.WaitForOrientationChangeAsync(1000)
      .then(() => {
        // Successfully received sensor input
      })
      .catch(() => {
        alert("Device orientation camera is being used but no sensor is found, prompt user to enable in safari settings");
      });
  }
});
```

See it in action here: <Playground id="#TAFSN0#230" title="Fallback Orientation Camera Example" description="A simple example of detecting if a fallback orienation camera is supported." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper1.jpg"/>

## Teleportation and Rotation

To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.

```javascript
const ground = BABYLON.MeshBuilder.CreateGround("ground", { width: 6, height: 6, subdivisions: 2 }, scene);
vrHelper.enableTeleportation({ floorMeshName: "ground" });
```

To teleport, hold up on the joystick to display where the user will be teleported to and then release to teleport.
To rotate, move the joystick to the left or to the right.

When WebVR controllers are connected, the teleportation will be based on where the controller is pointing.

When WebVR controllers are not connected, the user will teleport to where the user is looking and teleportation can be triggered with an Xbox controller.

## Teleportation events

Teleportation has two observables you can subscribe to:

onBeforeCameraTeleport: Observable raised when teleportation is requested, receiving target Vector3 position as parameter:

```javascript
vrHelper.onBeforeCameraTeleport.add((targetPosition) => {
  //Raised before camera is teleported
});
```

onAfterCameraTeleport: Observable raised when teleportation animation finishes, receiving target Vector3 position as parameter:

```javascript
vrHelper.onAfterCameraTeleport.add((targetPosition) => {
  //Raised after teleportation animation finishes
});
```

To enable teleportation in the scene, create a mesh that the user should be able to teleport to and then enable teleportation with that mesh's name.

```javascript
const ground = BABYLON.MeshBuilder.CreateGround("ground", { width: 6, height: 6, subdivisions: 2 }, scene);
vrHelper.enableTeleportation({ floorMeshName: "ground" });
```

## Enabling / disabling teleportation

Teleportation can be enabled or disabled on demand by using the property teleportationEnabled:

```javascript
// Enable teleportation
vrHelper.teleportationEnabled = true;

//Disable teleportation (teleportation mesh will not be displayed)
vrHelper.teleportationEnabled = false;
```

To customize the teleportation target mesh the following property can be set to the mesh you'd like to use:

```javascript
vrHelper.teleportationTarget = BABYLON.MeshBuilder.CreateSphere("sphere1", { segments: 4, diameter: 0.1 }, scene);
```

## Accessing cameras

The VR and non-VR camera can be accessed from the helper to handle any application specific logic.

```javascript
// Initial camera before the user enters VR
vrHelper.deviceOrientationCamera;
// WebVR camera used after the user enters VR
vrHelper.webVRCamera;
// One of the 2 cameras above depending on which one is in use
vrHelper.currentVRCamera;
```

## Accessing controllers

The controllers can be accessed from the helper to handle any application specific logic.

```javascript
vrHelper.onControllerMeshLoaded.add((webVRController) => {
  const controllerMesh = webVRController.mesh;
  webVRController.onTriggerStateChangedObservable.add(() => {
    // Trigger pressed event
  });
});
```

Please note that the microsoft controllers are using the GLB file format and require the [GLTF Loader](/features/featuresDeepDive/importers/glTF).

## Accessing vr device position and rotation

Position and rotation in Babylon space can be accessed through the webVRCamera's devicePosition and deviceRotationQuaternion

```javascript
// Left and right hand position/rotation
if (vrHelper.webVRCamera.leftController) {
  leftHand.position = vrHelper.webVRCamera.leftController.devicePosition.clone();
  leftHand.rotationQuaternion = vrHelper.webVRCamera.leftController.deviceRotationQuaternion.clone();
}
if (vrHelper.webVRCamera.rightController) {
  rightHand.position = vrHelper.webVRCamera.rightController.devicePosition.clone();
  rightHand.rotationQuaternion = vrHelper.webVRCamera.rightController.deviceRotationQuaternion.clone();
}

// Head position/rotation
head.position = vrHelper.webVRCamera.devicePosition.clone();
head.rotationQuaternion = vrHelper.webVRCamera.deviceRotationQuaternion.clone();
```

See an Example here: <Playground id="#VIGXA3#7" title="Accessing VR Device position and rotation Example" description="A simple example of accessing a vr device's position and rotation."/>

## Gaze and interaction

Gaze and interactions can be enabled through the enableInteractions method. See Example: <Playground id="#JA1ND3#40" title="Gaze and Interactions Example" description="A simple example of handling VR Gaze and Interaction." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper2.jpg"/>

```javascript
vrHelper.enableInteractions();
```

This will start casting a ray from either the user's camera or controllers. Where this ray intersects a mesh in the scene, a small gaze mesh will be placed to indicate to the user what is currently selected.

Please note the gaze controllers will simulate pointer events so `scene.onPointerObservable` will be raised when gaze is enabled.

To filter which meshes the gaze can intersect with, the raySelectionPredicate can be used:

```javascript
vrHelper.raySelectionPredicate = (mesh) => {
  if (mesh.name.indexOf("Flags") !== -1) {
    return true;
  }
  return false;
};
```

This will cause the user's gaze to pass through any mesh which results in the raySelectionPredicate returning false.

As the user moves between meshes with their gaze, the onNewMeshSelected event will occur.
Note: This only works after interactions have been enabled.

```javascript
vrHelper.onNewMeshSelected.add((mesh) => {
  // Mesh has been selected
});
```

This will return the single closest mesh that was selected.

Prior to onNewMeshSelected an event called onNewMeshPicked is raised when a mesh is selected based on meshSelectionPredicate successful evaluation. This observable notifies a PickingInfo object to subscribers.

```javascript
vrHelper.onNewMeshPicked.add((pickingInfo) => {
  //Callback receiving ray cast picking info
});
```

As the user unselects a mesh with their gaze or controller, the onSelectedMeshUnselected event will occur.

```javascript
vrHelper.onSelectedMeshUnselected.add((mesh) => {
  // Mesh has been unselected
});
```

You can add your own filtering logic with meshSelectionPredicate.
Note: This will be applied after the raySelectionPredicate.

```javascript
vrHelper.meshSelectionPredicate = (mesh) => {
  if (mesh.name.indexOf("Flags01") !== -1) {
    return true;
  }
  return false;
};
```

The logic order for raySelectionPredicate, meshSelectionPredicate, onNewMeshPicked, onNewMeshSelected are as followed:

1. Ray is casted from the controller
2. When the ray hits an object the raySelectionPredicate will be called and if true the ray will collide there and be stopped otherwise the ray will pass through the object
3. Teleportation target location is updated to where the ray collided if the collision is also a floor mesh
4. If the collision object was not collided with on the last frame meshSelectionPredicate is checked, if it returns true the onNewMeshPicked event is fired and then onNewMeshSelected is fired

The gaze tracker can be customized by setting the gazeTrackerMesh. <Playground id="#ZHYP5K" title="GazeTrackerMesh Example" description="A simple example showing how to use the gazeTrackerMesh." image="/img/playgroundsAndNMEs/vrglasses.png"/>

```javascript
vrHelper.gazeTrackerMesh = BABYLON.MeshBuilder.CreateSphere("sphere1", { segments: 4, diameter: 0.1 }, scene);
```

On specific devices like iOS (where fullscreen is not supported), you may want to set `vrHelper.enableGazeEvenWhenNoPointerLock = true` to let the gaze controller run even when not under fullscreen and pointer lock.

## Grab

By combining By combining WebVR controller method and add/removeChild method, you can grab objects by pressing trigger button.

```javascript
webVRController.onTriggerStateChangedObservable.add((stateObject) => {
  if (webVRController.hand == "left") {
    if (selectedMesh != null) {
      //grab
      if (stateObject.value > 0.01) {
        webVRController.mesh.addChild(selectedMesh);
        //ungrab
      } else {
        webVRController.mesh.removeChild(selectedMesh);
      }
    }
  }
});
```

Selected mesh is detected by onNewMeshSelected method.

```javascript
VRHelper.onNewMeshSelected.add(function (mesh) {
  selectedMesh = mesh;
});

VRHelper.onSelectedMeshUnselected.add(function () {
  selectedMesh = null;
});
```

See the example.

- <Playground id="#B4C2AH" title="Grabbing Objects Example" description="A simple example of how to grab objects using the WebVRController." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper3.jpg"/>

## Multiview

To improve rendering performance by up to 2x, try using [Multiview](/features/featuresDeepDive/cameras/multiViewsPart1) which will render both eyes in a single render pass

## Examples

Scenes:

- <Playground id="#JA1ND3#785" title="Sponza" description="Sponza scene." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper4.jpg"/>
- <Playground id="#JA1ND3#15" title="Mansion" description="Mansion scene." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper5.jpg"/>
- <Playground id="#JA1ND3#18" title="Hill Valley" description="Hill Valley scene." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper6.jpg"/>

Games:

- <Playground id="#32DWVS#4" title="Minecraft JMJ" description="Minecraft JMJ Game." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper7.jpg"/>
- <Playground id="#FAXLY2" title="Fruit Ninja" description="Fruit Ninja Game." image="/img/playgroundsAndNMEs/divingDeeperWebVRExperienceHelper8.jpg"/>


Controls

Video Overview

Further reading

Coming next

Timeline Control

The Babylon.js Timeline Control is a web control built on top of Babylon.js that accelerates some of the typical features associated with video timelines.

Resizer Control

The Babylon.js Resizer Control is a web control built on top of Babylon.js in order to accelerate the resizing of pictures in web pages with blazing fast speed.

Image Filter Control

The Babylon.js Filter Control is a web control built on top of Babylon.js in order to apply filter to pictures in web pages with blazing fast speed.