
## How to use the navigation mesh?

There are many cases to use a navigation mesh: AI and path finding, replace physics for collision detection (only allow player to go where it's possible instead of using collision detection) and many more cases theBabylon.js users will find.

First, create the navigation plugin

```javascript
let navigationPlugin = new BABYLON.RecastJSPlugin();
```

Prepare some parameters for the agent constraints (described below)

```javascript
const parameters = {
  cs: 0.2,
  ch: 0.2,
  walkableSlopeAngle: 35,
  walkableHeight: 1,
  walkableClimb: 1,
  walkableRadius: 1,
  maxEdgeLen: 12,
  maxSimplificationError: 1.3,
  minRegionArea: 8,
  mergeRegionArea: 20,
  maxVertsPerPoly: 6,
  detailSampleDist: 6,
  detailSampleMaxError: 1,
};
```

Call the navigation mesh generation with the parameters and the list of meshes

```javascript
navigationPlugin.createNavMesh([groundMesh, wallMesh1, wallMesh2, stair1, stair2], parameters);
```

And that's it! you can now use the navigation mesh with the crowd system or make queries.

Optionaly, you can get a display of the navmesh to ensure it corresponds to your space constraints

```javascript
navmeshdebug = navigationPlugin.createDebugNavMesh(scene);
const matdebug = new BABYLON.StandardMaterial("matdebug", scene);
matdebug.diffuseColor = new BABYLON.Color3(0.1, 0.2, 1);
matdebug.alpha = 0.2;
navmeshdebug.material = matdebug;
```

<Playground id="#KVQP83#0" title="Simple navigation mesh computation" description="Simple navigation mesh computation"/>

## Parameters

cs - The meshes are voxelized in order to compute walkable navmesh. This parameter in world unit define the widthand depth of 1 voxel.

ch - Same as cs but for height of the voxel.

walkableSlopeAngle - Angle in degree for the maximum walkable slop.

walkableHeight - The height in voxel units that is allowd to walk in.

walkableClimb - The delta in voxel units that can be climbed.

walkableRadius - the radius in voxel units of the agents.

maxEdgeLen - The maximum allowed length for contour edges along the border of the mesh. Voxel units.

maxSimplificationError - The maximum distance a simplified contour's border edges should deviate the original raw contour. Voxel units.

minRegionArea - The minimum number of cells allowed to form isolated island areas. Voxel units.

mergeRegionArea - Any regions with a span count smaller than this value will, if possible, be merged with larger regions. Voxel units.

maxVertsPerPoly - The maximum number of vertices allowed for polygons generated during the contour to polygon conversion process. Must be > 3.

detailSampleDist - Sets the sampling distance to use when generating the detail mesh. World units.

detailSampleMaxError - The maximum distance the detail mesh surface should deviate from heightfield data. World Units.

## Queries

Basically, query functions help at getting constraint point and vector by the navigation mesh.

```javascript
getClosestPoint(position: Vector3): Vector3;
getRandomPointAround(position: Vector3, maxRadius: number): Vector3;
moveAlong(position: Vector3, destination: Vector3): Vector3;
```

Respectively:

- get a point on the navmesh close to a world position parameter
- get a random world position, on the navmesh, inside a circle of maxRadius.
- constraint a segment by the navmesh and returns the ending world position. Like walking on the navmesh and stopping at the edge.

When the query can't find a valid solution, the value (0,0,0) is returned.

Those functions use a bounding box for querying the world. The solution returned is within that bound. To properly set the default box extent to get a finer or broader result, call:

```javascript
setDefaultQueryExtent(extent: Vector3): void;
```

If your query returns a point too far from the expected result, use a smaller extent.

It's possible to get a path built for navigation as a point array. It's up to the user to use this array for drawing prediction path, trigger events,...

```javascript
const pathPoints = navigationPlugin.computePath(crowd.getAgentPosition(agent), navigationPlugin.getClosestPoint(destinationPoint));
pathLine = BABYLON.MeshBuilder.CreateDashedLines("ribbon", { points: pathPoints, updatable: true, instance: pathLine }, scene);
```

## Baking result

Building a navigation mesh can take a lot of cpu and network resources. In order to lower the download size and cpu needed, it's possible to bake the result of the navigation mesh computation to a byte stream. That byte stream can later be restored to get the navigation mesh back.

To retrieve the binary representation of the computed navigation mesh:

```javascript
const binaryData = navigationPlugin.getNavmeshData();
```

binaryData is an Uint8Array that you can save to a file for example.
To restore an UInt8Array to a navigation mesh:

```javascript
navigationPlugin.buildFromNavmeshData(uint8array);
```

## Web Worker

Building a navigation mesh can be time and resource heavy. For many use cases, it can be necessary to delegate navmesh computation to a webworker. Work will be done in parallel, letting the engine display the content without slow downs.

To enable web worker, specify a web worker .js script URL to use:

```javascript
let navigationPlugin = new BABYLON.RecastJSPlugin();
navigationPlugin.setWorkerURL("workers/navMeshWorker.js");
```

A default web worker is provided at this URL : https://github.com/BabylonJS/Babylon.js/blob/master/packages/tools/playground/public/workers/navMeshWorker.js

Then, provide a completion callback to `createNavMesh` method. This callback will be called when the navigation mesh is computed and ready to use by the plugin.

```javascript
navigationPlugin.createNavMesh([staticMesh], navmeshParameters,(navmeshData) =>
{
    console.log("got worker data", navmeshData);
    navigationPlugin.buildFromNavmeshData(navmeshData);
    ...
```

`navMeshData` is a binary version, ready to serialize, of the the navmesh. It can be saved, streamed. User has to call `buildFromNavmeshData` to deserialize datas. Once the navmesh is fully loaded, it's possible to create crowd, query the navmesh,...

Performance note: The navmesh is constructed from geometry datas. If multiple meshes are needed, their geometry will be merged before passing the geometry positions and indices to Recast. This part of the code can be CPU intensive and cannot be done in a worker because of dependencies, copies, memory footprint.

An example of use with web worker : <Playground id="#TN7KNN#2" title="Navigation mesh computation with a web worker" description="Navigation mesh computation with a web worker"/>

## NPM

Loading Recast-Detour NPM module is different between version 1.3.0 and 1.4.0+ as later version is asynchronous. User has to use `await` like this:

```javascript
const recast = await Recast();
```



## Crowds and navigation agents

![a crowd ](/img/extensions/navigation/CrowdExample.png)

Now we have a navmesh, we can create autonomous agents and make them navigate within that navmesh constraint.
The agents will find the best path to that destination while avoinding other crowd agents.
An agent is attach to a Transform. That means that you have to attach a mesh to see them but also that you can attach pretty much anything.

A demo can be found at: <Playground id="#X5XCVT#240" title="Crowd and Navigation Agents" description="Crowd and navigation agents."/>

Click anywhere on the navmesh to make the agents go to that location.

## How to use it?

First thing is to create a crowd that all agents will belong to. Parameters are the maximum number of agents in the crowd, the maximum agent radius and the scene.

```javascript
const crowd = navigationPlugin.createCrowd(10, 0.1, scene);
```

Then to create an agent and attach it to a transform, call:

```javascript
const agentIndex = crowd.addAgent(position, agentParameters, transform);
```

And that's it! You will get a non moving agent. We now want to move it.

```javascript
crowd.agentGoto(agentIndex, navigationPlugin.getClosestPoint(endPoint));
```

This code will get the closest position on the navmesh to endPoint. Then it asks the agent to go to that position.
Depending on your agent parameters, it will get there faster of slower.

## Agent Parameters

radius - Radius of the agent. World Unit.

height - Heigh in World Unit.

maxAcceleration - Acceleration max in World Unit per second per second

maxSpeed - Max speed in World Unit per second.

collisionQueryRange - The agent collision system will take care of others within that radius in World Unit.

pathOptimizationRange - How the path will be optimized and made more straight.

separationWeight - How hard the system will try to separate the agent. A Value of 0 means it will not try and agents might collide.

You can update any of these parameters, per agent, by calling :

```javascript
// change speed and max speed
crowd.updateAgentParameters(agentIndex, { maxSpeed: 10, maxAcceleration: 200 });
```

## Teleport

You can teleport an agent to any position using this call:

```javascript
crowd.agentTeleport(agentIndex, navigationPlugin.getClosestPoint(destinationPoint));
```

Please note the navigation state is reseted when teleporting. You'll have to call `agentGoto` to choose a new destination.

## Agent orientation and next path target

Recastjs crowd system does not handle agent orientation. But the velocity is available and it's possible to orient the geometry toward it.
To do so, you will need to use Math.atan2 like in the following example. Please take care of the length of the velocity vector. If it's not big enough, you may encounter jittering.

```javascript
let velocity = crowd.getAgentVelocity(agentIndex);
if (velocity.length() > 0.2) {
  const desiredRotation = Math.atan2(velocity.x, velocity.z);
  // interpolate the rotation on Y to get a smoother orientation change
  ag.mesh.rotation.y = ag.mesh.rotation.y + (desiredRotation - ag.mesh.rotation.y) * 0.05;
}
```

In this PG <Playground id="#6AE0RP" title="Agent Orientation and Next Path Targeting" description="Example of agent orientation and next path targeting."/>

The agent's cube is oriented by the velocity and a grey little box is placed at the position of the next path corner.

## Agent reaching target Observer

An observable automaticaly fires when an agent reaches the destination (ie, is within radius of destination). The radius is by default the agent radius but it can be changed using `reachRadius` number propery in the `IAgentParameters` object.
If there are too many agents in the crowd trying to reach the same destination, a bottleneck can happen and few agents will reach destination. Be sure to properly set those values.
To add an observable, simply add your function:

```javascript
const crowd = navigationPlugin.createCrowd(10, 0.1, scene);
...
crowd.onReachTargetObservable.add((agentInfos) => {
    console.log("agent reach destination: ", agentInfos.agentIndex);
});
```



## Prerequisites

Internally, navigation mesh with support for obstacles differs from 'standard' use case with one static navigation mesh.
Obstacles mark tiles as dirty. Those tiles need to be reprocessed to compute a portion of the navigation mesh.
To do so, Recast introduces the concept of tiles. Each tile is a square portion of the nav mesh. Tiles a processed each frame.
Making huge changes processed of a couple of frames.

In order to build a Tiled nav mesh instead of a single mesh, add the following lines to the nav mesh creation parameters:

```javascript
const navmeshParameters = {
  cs: 0.2,
  ch: 0.2,
  walkableSlopeAngle: 0,
  walkableHeight: 0.0,
  walkableClimb: 0,
  walkableRadius: 1,
  maxEdgeLen: 12,
  maxSimplificationError: 1.3,
  minRegionArea: 8,
  mergeRegionArea: 20,
  maxVertsPerPoly: 6,
  detailSampleDist: 6,
  detailSampleMaxError: 15,
  borderSize: 1,
  tileSize: 20,
};
```

Note the addition of `tileSize`. It's the world unit size of each tile. If it's not present or has a value a zero, it falls back to the standard use and obstacles won't work.
Also, depending on your use case, this value must be carrefully chosen to trade between too many tiles and too much cpu intensive updates.

## Obstacles API

Once the navigation mesh is updated to take tiles into account, obstacles are accessible thru 3 simple functions:

```javascript
addCylinderObstacle(position: Vector3, radius: number, height: number): IObstacle;
addBoxObstacle(position: Vector3, extent: Vector3, angle: number): IObstacle;
removeObstacle(obstacle: IObstacle): void;
```

Keep a list of added obstacles in order to remove them later. An obstacle that's not colliding with the navigation mesh will have no influence.

<Playground id="#WCSDE1" title="Adding a door to a navigation mesh" description="Example of door simulation using nav mesh obstacles."/>



![Generated Navmesh ](/img/extensions/navigation/NavMeshGeneration.png)

A Navigation Mesh (or navmesh for short) is a surface topology describing the space where an agent can go based on constraints.
Based on parameters like agent radius, agent climbing capability, agent height,... the navmesh generation computes a topology from source meshes (the world geometry).
Then, this topology can have a debug display (blue mesh on screenshot above) to validate the parameters. 

A demo can be found at: <Playground id="#HFY257#4" title="Crowd Navigation Demo" description="Simple example showcasing the crowd agent and nav mesh systems."/>



# Image Filter Control

The Babylon.js Filter Control is a web control built on top of Babylon.js in order to apply filter to pictures in web pages with blazing fast speed.

## Introduction

Filtering images to apply effect might be tedious and slow on the CPU. The best place to do so (for a wide variety of effects) is on the GPU. But setting up an entire WebGL pipeline simply to process images might be tricky even more if you wish to benefit from WebGL 2 to 1 fallback and workaround famous platform issues.

To greatly simplify this task we introduced the `ImageFilter` Control.

![ImageFilter](/img/features/controls/imageFilter.png)

## How to use

### Installation

To begin with the image filter control, you first need to install the controls npm package.

```javascript
npm install @babylonjs/controls
```

To reduce the size of your web page, the controls library is based on the es6 version of `@babylonjs/core` used as a peer dependency. Therefore if you are not relying on it so far in you project, you also need to install core:

```javascript
npm install @babylonjs/core
```

### Instantiation

Once done, you can now import the control in your code:

```javascript
import { ImageFilter } from "@babylonjs/controls/imageFilter";
```

And simply instantiate it in your page:

```javascript
const imageFilter = new ImageFilter(filterCanvas);
```

You simply need to provide a canvas on which we will be able to use a WebGL context. You could as well provide another Babylon.js control in order to share the WebGL context.

Also you, would you need more specific engine configurations, or if you want to use post processes, you should manually use your own engine:

```javascript
const engine = new Engine(filterCanvas);
const imageFilter = new ImageFilter(engine);
```

By default the controls relies on ThinEngine in order to optimize your bundle but it might have some limitations you do not want to have as part of your experiences.

## Using Post Process

In order to apply a custom shader as the image filter, you can use the following code:

```javascript
const blackAndWhitePostProcess = new BlackAndWhitePostProcess("bw", 1, null, undefined, engine);
backAndWhiteFilter.filter(imageToProcess, blackAndWhitePostProcess);
```

Where imageToProcess could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This will apply the post process to the provided input and display it in the canvas.

## Using Custom shader

In order to apply an existing post process as the image filter, you can use the following code:

```javascript
const customEffectWrapper = new EffectWrapper({
    name: "Custom",
    engine: customFilter.engine,
    fragmentShader: `
        // Samplers
        varying vec2 vUV;
        uniform sampler2D textureSampler;
        
        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
        }
    `,
    samplerNames: ["textureSampler"]
});
customFilter.filter(imageToProcess, customEffectWrapper);
```

Where imageToProcess could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This will apply the custom shader in parameter to the picture. By default, `vUV` is available as a varying defining the full ouptput as texture coordinates. `textureSampler` needs to be present and defines the texture corresponding to the input parameter.

Please note that if you need to add custom unifoms or samplers, they should be defined in the effect wrapper list:

```javascript
const customEffectWrapper = new EffectWrapper({
    name: "Custom",
    engine: customFilter.engine,
    fragmentShader: `
        varying vec2 vUV;
        
        // Default Sampler
        uniform sampler2D textureSampler;

        // Custom uniforms
        uniform sampler2D otherTexture;
        uniform vec3 colorOffset;

        const vec2 scale = vec2(0.25, 1.);

        void main(void) 
        {
            gl_FragColor = texture2D(textureSampler, vUV);

            // Swizzle channels
            float r = gl_FragColor.r;
            gl_FragColor.r = gl_FragColor.b;
            gl_FragColor.b = r;
            gl_FragColor.rgb += clamp(colorOffset, 0., 1.);

            gl_FragColor.rgb *= texture2D(otherTexture, vUV * scale).rgb;
        }
    `,
    // Defines the list of existing samplers (default + customs).
    samplerNames: ["textureSampler", "otherTexture"],
    // Defines the list of existing uniform to be bound.
    uniformNames: ["colorOffset"],
});
```

Then you can simply bind them either in `onApply`:

```javascript
customEffectWrapper.onApplyObservable.add(() => {
    // Sets the custom values.
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", 0.2, 0, 0.2);
})
```

Or during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
    // Only render if the custom texture is ready (the default one is 
    // checked for you by the render function)
    if (!otherTexture.isReady()) {
        return;
    }

    // Sets the custom values.
    time += engine.getDeltaTime() / 1000;
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

    // Render. Please note we are using render instead of filter to improve 
    // performances of real time filter. filter is creating a promise and will therefore
    // generate some lags and garbage.
    customFilter.render(mainTexture, customEffectWrapper);
});
```

Finally, if you are relying on new textures, you need to wait for them to be ready before rendering.

## Process to the canvas

This is by far the simplest, if you have a canvas in your page. You simply need to use the following code to fit the provided element to the canvas size:

```javascript
imageFilter.filter(imageToProcess, filter);
```

On the previous line, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used)
* another canvas element (the current visible state will be used)

This is the default behavior.

## Process to a Babylon Texture

Instead of filtering directly to a canvas, you could prefer to only create a Babylon.js texture on the GPU. For this, you can use the following function:

```javascript
const texture = imageFilter.getFilteredTexture(imageToResize, { width: 128, height: 100 }, filter);
```

Like before, imageToResize could be either:

* the url of a picture.
* a video element (the current visible frame of the video will be used).
* another canvas element (the current visible state will be used).

You also need to provide the size you want your texture to have on the GPU.

Now you are free to use this texture with any other controls.

## Real Time filtering

Instead of filtering only one time, you might want to create dynamic real time effects. For this, you can simply render the effect during the render loop:

```javascript
// Rely on the underlying engine render loop to update the filter result every frame.
engine.runRenderLoop(() => {
    // Only render if the custom texture is ready (the default one is 
    // checked for you by the render function)
    if (!otherTexture.isReady()) {
        return;
    }

    // Sets the custom values.
    time += engine.getDeltaTime() / 1000;
    customEffectWrapper.effect.setTexture("otherTexture", otherTexture);
    customEffectWrapper.effect.setFloat3("colorOffset", Math.cos(time) * 0.5 + 0.5, 0, Math.sin(time) * 0.5 + 0.5);

    // Render. Please note we are using render instead of filter to improve 
    // performances of real time filter. filter is creating a promise and will therefore
    // generate some lags and garbage.
    customFilter.render(mainTexture, customEffectWrapper);
});
```

This requires to use the render function instead of the filter one to enhance your experience performances.

## Full Code Sample

You can find the integrallity of the code sample above on [Github](https://github.com/BabylonJS/Controls/blob/master/www/imageFilter/index.ts) if you want to see it in action and better see how some of the functionnalities could be used.

## Live Demo

Please, have a look at the [Live Image Filter Demo](https://controls.babylonjs.com/imageFilter) to better appreciate how it works.


Crowd Navigation System

Coming next

Creating A Navigation Mesh

Learn how to create a mesh as a confinment system for crowd agents.

Crowd Agents

Learn how to create crowd agents that follow a path along a navigation mesh.

Adding and removing obstacles

Learn how to add and remove obstacles for crowd agents.