Tutorial 4: rendering your own stuff

screenshot of tutorial4

We now go in the meat of these tutorials. That's the real stuff. The one for big boys. This is where we will try to get a good understanding of the rendering process in Ogre and see how we can act upon it.

This tutorial is broken for the moment (code won't compile). With the new release (Dagon 1.2.1), I have to update the code (done for previous tutorials but not for this one yet). Moreover, with Dagon, some content of this tutorial is now obsolete. I have to think about the way to update. Should be done pretty soon (couple of days).

Step A: understanding geometry in Ogre

Up to now we have used existing meshes that were build outside Ogre and exported to the .mesh format. Ogre provided the loading and rendering mechanisms through entities. That's great for populating the scene. But during rendering, there may be situations where you would want to generate some geometry on the fly. A good example is shadow quads in stenciled shadow volumes. We have already used the MeshManager in previous tutorial to create a simple plane. We will now see how to use this class to create meshes in a general way. For that I must first lecture you about meshes in Ogre. Fasten you seat belt and let's go.

Vertex data and materials

Let's start with how 3D models are described in Ogre. The most efficient way to describe 3D data for the graphic card is indexed face sets. This representation is composed of :

This decouples the vertices from the face connectivity, the benefit being that the graphic card can process vertices only once although the vertex is "traversed" several times for different faces it belongs to

Another important decoupling occurs. The indexed face set is just vertex and index data. To render it, we must also describe how to use this data. For example, we must indicate which texture the texture coordinates refer to. In Ogre's terminology, it means we must specify the material the model is to use with. The terminology might be confusing to you as some of the attributes -- for example color -- might be what you naturally think of as the material. What I can suggest you is to think of the material as the program indicating how the data is to be processed to render the model. Indeed, today's hardware offers the possibility to re-program the standard pipeline so semantic (i.e. vertex position, vertex color) inherited from this standard pipeline can be dropped in favor of generic "vertex attributes" (although we ususally stick to the old semantic as mnemotechnic helpers). Besides complex vertex or fragment programs, rendering may also involve several passes so material is really an important concept. We'll discuss it a bit more later.

To summarize, you can think of the vertex/index data as the "geometry" and of the material as the "appearance" although this vision of things is outdated with programmable hardware What you really have are "vertex attributes" send to "programs". By the way vertex and fragment programs are historically called shaders as they are typically used for (and were designed to) specify complex appearances (aka shading). Personnally I really see them as programs (and many people indeed use them for other things than rendering).

Meshes and sub-meshes

So we have seen there is data and program/material assembled to describe models to be rendered. In Ogre a model is called a mesh and the class that encapsulates this is Mesh. A mesh can represent arbitrary things but it usually describes a whole "object". Indeed, you could organize your scene in any way but for rendering efficiency and for ease of manipulation (e.g. to move things in the scene), you will prefer an organization per logical object (e.g. a character or a vehicle). A Mesh is such a logical unit, for example our robot. An important point is that such a unit is typically composed of differents parts with different materials although they may share some vertex data such as the 3D position. For that reason Ogre introduces the SubMesh class. A SubMesh has a unique material. A Mesh is a set of SubMesh as you would expect from the names.

Just a final note : if the SubMesh is really atomic and stores only vertex/index data, the Mesh is actually a bit more that a collection of sub-meshes. It also store information on which resource group it belongs to, what is its status (loaded, touched, etc.) and methods for shadow rendering.

Entities and sub-entities

One important point about materials is that they change more often than vertex/index data. What I mean here is that typically a scene contains similar object that are only different in their appearance. A good example is the cars in a racing game. In such cases, you want to share as much as you can. In particular, you would share the vertex and index data and only store a different material per instance. To offer such flexibility, Ogre has a layer about meshes and sub-meshes that are entities and sub-entities represented by the Entity and SubEntity classes. An Entity has a pointer to a Mesh but can overidde the materials of its sub-meshes. For that, it has a SubEntity per SubMesh of the pointed Mesh and each SubEntity stores its own material. Therefore, you will find a setMaterialName(...) method in the SubMesh and the SubEntity classes. If you call it for a submesh, you will impact all the entities that use them and if you call it for a subentity you will impact only this particular instance. Note that you also find a convenient (but at first confusing) setMaterialName(...) method in the Entity class. Here is what its API reference says :

This is a shortcut method to set all the materials for all subentities of this entity. Only use this method is you want to set the same material for all subentities or if you know there is only one. Otherwise call getSubEntity() and call the same method on the individual SubEntity.

The class Entity actually serves other purposes such as maintaining a bounding box and various flags about how it must be rendered (is it visible, does it cast shadows, etc.) or queried (for example in a ray/scene query). As we have seen, an Entity is itself attached to a scene node that holds information on how this particular instance of a mesh is placed in the scene.

Hardware buffers

To finish, let's describe how vertex/index data are specified in Ogre. Ogre is a platform for developing performant 3D applications. Thus, it uses state of the art (i.e. most efficient) mechanisms. Concerning vertex data, it means using hardware buffers. From the API reference of class HardwareBuffer :

A 'hardware buffer' is any area of memory held outside of core system ram, and in our case refers mostly to video ram, although in theory this class could be used with other memory areas such as sound card memory, custom coprocessor memory etc. This reflects the fact that memory held outside of main system RAM must be interacted with in a more formal fashion in order to promote cooperative and optimal usage of the buffers between the various processing units which manipulate them.

Hardware buffers are efficient because they limit the (costly) transfers from the CPU to the GPU (the graphic card) and let the data exists as close as possible to where they will be used. There are various types of hardware buffers : vertex buffers, index buffers, texture memory or framebuffer memory etc. For the moment, we will focus on vertex and index buffers but the principles we'll discover are general.

The price to pay for that efficiency is twofold. First hardware buffers are much less flexible than system memory. Second, if the data is to be also used on the CPU (e.g. for collision detection) and not simply rendered, you might have memory duplication. From the API reference of HardwareBuffer :

Buffers have the ability to be 'shadowed' in system memory, this is because the kinds of access allowed on hardware buffers is not always as flexible as that allowed for areas of system memory - for example it is often either impossible, or extremely undesirable from a performance standpoint to read from a hardware buffer; when writing to hardware buffers, you should also write every byte and do it sequentially. In situations where this is too restrictive, it is possible to create a hardware, write-only buffer (the most efficient kind) and to back it with a system memory 'shadow' copy which can be read and updated arbitrarily. Ogre handles synchronising this buffer with the real hardware buffer (which should still be created with the HBU_DYNAMIC flag if you intend to update it very frequently). Whilst this approach does have it's own costs, such as increased memory overhead, these costs can often be outweighed by the performance benefits of using a more hardware efficient buffer.

So harware buffers involve some work and some knowledge from you, mainly because you must know the various type of access they define and decide which one is best suited to your particular situation. I am not going to detail them here as you can find dedicated tutorials on the web. As helpers, I repeat here the doc from Ogre API reference for the HardwareBuffer::LockOptions :

HBL_NORMAL Normal mode, ie allows read/write and contents are preserved.
HBL_DISCARD Discards the entire buffer while locking; this allows optimisation to be performed because synchronisation issues are relaxed. Only allowed on buffers created with the HBU_DYNAMIC flag.
HBL_READ _ONLY Lock the buffer for reading only. Not allowed in buffers which are created with HBU_WRITE_ONLY. Mandatory on statuc buffers, ie those created without the HBU_DYNAMIC flag.
HBL_NO_OVERWRITE As HBL_NORMAL, except the application guarantees not to overwrite any region of the buffer which has already been used in this frame, can allow some optimisation on some APIs.

And for the HardwareBuffer::Usage :

HBU_STATIC Static buffer which the application rarely modifies once created. Modifying the contents of this buffer will involve a performance hit.
HBU_DYNAMIC Indicates the application would like to modify this buffer with the CPU fairly often. Buffers created with this flag will typically end up in AGP memory rather than video memory.
HBU_WRITE_ONLY Indicates the application will never read the contents of the buffer back, it will only ever write data. Locking a buffer with this flag will ALWAYS return a pointer to new, blank memory rather than the memory associated with the contents of the buffer; this avoids DMA stalls because you can write to a new memory area while the previous one is being used.
HBU_DISCARDABLE Indicates that the application will be refilling the contents of the buffer regularly (not just updating, but generating the contents from scratch), and therefore does not mind if the contents of the buffer are lost somehow and need to be recreated. This allows and additional level of optimisation on the buffer. This option only really makes sense when combined with HBU_DYNAMIC_WRITE_ONLY.
HBU_DYNAMIC_WRITE_ONLY Combination of HBU_DYNAMIC and HBU_WRITE_ONLY. If you use this, strongly consider using HBU_DYNAMIC_WRITE_ONLY_DISCARDABLE instead if you update the entire contents of the buffer very regularly.

Let's now see how (explicit) meshes construction is done in Ogre.

Step B: constructing your own mesh