[SOLVED] CS GPU Java javascript flex assembler IOS Computer Graphics

Computer Graphics
Jochen Lang
[email protected]
Faculte de genie | Faculty of Engineering
Jochen Lang, EECS [email protected]

This Lecture
Introduction to OpenGL, WebGL and Three.js
Game or Rendering Engines, Graphics API, Low-level APIs
Three.js, OpenGL (ES) and WebGL
Programmable pipeline
Drawing a triangle (yeah!)
Jochen Lang, EECS [email protected]

Graphics API
Different API levels
Low-level APIs
Goal: exposing features of the hardware
Intermediate-level APIs
Goal: hardware access but also support of rendering pipeline
Game engines and other development platforms Support specific task irrespective hardware
Jochen Lang, EECS [email protected]

Low-level APIs
Examples are Microsoft Direct3D 12 (Windows platforms only), Metal (Mac, iOS only), Vulkan (cross platform), WebGPU (work in progress)
Provide access to the rendering hardware exposing hardware features
Graphics card as stream processor
Must use other libraries for other tasks, e.g., windowing, math etc.
Must setup interactions with hardware
Large resemblance with OpenGL or Direct3D 11 but increased flexibility
Image source: Wikimedia
Jochen Lang, EECS [email protected]

Intermediate-level APIs
Examples are OpenGL, OpenGL ES and WebGL or Direct3D 11
Design mismatch of APIs to new hardware
Lack flexibility as particular processing pipeline is assumed
Easier to use than low-level APIs
Large community and many tutorials
No direct replacement
Image source: Wikimedia
Jochen Lang, EECS [email protected]

General-purpose Graphics API for the Browser
Examples are Three.js and Babylon.js for browsers
Supporting one or more lower-level APIs
Provide ready-made solution for typical tasks
Easier to learn and faster success
Not flexible, quirks may be hard to understand and change
See https://threejs.org/ for examples See https://www.babylonjs.com/ for examples
Jochen Lang, EECS [email protected]

Game engines
Examples for full-featured application development kits are Unity or Unreal
Support multi-low-level APIs, e.g., Unity supports DirectX, Metal, OpenGL, and Vulkan graphics APIs as of 2020.2
Locked-in to specific approach
Features are limited by API
Enable to concentrate on task at hand
Other examples: Amazon lumberyard, OGRE, jMonkeyEngine
Image source: Wikimedia
Jochen Lang, EECS [email protected]

Rasterization Rendering Paradigm
Idea:
Geometric primitives (points, lines, and triangles) are projected into 2D image space
Pixels in image are tested whether they belong to the primitive or not
If yes, the pixels colour is updated*)
Current graphics hardware implements rasterization
*) Z-buffer test, blending etc. may prevent or change the update
Jochen Lang, EECS [email protected]

Graphics Rasterization Pipeline
Four major tasks:
1. Modelling (mostly a pre-process)
Produces geometric primitives and description of surface, volume and light properties
2. Geometry processing
Normalization (change to camera, canonical coordinates or normalized device coordinates), clipping, culling, hidden- surface removal and possibly shading
3. Rasterization
Deciding on mapping of pixels to primitives, e.g., triangle rasterization and line drawing, followed by fragment processing
4. Display
transfer of frame buffer to screen; may require, e.g., anti- aliasing and tone mapping
Jochen Lang, EECS [email protected]

2D Triangle Example: 1. Modelling
Define the vertices for the triangle Define colours
Decide on image borders
Use model coordinates
Jochen Lang, EECS [email protected]

2D Triangle Example: 2a. Geometric Processing
Normalization
Change the vertices for the triangle into normalized device coordinates
Normalized device coordinates are and
Jochen Lang, EECS [email protected]

2D Triangle Example: 2b. Geometric Processing
Viewport Transform
Map continuous normalized device coordinates to fragments (potential pixels)
Fragment coordinates dependent on the number of pixels of the viewport (buffer, window)
Here and
Jochen Lang, EECS [email protected]

2D Triangle Example: 3. Rasterization
Fragments in viewport are tested whether they belong to the primitive (here: triangle) or not
Jochen Lang, EECS [email protected]

2D Triangle Example: 4. Transfer to Framebuffer
Fragments in viewport are transferred to pixels on display device
Jochen Lang, EECS [email protected]

WebGL
Runs in your browser
Supported by all major web browsers (including IE)
Uses the HTML 5 canvas element
Requires only basic programming experience
Javascript is used to interface with OpenGL
No fixed pipeline, only vertex and fragment shaders
Based on OpenGL ES 2.0
Jochen Lang, EECS [email protected]

Basic WebGL Program
function main() {
// Retrieve element
var canvas = document.getElementById(webgl);
// Get the rendering context for WebGL
var gl = getWebGLContext(canvas);
// Initialize shaders from two strings initShaders(gl, VSHADER_SOURCE, FSHADER_SOURCE);
// Specify black for clearing to black gl.clearColor(0.0, 0.0, 0.0, 1.0);
// Clear color
gl.clear(gl.COLOR_BUFFER_BIT);
// Draw the triangle(s)
gl.drawArrays(gl.TRIANGLES, 0, 3);
}
Jochen Lang, EECS [email protected]

Simplified OpenGL Pipeline
Geometry
Vertex Processor
Geometry Processing (Per- vertex operation)
Rasterization
Texturing (Per-fragment operations)
Textures and Images
Fragment Processor
Frame- buffer
Per- fragment operations
Jochen Lang, EECS [email protected]

GL Operations
Group in the three areas: Geometry
Different geometric primitive are available: points, lines, triangles and quads
Texturing/Image primitive operations
Bitmaps, e.g., glBitmap, Bitmapped fonts
Images, pixels, e.g., glDrawPixels, glReadPixels Texture mapping, e.g., glTexImage2D
Set states of the OpenGL state machine
Color, e.g., glClearColor, glColor
Line styles, e.g., glLineStipple, glLineWidth
Transformations, e.g., glMatrixMode, glMultMatrix
Display modes, e.g., GL_LIGHTING with glEnable, glDisable etc.
Jochen Lang, EECS [email protected]

Using Shaders
Why shaders?
Immediate mode rendering with fixed pipeline:
restrictive
slower (?)
harder/impossible to program more advanced techniques
deprecated for a while (since OpenGL 3)
not available in WebGL
But must do more work on setting up the program
get shader text (typically from file) compile shader text
link shaders into a program
Jochen Lang, EECS [email protected]

Setting Up for Using a Shader
Originally different shading languages were used: ARB assembler, HLSL, GLSL etc.
Now GLSL is tightly integrated with GL High level, C like language
Not vendor specific
Requires at least OpenGL 2.0
Used by OpenGL 3.3, OpenGL ES and WebGL
Jochen Lang, EECS [email protected]

Using a Shader
Load shader form file or hardcoded string
Compile shader inside your host code
Bind shaders into a program
Typically, have at least:
Vertex shader (here, triangle.vs)
Fragment shader (here, triangle.fs)
Set up buffers and uniforms to transfer data to the shaders
Use them in the drawing program see gl.useProgram
Jochen Lang, EECS [email protected]

A Quick Look at the Vertex Shader
Vertex shader
#version 300 es
uniform mat4 mvp_matrix;
layout (location=0) in vec4 a_vertex;
layout (location=1) in vec3 a_color;
out vec3 v_color;
void main() {
// map the vertex position into normalized device coordinates gl_Position = mvp_matrix * a_vertex;
// Pass on the color to the fragment shader
v_color = a_color;
}
Jochen Lang, EECS [email protected]

A Quick Look at the Fragment Shader
Fragment shader
#version 330 core
#ifdef GL_ES
precision mediump float;
#endif
in vec3 v_color;
out vec4 frag_color;
void main() {
// Pass on the fragment color to the frame buffer frag_color = vec4( v_color, 1.0 );
}
Jochen Lang, EECS [email protected]

Geometry Transfer
Objects consist of many vertices, (and normals, texture coordinates, vertex colors)
Function calls are slow
Small data size transfer from PC memory to graphics card (GPU) memory has large overhead
Memory transfer from PC to GPU is slower than memory access on the GPU
Need method with less function call overhead than immediate mode rendering which can store data on the GPU
No. Vertices : 863,210
Source: Aim@Shape
VCG-ISTI, P. Cignoni
Jochen Lang, EECS [email protected]

Buffer Objects
Buffer objects hold data in OpenGL also called data stores
Buffer objects can hold different type of data
we look first at vertex data
A buffer object to hold vertex data is also called a Vertex Buffer Object (VBO)
OpenGL will store the data in PC memory or the GPU to get the best performance
we can supply hints based on how we intend to use the data
Jochen Lang, EECS [email protected]

Buffer Memory Allocation
Need to perform the following steps
1. ask WebGL for a buffer (get an unsigned integer number)
2. bind the buffer to a binding point
here use the gl.ARRAY_BUFFER binding point
3. Fill the buffer with data
var vertexColorBuffer = gl.createBuffer();
// Bind the buffer object to target gl.bindBuffer(gl.ARRAY_BUFFER, vertexColorBuffer);
// Write date into the buffer object gl.bufferData(gl.ARRAY_BUFFER, vertices, gl.STATIC_DRAW);
Jochen Lang, EECS [email protected]

Vertex Array Objects (VAO)
Organize one or several vertex array buffers into a common object
Available in WebGL 2.0, OpenGL ES 3.0
OpenGL 4 always use a VAO
Once the VAO is bound VBOs can be bound to it
// Generate an VAO and bind it
VAO = gl.createVertexArray();
gl.bindVertexArray(VAO);
// Disable it
gl.bindVertexArray(null);
Jochen Lang, EECS [email protected]

Drawing with a VBO/VAO
Let OpenGL know how the data is to be used
index, size of a single data, data type, normalize, stride (if there are gaps in the array), offset
Enable the attribute in the buffer object for drawing
gl.vertexAttribPointer(a_vertex, 2, gl.FLOAT, false, 5*fsize, 0);
gl.enableVertexAttribArray(a_vertex);
Drawing with the currently bound VAO
type of primitive, starting index and number of vertices
gl.drawArrays(gl.TRIANGLES, 0, n);
Jochen Lang, EECS [email protected]

Program Structure with VAO/VBO
Set-up in initialization routine
generate and bind VAO
generate, bind and fill (multiple) VBO for storing vertices and their attributes (e.g., colors, normals, texture coordinates)
unbind VAO and possibly go the next set-up
Drawing routine
bind the desired VAO
call draw using data stored in the VBOs of the VAO unbind the VAO
Clean-up
If there is lot of data in WebGL, free up the space taken up by the VAO/VBO (see glDeleteVertexArrays)
Jochen Lang, EECS [email protected]

Next Lecture
2D Transformation
Textbook: Chapter 6.1, 5.2-5.3
2D Transformation
Translations and homogenous transforms
Matrix composition and decomposition
Self-Study: Basic linear algebra, Textbook, Chapter 5.1-5.4
Jochen Lang, EECS [email protected]

Next lecture
Drawing and Animation
Drawing and graphics pipeline Modelling a Tetrahedron
Projections from 3D to 2D
Rasterization
Animations
Jochen Lang, EECS [email protected]