OpenGL 3D Game Tutorial 29: Mouse Picking
Why Game Development Stack Exchange recommends it
Game Development Stack Exchange answerers point people to it when someone needs to convert a screen or cursor position into a 3D ray or world coordinate, or pick a voxel or object under the camera.
Recommended by 3 people across 3 different months, 2015 to 2017.
Why Stack Overflow recommends it
Stack Overflow users link it when answering or asking how to turn a 2D mouse position into a 3D ray for picking objects in OpenGL scenes, whether in Java, C++ or C#.
Recommended by 1 people across 1 different months, 2017 to 2020.
What it is
A Java and LWJGL tutorial from ThinMatrix's OpenGL 3D game series that implements mouse picking using ray casting. It walks through turning the cursor position into a 3D ray direction and links to the author's code and a text article on the same math.
What people said
Recommends l'arbre · 2015-08 · Game Development Stack Exchange · on “What algorithm is used to select a voxel (from a player's perspective)?”
Raycasting is what you want. As far as I know Minecraft uses raycasting aswell. How it works is basicly, that you send a "ray", represented as a 3D vector from the middle of your camera. The voxel you look at then is the first one the ray " collides with. ThinMatrix made a tutorial on ray casting [this video] .
Recommends jbakker · 2015-09 · Game Development Stack Exchange · on “Cursor position to a 3D ray using angles”
A month ago I came across a youtube video [this video] which explains how to do it, although this video is a tutorial for Java, you will probably be able to use it. The video was based on this text article [LINK] . I hope this is usefull for you. PS: This tutorial uses the LWJGL Matrix functions, this StackExchange answer [LINK] has some code which you will be able to copy if you need the inverse of a matrix.
Recommends Odney · 2017-01 · Stack Overflow · on “How to detect touch events within a certain Renderer-area in android?”
Your question is much more complex than just detecting touch events. The renderer doesn't only have the sphere object, it holds the entire OpenGL ES content. So even after you detect your touch events, you will need to convert your screen 2D coordinates to 3D ray and check if it intersects with the sphere. This is called ray casting . Here [LINK] is a nice explanation of the ray casting algorithms. Also there is a good video tutorial [this video] for implementing ray casting Java. Also you will want to pass the renderer in the constructor of the GLSurfaceView as you and @satm12 suggested.
Recommends J22o · 2017-07 · Game Development Stack Exchange · on “LWJGL 3 how to convert screen coordinate to world coordinate?”
There are to steps to converting cursor coordinates to world coordinates: 1) Convert the mouse coords to a direction. 2) Perform something like a raycast to find to world coordinate. Step 1 : First you will need to convert mouse coords to a direction. This is a simple illustration of how you could do this: [LINK] (Source: [LINK] ) I also recommend taking a look at this tutorial: [this video] . You should be able to use the math classes with LWJGL 3, so you shouldn't have a problem. Step 2 : This part is the hardest out of the two, if you don't already have some sort of physics engine up and running that supports raycasting. If you don't have one available, then it shouldn't be that difficult if y
Cites Nublin · 2017-08 · Stack Overflow · on “Mouse picking miss”
I did mouse picking with terrain for these lessons (but used c++) [this video] [LINK] The problem is that the position of the mouse does not correspond to the place where the ray intersects with the terrane: There's a big blunder on the vertical and a little horizontal. Do not look at the shadows, this is not a corrected map of normals. What can be wrong? My code: void MousePicker::update() { view = cam->getViewMatrix(); currentRay = calculateMouseRay(); if (intersectionInRange(0, RAY_RANGE, currentRay)) { currentTerrainPoint = binarySearch(0, 0, RAY_RANGE, currentRay); } else { currentTerrainPoint = vec3(); } } vec3 MousePicker::calculateMouseRay() { glfwGetCursorPos(win, &mouseInfo.xPos, &mouse
Cites Jack · 2019-06 · Stack Overflow · on “Unable to mouse pick a quad rendered in a framebuffer”
Thank you for your response. 1) I am using a UI library, so my framebuffer is positioned in a widget located at (174f, 5f), therefore I believe I should offset the mouse position? 2) I don't think this is needed as I am using LWJGL? (see: [this video] ) 3) I did not do perspective division as "we only need to un-project the x,y part, so let's manually set the z,w part to mean 'forwards, and not a point'" (see: [LINK] ). Updated my post with some more info.
Cites wotnot · 2020-05 · Stack Overflow · on “Modern OpenGL: Object picking (C#, OpenTK)”
Answering my own question here so that I can post the working code for the benefit of other users, but at least half the answer was provided by Rabbid76, whose help I am very grateful for. There were two errors in my original code: Vector4 pos2 = Vector4.Transform( pos, Matrix4.Invert(modelView) * projection ); where the two matrixes were multiplied in the wrong order, and the projection matrix was not inverted. float dist = ObjectPicker.DistanceFromPoint(new Point(e.X,e.Y), new Vector3(0,0,0), model, projection); where I passed in the model matrix not the modelview matrix (which is the product of the model and view matrices). This works: private void GlControl1_MouseMove(object sender, Mous