notes.nt

[11/29/24]
so, almost there, but when implementing the renderer and rendercmd structs, i used "void*" like a dumbass. i have to fuck with Templates; this'll suck, but it'll be good for me. it's all just so i can use arrays of unknown lengths in constructors for the vertex attributes and shit :3

[12/02/24]
Fuck that; the entire codebase is flawed. okay, not the entire thing, but I'm abstracting files too much. need to rethink and keep it stupid simple. I want all "gl" functions to be handled by one class/struct/whatever, so that I keep it contained. That's why I'll need a "Renderer" class. It should be given simple commands to then perform with the "gl" functions. Basically, it should be told what textures to use, what vertex data to use, etc. Keeping everything in one area should be nice for my brain; once it's all done, I'll tackle if and how the "Renderer" should be split up (i.e: separating mesh code from texture code blah blah blah).

Okay, so i think i know what i should do. sadmansk's engine is flawed (or, rather, just very hard-coded) but that's just because it's super bare bones. However, that's actually fine for me since the code still works and I can still use it as good reference but in a different context. That context is my "Renderer" instead of sadmansk's "CoreEngine" and "Engine" and separate "Mesh", "Vertex", and other classes. In place of those classes, I'll use functions or small structs. I'll take notes on what all the classes in sadmansk's engine do and how I can translate them.

sadmansk Vertex class:
basically a small container for vertex position, texture coordinate, and normal data. there only exists a single header for this class, since all it does is store data that you can retrieve later. obviously, this shouldn't be a class.

my Vertex enum:
I'm choosing an enum (for now), since vertex data shouldn't change once it's loaded... I think. If I'm wrong, switching from an enum to a struct shouldn't be too difficult, and I think that keeping things simple in the beginning is best.

my Vertex struct:
I'm wrong! lmfao, I forgot how enums worked. I am basically desribing a struct. So I'll use a struct.

-- Why is the renderer a class?
Why is the renderer a class when there will only ever be one renderer? That's dumb. The idea is to keep rendering code contained in one spot, but a class is just overkill. I'll make rendering functions in a separate rendering file (or files), and then I will consciously limit the use of those functions to one part of the program (i.e: the highest level of the game-loop). This does the same thing as using a class, but without the arbitrary limitations and overhead and blah blah blah of using a class. Just keep the code in one spot and don't put up "baby gates" like limiting rendering to a class so you don't "accidentally" write code for rendering in other places. That's just... kinda stupid.

Okay, let's try this again. Here's a heirarchy:

The Renderer (rendering code)
	- init meshes
		- buffer verts
		- buffer indices
	- draw meshes
		- call shader(s)

The Meshes (structs)
	- contain formatted data
	- return that data in any required format
	- will NOT buffer anything
	- will NOT make calls to "gl" functions

The Shader(s) (classes? functions?)
	- interact with shaders (i.e: uniforms and such)
	- bind relevant textures

-- Actors and Thinkers
I was talking to myself about my Actor.isRenderable() check which uses a Mesh.isEmpty() check, and was thinking about how I used a function instead of a variable (which was a good idea). However, while thinking about this, I eventually realized that I should make Thinkers as well as Actors. This is because an Actor will have all this code for 3D positioning and rotation and all sorts of stuff for rendered objects, but if you want to make something that doesn't need all that, making an Actor with no mesh (or setting the Actor.visible flag to "false") leaves you with all of this bloat-code for 3D handling that you're not going to use. Instead, Thinkers should be a distinct type from Actors and shouldn't have meshes or any other code relating to 3D rendering. This... unfortunately... does end up nullifying the need to check if an Actor's mesh is loaded when decided if it's renderable or not. Even if I am still mildly proud of my Mesh.isEmpty() idea. Also, as a side-note: the game should not crash if an Actor has no mesh, so either a check still needs to be made, or the rendering code should be able to skip rendering a mesh if it's empty. My little idea may still be of use; however, it may be better to simply write the rendering code in such a way that failing to render a mesh results in a simple error/warning message and nothing else. Basically, I could give up on rendering a mesh if it says it's empty (my Mesh.isEmpty() check), or I could try to render all meshes but not let empty/unrenderable meshes crash the game (or cause massive issues). The only problem with a missing mesh is that something that you should be able to see is "invisible", but everything else about the game can still function. Since rendering and game logic are separate, one way or another I'd like to make sure that these potentially inconsequential rendering problems do not result in a game crash. I mean, it'd suck if the game crashed just because a random-ass prop somewhere had a missing mesh. This does bring up the potential problem of invisible actors causing problems with gameplay (i.e: invisible physics objects blocking the player), so perhaps all Actors could have an "important" flag (that's true by default), and any Actors that aren't "important" and have missing/unusable meshes can just get removed all-together. This might be too granular of an approach, however, and a simple replacement/default mesh (and maybe a toggle for people who actually want to use an Actor with no mesh for some reason) may be the best approach. This is a very high-level problem/thought that doesn't concern me yet, but is good to keep in mind.

Okay, first things first: I need to choose a heirarchy. I'm stuck between these two:

Actor
	- Mesh
		- Verts
	- Material(s)
		- Texture(s)

Actor
	- Model
		- Mesh
			- Verts
		- Material(s)
			- Texture(s)

I think I like the second one better, but it means more arbitrary classes/abstraction potentially. I'll test a Model struct idea and see how I like it.

Also: can I make some of these functions instead of structs/classes?

I'm going to look back at that gamedev.stackexchange answer and follow it. Using a Renderable interface and such.

That's too difficult for me. I will stick to simple things and change them later. I know this isn't great practice, but I'm still learning and this isn't a very big or important project.

FUUUUUUCK me, dude, i don't know what to fucking do.

okay, I wrote down some shit in my book; basicallly, have the model hold rendercmds that will get plopped into the renderarray every frame (with the init commands being plopped in only once, during the init call). then, render functions will go through that array and execute each "command" and then toss it out and move on to the next one. this'll require a custom resizeable array that can be initialized with an indeterminate size, but if i can do this then i'll finally fucking be rid of the "Renderer" class archetype and can rub it in everyones' faces. fuck yeah. okay, i need to un-autism myself and spend some time with my boyfriend who i fucking love. fuck, man, fuck fuck FAHK. aight.

[12/03/24]
Okay, here's what I'm going to do. I'll first block out the basic design of an engine. Everything to do with the core loop, basic game logic, and basic rendering. Then I'll implement it with code. Hoepfully this will help me break through this wall and I presume that after I have a way to render things dynamically, building the rest of the engine should come naturally. This means I'll be doing a lot of writing and planning before I actually code, but that's good. That should limit the amount of times I redo things and should also just limit confusion and give me a path to follow. Even if I end up tossing it out and redoing everything, it'll be a much better experience overall.

I'm also going to download the Godot source code and use it as another source of inspiration; specifically, I'll start looking at how they structure rendering tasks, what their pipeline for rendering is, and how they delegate rendering tasks (or if they do any of this at all!).

Here's a thought:
Replace the main loop with a recursive function. The function will check if it should still be going and if it should, it will call itself at the very bottom, thus emulating a loop without using a while() loop.

I've made peace with object oriented programming. In reality, programming requires a balance of all three forms: Imperative, Object Oriented, & Functional. Now, I have a bias against object oriented as it is still technically a flawed form of programming (in my opinion), but it aids the human aspect of programming. I just wish it didn't also directly interfere with the end result (like how naming variables won't impact how fast the code runs since the computer never sees them).

Okay, I've got an idea now; here's a rough estimate of fixed and unfixed parts of the render pipeline:

Fixed:
 - storage
 	- meshes
 		- vertex data
 	- GPU buffers

Unfixed:
 - rendercmds
 - rendering functions (obviously)

The reason for separating these is for my brain mostly. Mesh data will be stored once per time it's loaded (i.e: loading a map), and commands are generated every frame which can use/be based on the mesh data/buffers. I don't think there'd be a reason or a condition where render commands wouldn't be based on mesh/vertex data (aside from 2D rendering), so I won't account for that unless I run into it later. This means I should design the code to be flexible, which means abstracting the rendering from the API(s).

I'm also foregoing the old file-naming conventions, but I'll paste them here for sake of documentation/logging:
:::C++
// File Naming Conventions:
// 1. Source code file names should clearly represent the contents of the file (i.e: code in "g_rendering_F.cpp" deals with rendering)
// 2. All file names are prepended with a single letter which acts as a "grouping tag" (i.e: in "g_rendering_F.cpp", "g" means "game")
// 3. All source files are split between "Functional" code and "Object-Orientated" code but will share header files, for convenience
// 		3a. Functional source files are appended with "_F"
// 		3b. Object-Orientated source files are appended with "_OO"
// 4. Header code may be one file specific to one pair of source files, or many files relating to one pair of source files
// 		4a. When naming the latter, the "grouping tags" should represent the name of the source files (i.e: in "r_common.hpp", "r" means "rendering")
//
// Ex:
// 		The source code for rendering is split between g_rendering_F.cpp and g_rendering_OO.cpp
// 		The header code for rendering is split between many files, including "r_common.hpp" and "r_renderer.hpp"
//		The source code for game actors is split between g_actors_F.cpp and g_actors_OO.cpp
//		The header code for game actors is all in g_actors.hpp
:::
OI!! REMEMBER THIS!! -> you can have up to ~16 vertex attriutes BUT YOU CAN HAVE AS MANY BUFFERS AS YOU LIKE (more or less)

Hmmpf, I think the reason you make a renderer class and abstract so much is because globals are a pain in the ass and while they may be programmatically more efficient, they are not kind to humans. Still, I'd like to try with a few.

[12/05/24]
I need to re-think and re-word the problems that I am trying to tackle whilst making this game engine; specifically following the lessons from "Algorithms in C (Third Edition)".

[12/06/24]
Brain blast happened; I'm going to keep the gl functions in the main.cpp file and that should give me a more accurate way of testing my abstraction and isolation (instead of putting them in the Mesh class??? like, why was I doing that???).

I've managed to get rendering working, but there's a catch: only when there's one object in the scene. In order to render multiple objects at different locations, I need to send their location to the shader *AND* draw them all one by one (in a for loop). There's probably a better way of doing this, but that better way will still require me to make the same changes to the rendering pipeline that I'd make in implementing the for loop version, so I'll start with that.

[12/09/24]
I forgot to write this down, so I'll just paste in the git commit note from yesterday:

"FUCK FUCKING YEAH!!!! GAMETICK AND FRAMERATE ARE SEPARATE AND GAMETICK CAN BE WHATEVER I SET IT TO BE!! (that and also I can render two or more separate objects at different locations). Now, this system is far from perfect or complete in any way, shape, or fashion, BUT it works and it's mildly flexible (the gameticks are on a separate thread and loop), so that means that the core idea is working and I can make changes to it without having to re-write the entire thing. One step closer to a full engine. holy shit"

Okay, now for today; I'm going to attempt to separate/abstract more of the rendering code and not touch the gametick code until I've chipped away at making rendering more stable/solid. Mainly the core idea of rendering, as I have a few throwaway functions and global vectors and that's it. I want to get to a place where I can render any and all renderable objects in their proper orientations. Once I've got a small system that works decently well, then I'll let myself work on game logic.

More concrete goal: remove all "gl" functions from main.cpp and isolate them to either r_renderer.cpp or whichever file I decide the renderer will stay if I change it (but keep it in ONE place!).

The original idea for rendering was something like this:
	1. All renderables are stored in a vector
	2. All meshes belonging to the renderables generate their buffers and store their IDs in a vector
		2a. For now, they should also store the number of indices in a separate vector until I streamline that better
	3. The render function (whatever that may be), will go through the vectors and render the buffers with the global positions from the renderables

A better way would be to combine all this render-related data into one vector, using a struct (that's the idea of the RenderCmd class, dumbass). This'll be
what I work on first: a RenderCmd struct that'll house all relevant data for rendering, that the relevant Actor can generate and store.

There's at least one problem, though: vertex data is only ever stored once (when it's first loaded) and should never be re-generated/re-stored. However, other data like global position will be allowed to change and thus must either be buffered when it's changed or simply re-loaded every frame. I'd rather it be buffered once it's changed, but that might conflict with the RenderCmd design. However, since the call to load vertex data should only get called once, I might not actually have to worry about this too much. Remember: consolidate the problem into its most basic form; i.e: get the problem down to "what data do we need and where does it need to go?" or as close as possible to that.

Big thing to note: switching VAOs is costly. Try to remember what exactly a VAO is doing/containing.

Okay, I think I'll be making VAOs for specific sets of meshes (i.e: one for environment/geometry, one for characters, etc.), then I'll be making sure they take up a very specific slot in the global vector (i.e: environment VAO is in the first slot). Finally, every actor/actor's mesh will have an associated type that corresponds to that VAO, as well as having offsets for the vertex and index data (at least, I assume). Then we can render using as few VAOs as possible.

FUCK

[12/10/24]
Okay, I've finished the function for processing all the RenderStorageCmds, but now a problem: RenderCmds. See, at first I thought that since we want to sort the queue by VAO ID (to minimize CPU -> GPU data transfer), then we'd be sorting the queue every frame, which at first glance seems wildly inefficient. However, even using my simplistic modified bubble sort, since we'd be replacing RenderCmds who's relevant data has been changed (i.e: global position) and not refreshing the entire queue every frame, the average time cost would be closer to O(N) and not the worst case O(N^2). This means that my new problem is finding a way to not only populate the queue, but replace RenderCmds specific to objects that need to replace them. One thought is to give each RenderCmd a pointer/reference to the object that created them, and then upon sorting the vector update a value/pointer/reference on that object that corresponds to the RenderCmd's new index in the vector. However, this might not even be neccessary since intuitively it makes sense that using references/pointers, the object could just always know where its RenderCmd is in the vector. Since I don't know, though, I'll need to do some research.

OH! WAIT!

The RenderCmd queue could be a vector of REFERENCES to RenderCmds of each renderable object; they get added to the vector when they enter the scene or transition from being "not renderable" to "renderable". This would not just remove the need to replace RenderCmds, but it would also remove the need to run the modified bubble sort every frame, which would always have a chance of somehow being a worst case of O(N^2). There might be downsides to this (in fact, there most likely are, since I'm not an expert programmer and am new to this entire field of game engine programming), but it seems like a good solution so let's try it.

I just finished a simple test implementation and the idea will work! However, I need to remember that if this RenderCmd queue is not being refreshed every frame, then I need safe ways of adding and removing RenderCmds from it... is what I thought until I literally when writing that just realized I could instead give each RenderCmd a flag that basically says "skip me!" and either implement a fast way of iterating through the vector that reduces the runtime from the inevitable O(N) or just... not care and let it be. The problem I forsee is related to every object that COULD be renderable having a RenderCmd in the queue. However, I could do a tricky little trick to skip these RenderCmds without iterating over them. Something like setting their "vao_id" to "-2" which would (because of the nature of the modified bubble sort and iterator) cause the "R_Render" function to just... not even consider them... I think... actually, I don't think that specific implementation would work, but something similar could. Basically, find a sneaky way of making every RenderCmd that has an "unrenderable" or whatever flag avoid being iterated by the "R_Render" function entirely. At the end of the day, overhead could be very small, however, since checking a flag SHOULD (if I'm correct) take extremely less time than getting values and sending them places (i.e: sending render_position to the shader).

What needs to be done right now:
	- change the RenderCmd vector to a RenderCmd* vector
	- give each RenderCmd a "skip me!" flag
	- implement skipping the relevant RenderCmds (or a sneaky way of avoiding them entirely)
	- figure out how and when to populate RenderCmd* vector (could just be at scene load, like the RenderStorageCmd vector)

NOPE! I've found the problem with my technique: rendering interpolation. It can't work if we don't have a queue of RenderCmds that gets updated each tick and instead have a static list of RenderCmd pointers to structs that could have their values changed every tick. We need to know the past. We need a queue.

and i thought i was so fucking smart.

Well, okay, I didn't; that's why I realized this is (I assume) the reason for using a queue and not a static list. I knew my idea sounded too simple and too different. It might work with fenangling, but then the benefits of its simplicity would fall apart.

Fuck, here's where the book starts to come in handy.

The problem is that I can't just sort the RenderCmds by their VAO, since the whole reason for the queue is to be able to depend on it being sorted by time. Of course, the real problem is I don't fully know how the queue should work with interpolation, since I don't know how to keep track of the amount of gameticks per frame and use that in the interpolation code. I also don't have any ideas for the interpolation code. This is ESSENTIAL for my independant gametick and framerate based design. I need to do more research...

fuck me

Okay, so for implementing interpolation, lag is out of the equation (should have been obvious) and should be something to avoid/remove. What we're really concerned about is offsets due to having a higher framerate than tickrate or, more specifically, having a framerate that does not divide evenly by the tickrate. This leads to a worst case of any frame being one game tick behind, and a best case of any frame being exactly on time. At a shared tick and frame rate greater than ~59, this is a delay of either ~1/60th of a second or less. This is acceptable.

This is also why we store a "previous state" and a "current state" instead of accounting for every tick processed during the frame (since more than one tick per frame is a result of lag, NOT differing rates).

For my first implementation, I might actually end up using my static list after all, and simply giving RenderCmds a "previous_state" and "current_state". Probably by making the queue out of a different struct that contains two different RenderCmds for each state or something like that.

I'm also going to be choosing the official name for chunks of stuff in GraphX: "spaces". Basically what Godot and Unity call "scenes".

Final notes: as per the comments I left in main.cpp, the current gametick testing function (which is a surrogate for the eventual final gametick function) should call the Tick() function of every Actor in the current Space, then handle the state swapping and buffering logic for each respective Actor... unless I decide to have the Actors handle that themselves (in the Tick function or with another function that's called after Tick in the gametick function). If future me is confused by my thought process, I'm partially following this (https://gamedev.stackexchange.com/questions/12754/how-to-interpolate-between-two-game-states) but I'm also sticking to DOOM's structure a bit as well. The result is (hopefully) every actor having buffered state structs that either contain the variables like "global_position" and such, or get populated with the data from the Actor's relevant variables (I like that idea more). These state structs will be used to render the Actors with the relevant (and potentially interpolated) information. How the RenderCmd queue will work and implement with this is unknown; either I link Actors' states with their relevant RenderCmd, or store it in the RenderCmd, or have the RenderCmd be a struct containing the vao_id and vertex+index offsets AND the buffered states, or some other idea.

Piece by piece, you can do this; just focus, learn, and always be open to failure. Failure is always an option. Failure is progress. Success is just a reward for your consistent failures and the information you learn from them. This is a positive affirmation. Failure is positive.

I love you. You are capable of this

:3

[12/11/24]
Thank you, past me. Okay, let's do this! I think I can get back to my thought process pretty quickly, actually. Based.

I'm also biting the bullet and changing Actor from struct to class; it needs private functions (and potentially variables).

Sidenote: sanity.hpp might be causing the long compiling times (or just all the included header files in general). Check if gatekeeping header includes behind an #ifndef would break the program or benefit the compile times and also check if aborting use of sanity.hpp would likewise benefit compile times.

Addendum to sidenote: You can (and probably should) use include guards (#ifndef) in source files when including header files (the examples I'm seeing are specifically for custom header files and not libraries so experiment with these). Since, to the compiler and linker, it really doesn't matter which file includes a header as long as one of them do, this should benefit compile times *and* satisfy clangd.

Secondary sidenote: Also, you can use #else to forward declare things that other files need but would cause cyclic #include errors (i.e: class A needs class B but class B needs class A). This is actually a problem I've run into and could use this solution.

I'm also making a change to render storage and instead of using commands, I'll just store pointers to the meshes since they're the exact same thing in this context. As it is right now, the Mesh struct is just passing data to a new RenderStorageCmd struct and then pushing that to the storage commands vector. The engine shouldn't have such inefficiencies; however, a higher-level application that interfaces with the engine (e.g the Godot editor, the Unity workspace, etc) can have such inefficiencies as they serve to benefit human interaction (i.e: a higher level Mesh class that just passes information to various structs withing the engine is fine in that context as it leaves these inefficiencies out of the engine and they exist to serve the human interface).

All that to say: I'm replacing RenderStorageCmd with Mesh as they are functionally the same.

Had to pump the emergency brakes for Dexter; he feels like shit and he's more important to me than this. I'll make inline comment notes on things but I'm only giving myself five minutes for this and I'm down the last 55 seconds. Anyways, it's fine; the idea I have is to replace RenderStorageCmds with Meshes since they serve the same purpose, and the rest will be in the comment(s)

Cya later!

:3

[12/12/24]
Alright, time to finish what I started yesterday (hopefully, at least).

Idea: in the Actor constructor, instead of making an empty RenderCmd, what if the constructor was passed a reference to the Actor's mesh and the RenderCmd constructor handled all the data storage?

I'm getting rid of RenderCmd and just iterating through a list of Actor pointers. This is because the RenderCmd structure is just over-engineered for my simple and beginner-level knowledge. It's also more suitable for multiplayer and I'm not touching that with a ten foot pole yet. I also plan on re-writing most, if not all, of the engine when I'm done, so this should be suitable for now.

Okay, after a lot of coding I have this: the render function works, the Space struct works, the multithreading works (and I have yet to run into a race condition bc I used a mutex lock_guard), and the only thing that might be broken is the vertex+index data. Basically, I don't use any offsets when rendering vertices but I don't see anything wrong, however, since all the testers use the same cube verts and indices, it could look fine but they're all using the same exact vertices (when they shouldn't).