Fractured Foundations

The last couple of days were spent largely rooting through the innards of unexpected behaviours from Microsofts Visual C++ compiler and walking through my design process for how to organise nodes.
The first thing I did after the previous post was sit down with a pen and paper and map out the structure of my modules and interfaces.  It’s easy to get lost in the true relationships as you work with code and I found myself getting very grey about where code needs to maintain strict seperations.  When going through the design of the node libraries I realised that the ideal structure was actually quite straight forward.   Creating the concept of a “Node Module” which was the hub to any node library.  It is what the engine component deals with as well as what nodes within the libraries register with so that they can be listed and instanciated.  With a library of nodes and a node module being a self contained unit the engine component can then load any amount of these libraries and then notifies the Editor itself or any object that wants to know about the loading and unloading via callback registrations.

So with the redesign in place I set about implementing the code.  The interfaces for the nodes were reduced to just a single one.  I’m still a bit iffy on how inputs and outputs will interact, currently it’s a bit hack and slash but is quite generic and allows flexibility at the implementation end.  The engine’s loading function was modified to call the appropriate callbacks and ensure the node module was created.

It was at this time however the troubles started.  I found that upon calling into a node module I wasn’t getting back the appropriate information.  I couldn’t create, list or retrieve any of the node factories within the library.  I tried all manner of debugging to walk through the issue trying to track it’s origin.  Ultimately it was a difficult one to track.  The calls into the node module library seemed to never leave the local library and find no registered factories even though the factories registered properly and the modules data was fine.  I had been a bit worried about using Standard Template Library data types in my libraries but their usage was fairly self contained.  However with this issue I spent time rewriting my own templated data storage packages to try and improve the library compatibility.  This sank about a full days work, though I was able to improve on some of the workings of the classes, probably not speed but task specific features which made it a bit cleaner to work with.

After I’d taken a bit of an over enthusiastic jump into the world of STL replacement, I decided to log and visualise the process leading up to the issue.  For this I wrote a little tool in Windows to load up each library and list off the available registered factories.  To do this it simply gave the user a selection to go choose the library they wanted and then it grabbed the exported module class.  This module class is shared accross all of my libraries as a single common interface.

Selecting A Module To Inspect

For this tool I added interfaces to the module interface that allowes for simply itterating over all the registered factory names.  I hit yet another snag here.  It turned out the modules I was inspecting didn’t want to work with my new functions and refused to acknowledge the existance of any factories.  I noticed that it was a similar issue to the node module in that the function calls always called to an instance of the module class contained in the local calling library rather than the remote library.  This was even more perplexing because I could access my “RequestSystem” method and it would go and use the factories just fine.

Fast forward a few hours of rooting through forums and the Microsoft Developer Network, I was on my last nerve and had made changes that in hindesight could have caused a lot of issues later.  On a whim I copy and pasted the method declarations in the Module class to see if it was some strange compiler issue.  As I was doing so I removed the ‘virtual’ tags that were in front of the RequestSystem and ReleaseSystem module methods as they weren’t needed due any longer.  I had intended to use the module class as  a base class for further module specialisation but factories did away with that needed.  Upon removal of the virtual methods the test application began acting very strange.  For the request system it no longer called the right library, it was calling the local library version.

This realisation took me straight to the soloution.  If I declared a method in a class virtual and that class existed both on the local library and the remote one, the virtual declaration meant it would call to the correct place where as the normal methods always called the local instance no matter what.  I usually interface out all my cross library features but with the module and node module classes they were compiled out to a core library and I assumed that since they all rely on just a single class it’d be fine.  Well that was wrong, as soon as I interfaced out the module data and used the interface to invoke methods everything went back to working happy as can be.  The interface testing application was able to list off all registered factories just fine.

PrEditor Engine Interface Listing

By giving the node module the same treatment I was able to accurately list off the node factories and the modules root name.  This I passed to my UI manager in the PrEditor system.  Currently it just lists in a static text box the module name and the nodes it contains but now I’m hoping I can get round to adding the library name as a selectable list and associate the factory names with GUI nodes.  With any luck this will allow for a good start to the week.

Editor Interface Listing Registered Node Module and Nodes

I spent the last couple of days fighting APIs and limitations.  Bit of a pain and lost a lot of time to nothing but overall I got a large chunk of progress made on the implementation of the Editor UI.  I’ve also got a few pages of design work in place for the actuall structure of the core library for running the project.

First the boring design stuff, since I’m not scanning paper to speak for itself my rough description would go as follows.  UML type interface boxes showing overall code structure and call directions.  Pseudo class definitions for some of the core interfaces for listing, polying and querying contained interfaces.  Finishing off with more pseudo algorithm design for parsing graphs using stacks and reverse traversal.
After I’d gotten that design stuff out of the way I worked on the interface for loading, creating and saving graphs.  At first I was going to keep using the same GUI Toolkit that I’d been working with previously, however trying to push my code into it’s structure just didn’t want to play nice without requiring a lot of work involving scripting and over engineering a few classes.  After losing a day and a half to that I worked on my own task specific UI that I was able to get up and running to my liking.  It’s not overly pretty perhaps but in terms of allowing me to built on it how I want to,  it does the job.

First Pass of Graph Interface

With the outline in place and it rendering without too much hack and slash I was able to add in selection and dragging, so that the nodes aren’t static and can be ordered however you want. Or in the case of my examples to scatter nodes out of their lovely little triangles.

Node Graph Moved Around

With the selection stuff working adding in connections to the graph was my next little venture.  At the moment a created connection just represents a line between 2 nodes but I’ll be expanding on the slection modes, connection features and the shapes and end points of the rendered lines themselves.

Connections Created On Node Graph

The final piece of the puzzle for this little venture was the fact that I’d not abandoned the GUI toolkit entirely.  What I’d done was layered it ontop of my new GUI stuff so that it still existed as a context menu type system pulled up when the user clicks a certain button.  My goal is to put everything the user will need into the pop ups so that they can easily get to all the features.  Editors such as that for LOVE and some 2D tile editors feature this kind of interface and modelling tools such as 3D Max feature quite complex context menus containing many features.  Though technically I have to admit it’s also a handy way to run things as CEGUI essentially gets to run in isolation.

CEGUI Running On Top of Graph UI

That’s about it for now.  Next steps will be implementing the PrEditor engine so the graph UI can start linking up to real graph elements and creating/designing the UI to allow the user to change selection modes, connect node inputs and outputs, save and load graphs, create different node types and possibly edit properties of some nodes.

I spent today bouncing between semi useless and quite important tasks.  Most of my time got swept into the useless because it’s immediately more visuall interesting unless I can find a scanner.

Firstly, the important stuff was fleshing out my library interfaces and planning for the graphing system.  It’s getting a bit silly they design process revolving around this project.  It’s definately an interative design process but each itteration removes complexity and focuses the project more.  I had originally wanted to do all XML based graphs and node handling but I realised it’s not actually important.  A more concise and faster internal structure makes much more sense plus if all goes as planned and the runtime library is used for generation then extensibility isn’t really needed as I have full control over the formats and just need to ensure intput and output data types are strongly defined.  I also twiddled with the interface a little but it didn’t actually achieve much.

The useless things I did today revolved entirely around the demo framework.  I implemented a splash screen which I’ll post up if I have a day with nothing worth posting.  The other thing I really focused on was expanding on and further coding my shader class.  Adding texture sampling pass throughs and messing around with various effects.  I don’t think I’ve made any major revoloutions and infact I probably didn’t even do it right but I’m happy with how it’s turning out.  The loading of shaders from external files makes it so tempting to mess with them.  However the more complex I try to make it the more I find myself going back into my C++ code and revising or altering chunks to be more flexible.

Just before I put up the pictures of my messings around today, I do have a bit of method to mayhem.  I want to have a crack later, if I can, at using the editor to create materials, similar to UE3’s material editor which is one of the direct insipirations for this project.  And now, pictures! Nothing amazing, just doodling and messing.

Blending 2 Textures Based on Horizontal Screen Position (not photoshop like the previously posted blends)

Very Dark Blend( not sure why I took this, lets just pretend it's a nightime scene)

Blend Using Yellowness Of Overlay Texture

After developing the low level engine components of the demo framework I went back to working on the editor interface.  Trying to figure out what I was doing nearly 4 weeks ago when I first wrote it was a bit confusing as large chunks were “potential” features that won’t ever be developed.

What I did instead was figured out what could be kept and ported it to my demo framework engine.  I wanted to use OpenGL as a foundation for the node graph rendering and real time previews but the feature set of wxWidgets goes well above and beyond, most of which essentially gets in the way.  By using the framework I’ve come up with new library design.  The framework has an Editor mode built in to it that loads an intermediate library that handles GUI for the PrEditor TK stuff and loads up the core PrEditor libraries.  This should hopefully simplify the development since it’ll be bare bones.

The port didn’t take much of today and left me a bit of time to mess around with implementing and wrapping GLSL shaders.  Tomorrow I’m going to sit down and create the technical diagrams for the editor module, the run time module and the plugin system for the runtime module.

Textured Composite Helmet From SWG

Textured Rancor from SWG

After getting the models from Star Wars Galaxies rendering I set about cleaning up the storage medium and process.  The above renders still contained model specific hacks to get them running.  At this point I got talking to a friend who works as a professional environment artist and tried to find out the formats that he would use most often.  These turned out to be the Wavefront obj and AutoDesk fbx formats.  I decided to drop trying to get a usable format for the SWG assets and focus on an OBJ loader.  As I progressed with the obj loader I noticed several things that would be problematic but also some ways to improve my existing mesh data structuring.  Using that I reworked the format and it is now simpiler to work with and easier to manage.  However issues with OpenGL’s array drawing abilities have held up fully implementing them.  Below is a sample of the current OBJ loader.  The model has the general shape it’s meant to but is still far to much of a poly soup to be recognisable.

Polysoup of CityDemo.obj

I also finally got the Demo Applications sandboxed.  They run inside of an engine that is within the engine of the demo frame work.  This seperates the demo content from the core content and makes things a lot easier to shut off and destroy.  They are also very self contained with almost all loading, operating and destruction being managed autonomously.

During the day I had to spend a bit of time with my server too.  The machine that I run my SVN server on was compromised by a virus.  I don’t know how long it had been there  or whether the Anti Virus had managed to block it prior.  Once i realised the compromise i backed up my SVN repository and reinstalled the machine.  I’d needed to clean it up anyway but I’d rather it hadn’t been like that.  I’m thinking of virtualising the main envrionment on the server so that I can run the SVN service on the native OS but then use the PC through a virtual machine.  That way if it ever gets compromised again it’ll most likely be in the VM and that’s much less dangerous to wipe than the actual machine.

Tomorrow, other than going through and changing all my login details for this I access through my server, I’m going to look at bringing some procedural things into the terrain demo app.  I’m also going to try and do the rought design of the PrEditor run time library that will provide the ability to create things on the fly by loading graphs created within the editor.  I plan on starting with generic generators connecting to generic outputs.  Such as a mesh generator that takes parameters for segments and spacing and a mesh output that provides an interface to an array and index buffer.