Welcome

Yes, the title is a pun. Today I’m going to talk about sprites in The Immortal. And to do so, I have some art (diagrams).

What is a Sprite

Sprites are quite interesting, because they fundamentally differ between consoles and computers of the time. On a comparable game console like the Super Nintendo for example, sprites are handled partially by the software, and partially by a bit of dedicated hardware on the PPU. Essentially, the SNES has dedicated memory for sprite data. For the gfx, it has Video RAM, a dedicated portion of memory on the PPU that is converted directly into the screen layer (baring any HDMA shenanigans). For the properties (relative x, y, etc.), it has OAM, Object Attribute Memory. By contrast, on the Apple IIGS, there is no dedicated video memory per se, it’s simply a buffer in RAM that is assigned to be the screen memory. And there is no OAM, but this is because there is also no fundamental limit on sprites. This is where they start to differ. A console will have a certain amount of OAM, because sprites are applied to the screen by the hardware, not software. The game tells the PPU where on the screen it wants a given sprite, and the OAM data tells the PPU how to arrange the gfx data. But from there, the application onto the screen layer is done behind the scenes. This is unlike the Apple IIGS, where instead the game must perform both functions. It must handle the general sprite arrangement, and it must also handle the screen ‘munging’ to apply the sprite to the layer. This is a tradeoff, as the game gets to set its own limit on sprites, based on the rest of the game processing. However it must balance this with the processing needed to actually render the sprites.

To clarify, let’s start by looking at the screen:

On the SNES, the PPU is responsible for taking the sprites and flattening them into the layer, based on priority levels in OAM and transparencies. On the Apple IIGS however, there are several routines in Driver.GS dedicated to ‘munging’ the screen together. Ie. irreversibly changing the screen buffer by applying the sprites onto the direct byte data.
As a result, sprite data itself is a little bit different between consoles and computers, as it is forced to be more standardized on the console side. With the computer the methodology is, strictly speaking, up to the programmer.

What is a Sprite in The Immortal

Okay, so now the question is, what is that methodology in The Immortal? Well let’s start by sorting out a methodology ourselves first.
The first question towards that goal is, what does a sprite need to do? Fundamentally, all a sprite needs to do is display a bitmap at a point on the screen. Let’s start with that:
Sprite:
– X, Y
– *bitmap

But sprites have another characteristic as well, they tend to (not always) be animated. This is where it gets a lot more tricky. There are many ways to animate a sprite. We could start with:
Sprite:
– X, Y
– *bitmapFrame0, *bitmapFrame1

Although now we need a way to know which frame we are on, so really we need:
Sprite:
– X, Y
– Index
– *bmpF0, *bmpF1

Hmmm. But what if the bmp for one frame needs to be offset compared to a different frame? Then we would need relative point data for each frame entry. At that point we might as well make frame a separate data type:
Frame:
– rX, rY
– *bmp
Sprite:
– X, Y
– Index
– Frame f0, Frame f1

Okay, that seems fair enough. So let’s take a look at what The Immortal sprite data structure looks like:

Alright, some of this looks familiar. We have an X and a Y, and a frame index. However we also have something called ‘num’, as well as a file pointer, something called ‘on’, and a priority value.
I will save this post the trouble of tracing certain values backwards, but I will also mention that this is one of the reasons that I mentioned I am trying to translate in layers. If you start at Sprite, you don’t know where the file reference came from, or what the purpose of num is. But if you start earlier than that, you are introduced to those in the sprite_list.GS file, where you see how they are put together. For instance, the file reference comes from a table that is put together as files are loaded in. This table is then used by quite a few different routines. Num is also tricky, because it is not clearly defined by its name. It is, in fact, an index used to find sprite data. However which index, is the question. So to skip ahead, I sorted through all of the data references, and we can see just how much more there is to a sprite other than what we might first assume.

For starters, we know that it starts as a bitmap in a file, and ends at the screen layer. We also know there is a munging stage before that:

Bitmap -> … -> Munge -> Screen
But to handle the animation and which bitmap from what file in which order etc etc, we have a few extra steps:

Bitmap -> Frame -> DataSprite -> Cycle -> Sprite -> Priority Sorting -> Munge -> Screen
Just a few extra things in between…

Frame and DataSprite (the name I gave it, it does not have a name in the source) will make sense shortly, but Cycle is the one that looks strange from the outside. You see what Cycle is from the outside, is some kind of data structure that gets processed, has a file reference, a sprite index reference, and a series of ints that are referenced as frames. However, if we look at DataSprite, we will also see a series of frame references, a file reference, and a sprite index. For that matter, everything seems to have a file and sprite reference. So what’s really going on?

Well, the short answer is that the file and index references are necessary for the sake of space, but end up creating a very convoluted web of indirect references to each other. Let’s define each of these data types now:

Bitmap: The actual byte data of a given sprite (not compressed)
Frame: One instance of a sprite image. Contains a reference to the bitmap it uses, relative offsets, and the size of the rectangle (the size of the sprite data).
DataSprite: In the source, this is actually data that is copied from the file and stored in the heap, but then modified after the fact as if it were normal RAM sprite data. It is in fact, attributes of a sprite. You can think of it as the interface between the ROM sprite data and the RAM sprite instance. It contains centre X and Y values (these are what get modified by the game itself), a list of references to frames, and the total number of frames. Crucially, it does not contain an index to the current frame.
Cycle: What I originally thought of as an over-arching animation system, I believe is better expressed as simply, the index to the current frame, of a data sprite. But with lots of overhead.
Sprite: This is what is actually used for drawing, and can only have N amount in memory at once. A sprite has an X and Y relative to the viewport, an index to the current frame of the datasprite, a priority level, a flag for if it is active or not (the ‘on’ from before), and a reference to the datasprite.

All of this can be expressed as such:

The red lines representation data movement, and the blue lines show the path from the file all the way to the sprite data in RAM. They also reveal what I was getting at before, which is that Cycle is weird. Cycle is functionally just an index for the animation frame, but the source treats it practically like an object type. It is conceptually distinct from the rest, as though it were an over-arching animation, but in reality there is a ‘Cyc’ for each sprite in memory, and it is really more of an extension of any given sprite, that allows the sprite to change which frame index the datasprite points to at any given time. However, Cycles are also defined by the Story record, as opposed to the object that gets animated. Implying it is, or was supposed to be, a more generalized method of cycling through animation frames of anything, as anything could be given a Cyc. This is just speculation though, in the game we have it is used exclusively to iterate the animation frames of sprites, and also get the data sprite for them some times. Wait what? Why would the cycle be responsible for the data sprite? And why does it have a reference to it when the sprite already has one? Great questions! I am still asking that same question myself. Maybe I’ll come back to this post and explain it better one day. Or maybe it’s just redundant. Or maybe, it’s a relic of a potentially more powerful and interesting animation system. Perhaps we will never know for sure.

Lastly, let’s take a look at which the data types actually look like in practice for our purposes:

Alright, until next time!

Welcome

Since last week, I’ve worked mostly on Story.GS, and the relevant Level and associated functions. My interpretation of Story has changed a little bit, and Level 0 now has a filled out story function. However the bigger change is that I have now added a skeleton of the Room object, and have been working on getting that filled out. I have filled out many functions of room and related to room, but I don’t have quite enough to show anything big yet. I will soon though.

In the mean time, I have a few other things to talk about. Firstly, let’s update the overall game engine diagram because I’ve made a few changes:

As you can see, I’ve realized that I had included a few too many things in room, that were actually a part of Level. Basically, I was forgetting that although torches, objects, and monsters each have in-room sets, doors are actually not owned by rooms, but are in fact simply functions of the entire level, and genSprites, cycles, and sparks are all collective sets that don’t care what room they are in. As such, they are not inside room, but rather a part of the Level. They are still within the object layer however, as they are only applicable to the objects in the game, not the engine as a whole (cycle is a little iffy, but as far as I’m concerned right now, cycle is the object limb of the sprite, and as such it is one layer of abstraction away from sprite and datasprite). The other notable change is that bullet as a struct is now in monster instead of room. This is because the projectile handling code is room based, but the actual projectile creation/deletion is the job of the creature that owns the projectile, which is the monster or player.

Okay, one other thing to mention from recent findings. Story gets very confusing, very easily. This was probably already an assumption, but I’m going to make it a little bit more confusing but also more clear. Here’s a new diagram:

For most of the Story data types, it’s one level of redundancy. You have Story records which are ROM, and then per room you create RAM versions that exist and get cleared as needed. But with torches, it gets a little weirder. The Story record of torches actually gets turned into an array of all torches within the level, where each entry has a reference to the room and the flame pattern, as well as the x,y. Then, when the room is drawn, you create a new set of torches, this time only ones that are for the particular room. So you may ask, why not just do what objects and monsters do and simply be two arrays? Because this way, you can have action permanence in a small capacity. This may not seem like much these days, but having something the player did in a room remain that way the next time they reload the room, in terms of small scale things, is impressive. Basically, the game keeps track of which torches you have lit (and what flame patterns are being used generally as well), so that if you light a torch and then leave the room, it will still be lit when you return, as you would physically expect. On small things like torches, I think it’s a very nice touch that they did this. Especially since rooms can be ‘lit’ or ‘unlit’. However it does make the code more messy in the source and on our end as well.

So to sum up, I’ve added a number of new things, like the room object skeleton, the torch skeleton, and more story related data. More to come soon for sure. It’s been a bit of a slow week in terms of interesting developments unfortunately, but I imagine next week I will have a lot more to talk about.

Until next time, thanks for reading!

This week I worked on shaders, fixing bugs from previous weeks. The first of these was a missing light on the right side of the room. This was a general problem that effected all spotlights (implemented with projective texture mapping), including the flashlight that can be picked up in the first level. Initially I thought the problem was related to a wrong translation of the tex2DProj function of the CG shader language. Thought, after some time, I tried to modify the original game’s shaders in the same way and testing them on the original game. This didn’t reproduce the problem, so I knew I was looking in the wrong place. Turns out the issue was related to the order of the elements in the engine’s matrices. This was easily fixed by adding a transpose before passing the matrices to the GLSL shaders. This had gone unnoticed until now because other matrices are not passed directly but by other means and for which a transpose was done earlier in the process. The next bug was related to fog shaders for which I was very confused by the fact that both textures and other uniforms appeared to be incorrect or missing. This was because the shader wasn’t meant to be used, with the problem being a missing conditional.

Next, I worked on implementing the material classes for the lower-level setting (Material_Fallback(01|02)_BaseLight). The main purpose of these was to support systems that couldn’t use fragment shaders or with few texture units (the number of textures that can be accessed in shaders), with low requiring at least two. Implementing these was not as easy as I initially thought it would be. To replace the fragment shaders, the engine used either an ATI extension if this was supported or alternatively by specifying, through standard OpenGL, how the various textures would be combined. To overcome the low number of texture units, multiple passes are used (where, for example, the first pass is used to compute the light and the second to sample the object’s diffuse texture), and their results combined. At first, I had some difficulties understanding which texture to use at the various stages, and after that there was a problem of everything being too bright.

This happened because, when shaders weren’t used, the diffuse textures were multiplied by a color that reduced the overall brightness,  but when using shaders this was ignored. The issue was present only in the lower settings, while on the highest another mechanism was used. Since this was compatible with the constraints of the lower settings, I applied it to them as well.   

This week I also worked on implementing a limit for the audio tracks that are allowed to play at once. This was done to prevent the mixer from running out of available slots.  

Next week I’ll try to fix some bugs with the physics system and probably start working on saves.  

Thanks for reading.   

There were 3 issues I fixed this week. The rest of this blog will explain them.

The first one was the extra corner pixels being drawn in some rectangles.

While the first thing you will notice in this image is the distorted big black rectangle with colored streaks, that is not the issue I am talking about (Though that is the next one).

This was a simple fix of not overlapping the sides of rectangles at the corners in Graphics::drawRect()

The next issue was the faulty decoding of BITD resources, which gies us that garbled rectangle. This is in fact a regression, it worked well before BITD decoding of 32bpp was changed to support D4 movies.

I created a couple of test movies to check how BITD works. After checking the bytes of the movies I made, I realized that before D4, pixel data in BITD for 32bpp was saved in the ARGB format. But from D4, Director started saving the alpha of all pixels in a row, then the red value, then green and then blue. I was looking for some similarity in both the decodings, not wanting to branch it on the basis of Director version (I was thinking that Director might not have done such a change and there might be some uniformity I am missing), but I ended up writing decoding by a version check.

The end result:

I made sure this does not introduce a regression in D4 movies, and it works fine.

The next issue I picked up was no text in buttons in 32bpp mode. Now this one had more than one thing wrong with it. Buttons in ScummVM Director engine have a MacButton widget, which inherits from the MacText widget. While TextCastMember did work fine in 32bpp (Which uses the MacText widget), buttons had no text in them.

The first issue I found was that the foreground color and the background color being passed to the widget was same. I did change it, but there was this werid bit. The colors being passed to the widget were both black instead of white. Still, the buttons were all white (empty). When I switched the background color to any other 32bpp color, it did fill the button with that color, showing the text in white.

Also, MacText is not using the foreground color we pass for the font in the button, but uses the color specified in the _textLines chunk (A Graphics::MacFontRun struct)

The next problem I observed was that ScummVM was, for some reason, turning 32bpp black (255) into white. This was happening only for MacButton but not for MacText. This was the reason why buttons were white even when the background color we passed was black. When I passed 0x010101FF (nearly black) instead of 0x000000FF (black), the text rendering works. The solution I cam up with is quite hacky (replacing 0X000000FF with 0x010101FF for 32bpp button text), but I must find the root of why 255 is being converted to white and fix it.

This was all for this week. Looking forward to the next one 😀 !

3 months have passed since GSoC started! This week I started working on MacVenture which was also worked on by another GSoC student blorente in 2016. You can see his blog here. My work continues were he left off, fixing the engine to work with the newer MacGui while also implementing some new things.

The first thing I worked on was the borders.

The borders had a bunch of problems. The title wasn’t aligned properly in either direction and it was just broken when you had a window active. If you read my past blogs then you might remember how nine-patch gave me some troubles when I was working with WAGE. This time wasn’t any different either.

It took me a decent amount of time to wrap my head around how they work but eventually I figured things out and realized that the problem is in the division of the border into different areas.

The images below aren’t accurate as far as the scale of the area is concerned but it’s good enough to show the problem and how to solve it.

This was the older way the border was divided into 3 areas. The 2nd rect is stretched to fill the required width while the others are fixed in length. Also the title can only be printed at the start of an rect. This is why the title wouldn’t get centered.

To fix this we can instead divide the border into 5 rects. This time the 2nd and 4th rect are stretched to fill the remaining area while the 3rd one is of the size of the title. This way we can center the 3rd rect and the title with it.

The other problem was that active windows also shared the same set of offsets as the inactive ones. This wasn’t a problem when we were just printing the title in the beginning but with it being centered, I now had to make sure that the same offsets also worked with the active window borders. This required some tedious redrawing and testing but eventually I got it to work too.

After this I starting working on the leftover tasks from the previous work. There’s also the task of adding support for the Apple IIgs version of the games and that’s what I am working on right now.

That’s it for this week. Thanks for reading and see you next week.

This week, I tested and finalized the KEY* Pull Request. It is good to be merged now, and shouldn’t be adding any regressions.

moralrecordings streamed a playthrough of  Eastern Mind: The Lost Souls of Tong Nou on this twitch link. It seems that this game now runs much better than it did 7 months ago, that means that support for D4 games is getting better.

My first task for this week is to look in the BITD rendering regression for 8bpp. sev pointed me towards this commit, this did enable the BITD rendering for 32bpp mode. I have been advised to retain the 8bpp rendering which worked before this change along with the 32bpp which works after this, while introducing no new regressions.

This week, there wasnt much progress, as I had some interviews which ate up a lot of my hours. But that is over now, and I will resume work full time now on the tasks. The next issue in line would be fixing text in buttons in 32bpp, and I would move onto other issues then.

Looking for a great week ahead!

Hi there!

The title is a bit of a pun, because this week was spent primarily on the ‘story’ part of the engine. Last week I mentioned that the next step was to implement level and then move on to the skeleton of the room object. Before getting into Story, I’ll mention right now that I implemented level.cpp (in terms of functions within level and above, ie. calls to the room object are commented out), and I also made a lot of progress on Story, which is divided into story.h and story.cpp. I also needed to sort out a few git related issues, but they went smoother this time, so I think I’m getting more used to it.

Story Time

A game is made up of many parts, some static and some changing. Traditionally, there are many elements of a game that are static, such as the level data (this is not to say that animated tiles are static, but the data itself drawn, compressed and stored separate from the game). The Immortal however, is no traditional game. The further I unravel this ball of knotted up cables, the more surprises I am met with. Sometimes that has me looking like this:

But other times, there’s something really interesting in there. For instance, allow me to tell you the story, of Story.GS.
This file serves the purpose of what is usually ROM data. Data which is created and stored before the game is compiled and run. It contains all the information to describe each part of a level. The game is made up of 8 levels, each of which contains numerous traps, enemies, objects, and environmental elements, spread across a selection of rooms, connected to each other by doors. A level has many properties that must be read from during gameplay and when loading the level. The position of every element within the game universe (the viewport essentially), where the player gets loaded in, the location of the entrance and exit from the level, and much more. All of this can be stored as simple data that gets read from by indexing the level and the room the player is in. Sounds pretty straightforward at first, but things get interesting very quickly. The first thing you would notice, is that almost nothing in the file is anything other than a macro. Every line is a macro, which again sounds straightforward. However looking into the macros reveals the true complexity of this subsystem. And when you read through all of the macros and start to connect it all together, you see that while yes, some of the file is standard looking static data, the majority is actually determined dynamically. We’ll get to that dynamic system in a second, but let’s start by just mapping the file out.
The blue represents the dynamic component we will see in a moment, while the light green represents data specific to the individual level, but not stored dynamically and is treated the same as the darker green section, which represents the main static globally accessible definitions.

In this context, a definition is a macro that writes a set of ROM data, which will likely be used by the main level data, but might also be used by any other part of the game engine. Ex. The title screen text sequence is a set of str definitions (not to be confused with string definitions, or the macro for strings. str is a string macro that is also used for not strings).

The global definitions are very straightforward, just a contiguous series of data like you would expect. It’s the level specific data that gets weird.
By weird, I mean that this set of ROM data (referred to inside the source code as ROM data in fact), is written in what I can only describe as a ‘compiler function’. It is not an explicit function of the compiler, nor is it a routine run in the game. It is a function in the abstract, that runs at compile time. The result of which, is a series of ROM data entries, which are able to reference each other, change what position it is relative to, and return information at the end. It is, quite interesting. There are variables like roomIndex and doorIndex, which get updated as the compiler processes the macros, which are then subsequently used by other macros. They are then cleared and used again, just like memory inside a running program. The most interesting of these however, must be the variable roomNum. This is a variable which is set dynamically based on what the compiler ends up with after processing the story entries. It is then used and affected outside of this compiler routine, in the actual game code. This brings up the question of ‘lexical’ vs ‘dynamic’ variables, but I won’t go into that in this post. Suffice it to say, by using this compiler function at compile time, the compiler is sort of reaching into the future running code with certain variables. It’s fascinating stuff, but ultimately it makes translation very confusing.

What I decided to do, was to translate this compiler function as a real function run during the initialization of level. This does change the individual level initialization, but I believe I have translated that part fairly accurately as well. To understand the structure I decided on however, we need to know the connections between the different dynamically declared data types:

This is not the best diagram, as it does not show every data type, nor does it show the connections in more than a cursory way, but I think it gets the point across. Looking at a single entry in Story, you are effectively looking at a tree of indirect pointers going deeper and deeper. Let’s take a look at a single object to understand better. We’ll use my C++ code instead of the source for clarity:

This is luckily not the most complex object, so we can map it out without too much trouble. But I think it conveys how potentially complex it can get. For example those two ‘kDoNothing’ parameters could also be Use or Pickup structs, leading to further functions from there.

So what I decided on, was to have a struct for each level entry, called Story. This struct has a list of rooms, flames (the torches), objects (items, chests, etc.), doors, and monsters. The universe properties normally in univAt are instead just properties of Story. The rooms, flames, objects, doors, and monsters are all structs as well, and in the case of Obj, they can contain more structs from there. Then, during initialization, all of these structs are populated completely. When a level is loaded, it does not have to search through all the story entries like the original, because they are now in data structures. Instead, it indexes to the story struct, and loops over all the rooms/doors/etc. in the struct and creates the relevant objects and data within the ‘room’ object. I believe this is a fairly direct translation of what is in my opinion, a very non-standard method of handling these data types in the original game.

Next

For this next week, I will be continuing the work on Story I have been doing, as well as creating the skeleton of room, and hopefully I will fill out more if not all, of the story entries (filling out a story level is extremely tedious, it takes quite a while).

Thanks for reading, I’ll see you next week!

I started this week by finishing up the audio system. For now, the sound data class only stores the filename of the audio asset, loading it from the file every time a new sound channel is created (these could refer to anything from the main menu’s music to the sound of one of the keyboard keys in the intro). This, of course, is very inefficient, but it’s the only way I could figure out to get it working without issues.  

In general audio is now working correctly, but there are still some minor features missing implementation. Initially I thought the engine relied entirely on OpenAL for 3D audio but there is also code in the engine to do that and it was easy to enable. For now, to me at least, audio sounds the same as the original, though I’m generally bad at picking up differences in sounds.  

Next, I worked on fixing some memory issued that resulted from uninitialized variables and some leaks. There isn’t really anything interesting to say about this, the tools (valgrind and address sanitizers) did most of the work here.  

After that I started working on physics again. The main problem is that on some surfaces picking up objects is impossible. This leads to an unplayable game as, for example, the hatch in the second level, that needs to be opened to progress, is almost impossible to interact with. This was caused by the physics library ray-casting system that didn’t return correct distances for objects. This worked correctly in a previous version (2.31) of the library though, so fixing it was a matter of running both versions and seeing where they diverged.   

On a side note, sev tried to contact the library author, to see if he still had the original sources for the version 1.53 of the library, which was used in the original game. Unfortunately, he doesn’t have them anymore, which is a shame since everything works correctly in that version. 

After having completed the previous task, I tried playing the game. I had to fix some audio features that were disabled, but overall, it went smoothly. Now the biggest issue is with shaders. Playing the game without them is not the optimal experience.  

The next thing I did then was to get back to shaders. I fixed the most visible problem from the last time I worked on it which was related to normal mapping. The issue was caused by an incorrect transform matrix that was used on the light direction vector. This was the result of a difference between GLSL and the CG shader language. In the first a matrix can be constructed by passing the vectors that represent the columns, while in the latter, the vectors represent rows.

Now thought there are still some problems with spotlights and fog shaders. I’ll work on these next week, and hopefully implement the shaders for the lower-level settings.  

Thanks for reading.  

After last week’s blog post I opened the PR for both WAGE and PINK to get them reviewed. Let’s start with PINK:

For PINK there was a regression caused by a later commit which broke some things. The problem was that the window mode wasn’t being set and the fix was really easy. A bigger problem was the Hebrew text wasn’t being rendered properly. It has been there from the start but I never noticed it because I can’t read it.

Thanks to discord user rzil for the help with the hebrew text. It was kinda hard dealing with text you can’t read and it being an RTL language just made things more annoying. What I ended up having to do was to render the text in reverse order and that was enough to fix it.

With this PINK was done and it got merged.

WAGE spent quite some time in review. The first major issue was Engine code being called from inside Graphics. Another issue was with how I had solved some game specific issues. I had implemented some game specific hacks but a general solution was desirable.

Most of the issues were caused because of how object click detection was implemented. In the current system, each object had it’s own screen to draw on. To detect clicks, ScummVM would just see if something was drawn at that pixel. This was however too precise because the original game seemed to count any click inside the bounding box of the object. I rewrote the object detection code to work like that and it fixed all issues.

There was also a problem with how the bounds of the objects were calculated. Fixing that fixed the issues with cropped bitmaps.

Some formatting issues needed to be fixed too but after that WAGE finally got merged. However there are still some things which need to be addressed. Some functions were stubbed (not implemented) so I worked on those. There’s are some some TODOs from the original JAVA implementation which need to be looked at.

And that’s where I am right now.

Thanks for reading and see you next week!