Last updated:
May 20, 2019 08:00 PM All times are UTC.
Powered by:
PlanetPlanet
Welcome to the ScummVM planet - This aggregates the personal blogs of developers, teams members and active participants from all around the ScummVM community. If you wish to subscribe to updates to the planet or individual blogs please use the links on the right hand side. To add your blog to the planet contact DJWillis.
Last week Google announced the accepted projects for this year's Summer of Code. We are pleased and excited to say that ScummVM will be mentoring four students this year. All four of them will work on game engines:
Jaromír Wysoglad (a.k.a. Jarda) will be working on the second part of the Mission Supernova game.
Antonio Román Cerezo (a.k.a. Kyranet) will be working on implementing networking in ScummVM for the Moonbase Commander game.
Please welcome with us our four adventurous students for what we hope will be a productive and interesting summer! And as always you can follow their progress throughout the summer on the ScummVM Blogs.
Time flies so fast! It's been 10 years since we introduced the "Modern Theme" to ScummVM.
After all those years, it surely was time for an update.
Over the last couple of weeks, ScummVM team member mataniko carefully updated the theming of ScummVM. The new theme follows modern design principles while preserving our own design philosophy.
A couple of days ago, Google announced the organizations that have been accepted for Google Summer of Code 2019. We are very pleased to announce that ScummVM is one of them, together with our sister project ResidualVM. The Summer of Code is a yearly event organized by Google to encourage students to contribute to open source projects. Open source organizations provide mentors and projects to work on, while Google give some money to participating students. You can find more information on the official web site. We also have some information on our wiki, and in particular project rules that applicants will need to follow.
Are you ready for a great adventure? Do you want to contribute to one of the largest game preservation projects known to mankind? Then ScummVM and ResidualVM are waiting for you! Have a look at our ideas page. Of course, you are also very welcome to bring your own idea! Students have until April 9 to write and submit proposals for the project they want to work on. The accepted proposals will be announced on May 6, and there will be a few weeks for community bonding before the start of the coding period on May 27.
In any case, don't hesitate to chat with us on IRC! #scummvm and #residualvm in the Freenode network are the place to go!
Work on the ScummGlk engine is ongoing. Since Christmas holidays I've been primarily focusing on adding support for the version 6 graphical Infocom games to the Frotz sub-engine, with particular focus on Zork Zero to begin with. Whilst I don't count the version 6 games among the best of the original Infocom line-up, ScummVM has always been about providing as good a coverage of old games as possible. So from my point of view it's worth spending the extra time adding support for the remaining Infocom games now rather than later, and only moving onto the next sub-engine once that's done.
Below is a screenshot of Zork Zero as of when I came back on the 7th January:
As you can see from the screenshot, the background is starting to be properly rendered, and the 'A' and location glyphs are properly rendered and interleaved with the text. The status bar still needs some work, though.. although it is being rendered, it's then being completely wiped out by the status bar window, which isn't even displaying any text yet.
Right now, I'm focusing on adding the 8x8 fixed size font used in the original games. If you look closely at the screenshot above, the current resizable TrueType font doesn't really look crisp at that low a resolution, and is somewhat blurry. I'm currently making a tool to use the same font data that Frotz has, and produce bitmaps to add to the fonts Zip file for the different 8x8 font styles (normal, bold, emphasized, and bold & emphasized). It has had it's share of issues:
Somehow, I don't think I've got it quite right yet.. :)
Anyway, progress is gradually being made, and I hope to have Zork Zero fully supported soon. The other v6 games currently error out immediately during startup, but I'm hoping further adding support for them will be a simpler matter.
Seasons greetings to all, and I hope everyone has a happy time as we finish 2018 and look forward to the beginning of 2019. As I sit in the Los Angeles airport writing this, I'm reminded yet again of how much fun I have doing reverse engineering, since even the smallest thread of work can lead you in unexpected directions as you trace the flow of what methods call a given method, and what methods it calls.
For something to do, I opened up my Companions of Xanth in progress disassembly to do a bit of pottering around with it. Just by chance, scrolling through the disassembly, I noticed a call to a method I was pretty sure took in a string, but in this case was passing in a value. I then quickly realised it was an offset n the data segment that hadn't been properly resolved, so I did that. The message in question turned out to be a generic "insufficient memory". So from it's usage, I was able to identify the entire method it was in.. it's called during startup, and ensures that there's enough free memory to play the game. Ironically, it looks like the code to actually display the message if there isn't enough was broken, as evidenced when I ran the game in DosBox debugger.
From there, identified a method just before the message call that took in a number and passed out a string.. it wasn't too hard to identify it as a number formatting routine. I confirmed it in the DosBox debugger by putting a breakpoint in the machine code, and seeing the result from the method. I then noticed that there was one other method calling it, so I was intrigued to find out what it was. It turned out it too was a block of code in a method that printed out the free memory, in a method that looked like a big switch based on a single value. It looked like a keypress. Playing the game, I was able to confirm that pressing 'M' on the keyboard did call the code, which printed out the amount of free memory in the status area.
This was a good woo-hoo method. I had identified the method for handling keypresses. Likely, some of the other keypresses do fairly identifiable things, like saving and loading, so knowing where each key's code is, I'll be able to trace into what they do, and what methods they call. In fact, the free memory printing for the 'M' key already prove dividends. From it, I was able to identify the method that cleared the status area, and a version of 'sprintf' for printing text in the status area. In the future I'll be able to look further into other places that call the method to add lines to the status area, and start identifying other parts of the game code. For example, looking at an item produces a description in the status area, so by setting a breakpoint in the status text writing code, I'll be able to identify what method calls it, and start identifying the structure of items that likely have a "description" as one of it's fields.
All this progress today came from a minor diversion to pass the time sitting in the airport. The notice of an incorrect parameter to a method call has unraveled more of the game's secrets, and giving me multiple further areas I can investigate. A nice way to start the holidays.
One of the things I like about my work on ScummVM is that it's a hobby rather than a profession, which gives me the freedom to switch what I'm working on when I desire. I've previously had encouragement to resurrect my Gargoyle engine work from about a decade ago. It was an early attempt to add support for Frotz into ScummVM, and at the time it was rejected due to multiple reasons. It was specific only for Infocom/zcode games, lacked clipboard, truetype fonts, and Unicode support. Most of these things have since been solved by the ScummVM framework itself, which now supports all of them. So I thought, why the heck not, and decided to work on it again.
However, my original Gargoyle engine being limited to just Zcode was a problem. Not only wasn't it extensible to other IF systems, but it would also take work to properly reimplement it to support Unicode. In the end, it was just simpler to scrap it all, and start from scratch. Luckily, there's the very convenient Glk specification, that was specifically created for easing porting the various Interactive Fiction interpreters to other systems. I knew that by supporting it, it would make it easier to create a framework that could support multiple different interpreters as sub-engines.
So I created a new Gargoyle engine, which I've since renamed it to ScummGlk to avoid ambiguity with the stand-alone Gargoyle project which also implements the Glk specification. After over a month of work, I've created an engine that implements the bulk of the API, and implements two sub-engines under it so far.. Scott for Scott Adams games, and Frotz for playing Z-Code/Infocom games. Scott was useful as a test case, due to it's simplicity. With it as the first sub-engine, I was able to focus on getting the ScummGlk core working. Then Frotz was a fairly obvious second choice, given it's popularity and large number of available fan games.
As of last weekend, the review period was completed for the engine, and it was merged into master. Daily builds now include support for both sub-engines. I've spend the last week fleshing out detection entries for all the ZCode games on the if-archive, so all the games will be automatically detected by the launcher. I'm going to wait on announcing an official testing period until I finish support in Frotz for V6 games. But if you can't wait, you're certainly welcome to jump the gun and try playing some games. Maybe one of the early Zork from GOG, or the immortal classic Freefall :)
Having the engine included into master will be a real nostalgia trip for me. Even prior to my original Gargoyle module, and before I even started working on implementing adventures in ScummVM, I spent several years as part of the Interactive Fiction community. Indeed, the Hitchhiker's Guide to the Galaxy was one of the first computer games I ever had. I spent way too many hours getting frustrated with the game's puzzles.
Over the last couple of days, we moved our downloads to a new mirroring infrastructure. Previously, each and every download was delivered by our main server located in Austria.
Now, our downloads are delivered through MirrorBrain, a download redirection system that allows us to build our own content distribution network. MirrorBrain is already used by many open source projects.
MirrorBrain tries to always select the best mirror based on your location, so you will get the best download speeds possible.
While we already have a few mirrors located in Europe, we are currently looking for people willing to donate some bandwidth and storage located in the US and in other parts of the world. If you want to mirror the ScummVM repository, please contact us via E-Mail at serra@scummvm.org.
If you run into any issues with our new download system, feel free to leave a comment in our forums.
It's been a while I haven't posted something about what I'm doing... So here it is.
There's nothing new concerning Kingdom: The Far Reaches. I'm still waiting for a hand with the MVE player, and TMM (who implemented the MVE missing opcodes in ffmpeg) is still super super busy. So, well, let's consider it's rotting. :(
I made some very small progress on Adventures of Robin Hood (lilliput engine) on the sound, but it's still very imperfect. To implement it, I think I should add a mixer, we'll see...
More recently, let's say a week ago or so, I started documenting Escape From Hell. I currently have half of the functions renamed and most of the structures. I'm not really sure it'll be an engine for ScummVM as it doesn't use a mouse, but it's fun to make good progress on something I vaguely looked at more than 10 years ago. At the time, I guessed the compression format, picture format, and enough of the maps and monsters to display maps. I even brute forced the encryption...
All that is now far more clear with an eye in the binary :)
Ho, I forgot to mention: Escape From Hell is using a modified Wasteland engine, in case it rings a bell :)
During GSoC 2018, I was working on adding support of Pink Panther games to ScummVM: The Pink Panther: Passport To Peril and The Pink Panther: Hokus Pokus Pink.
added finding similar colors in the palette to represent ActionText’s RGB colors, which aren’t located in the palette.
bugfixes
Now the colors are implemented, only text is left. Text which is used in text files was written using Windows codepages(1250, 1251, 1252, 1255), depending on the language. To use text in ScummVM I need to convert it to UTF-32 and add unicode support for MacText.
Probably I will hardcode conversion tables, as I don’t see another way how to do it.
This week, in addition to finishing up the Harry Mudd mission, I’ve done the first half of
mission 5, “Feathered Serpent”. This is the only mission to be divided into two “away
mission” segments, with another ship segment between the two.
In classic Star Trek style, we encounter the Aztec god Quetzalcoatl (Quetzecoatl?)
attempting to bring enlightenment to the Klingons. Lt. Buchert must have jinxed us when he
mentioned the absence of any angry gods last mission.
The first half of the mission consists of eight rooms, which is technically a record so
far, but some rooms are little more than scenic pathways. It’s also very linear compared
to other missions. There’s only one case where you might have a reason to backtrack at
all, and often you’re prevented from going back to previous rooms entirely.
This mission has some rather pretty screens.
Mission oddities
There is a case where you can scan Lt. Stragey (the redshirt) while he’s dead, and
someone who is… clearly not McCoy, say “He’s dead, Jim”. Was DeForest Kelley sick that
day? Well, he says the same line in other places, so I substituted one of those in.
There are at least two ways to get an infinitely high score in this mission. The first way
is to repeatedly try to grab a snake in the second room. Whether you succeed or not, you
get a point each time. (Normally the snake retreats into a hole before you can grab it.)
The second way is to repeatedly solve one of the puzzles. I don’t really want to give away
any solutions, but there’s a particular action you can do repeatedly on the lake screen.
The first time, it wards off a monster, but after that it serves no purpose, despite
continuing to give you points.
Unfortunately, based on my limited testing, it looks like your score does cap at 100% in
the final report.
Unused stuff & version differences
The devs seemed to have a good sense of humor; in particular, the original floppy version
was snarkier in places. If you use a knife on yourself, it says “You won’t break your
contract to the network that easily!”
That line was changed later; this particular piece of text changed between the original
floppy release, the mac release, and the voiced release. Supporting all 3 will surely be
a nightmare.
Also, it turns out that the redshirts are named after the devs, which somehow makes the
redshirt deaths seem even funnier.
A personal favourite of mine.
I’ve also encountered the string “***Game Over, Man!***” in the game files multiple
times, at least back in the first mission; perhaps it was used before they implemented the
“game over” menu properly. Today, I even found a voiced version which seems to be unused,
done with all the professionalism of any other line. I wasn’t a fan at first, but this
narrator voice is growing on me.
Anyway, there are just two and a half missions to go now. I think I could manage to finish
this within GSoC, starship sections and all. I doubt I’ll have enough time to finish
Judgment Rites, but perhaps I can get a decent chunk done if space combat doesn’t turn out
to be too insane to implement.
Mission 3: Love’s Labor Jeopardized. It’s about 90% finished.
The mission starts with a rather ear-grating computer voice. Thank god they got Majel
Barrett for the sequel.
That said, this is a neat mission. You’re given information about various chemical
compounds that may be useful in the mission, then you need to figure out which ones are
important and mix up some ingredients to cure a virus that’s running loose through the
station.
Or you can just, you know, inhale some laughing gas instead.
Mission bugs
Last week, I promised there would be bugs in this mission. There’s nothing
mission-breaking that I’ve found, though, mostly text stuff.
The most eggregious is a weird textbox I encountered in my casual playthrough. As far as
I know, you’ll always encounter this if you take long enough during the mission.
I also encountered this following bug while writing this blog post. This is part of the
10% of the mission I haven’t rewritten yet, though, so I can’t explain what it’s about.
Aside from that, there are a couple of cases where voice acting doesn’t play when it
should. And, if you look at Dr. Marcus when she’s tied up, it plays the wrong file
entirely; only narrating the first half of the textbox. There was an unused audio file
that narrated the entire textbox, so I replaced it with that.
The one thing that might be considered a bug in the mission mechanics, is the way you can
give water to the Romulans. Apparently you can get a point in your mission score for doing
that. But, sometimes, if they’re unconscious, nothing happens. You lose the water, you
don’t get any points, and not even a textbox is shown. It’s honestly kind of odd. I’m
still debating what to do about that; for now I’ve made it so you can’t give water to them
while they’re unconscious.
Other oddities
There are a number of cases where a single line is read multiple times by the actors,
simply because the same textbox is shown in multiple different rooms. For example, if you
scan the air with the medical tricorder, McCoy says the exact same line, but with
a different voice clip depending on the room. I can just imagine the cast complaining
about why they have to record the same voice clip three times.
Anyway, this mission should be completable in ScummVM within the next day or two. Next
week will feature the return of Harry Mudd in one of this game’s weirder missions.
Our previous release is already a few months old, it's time for a bug-fix release! ResidualVM 0.3.1 is available on our downloads page. It fixes a couple of Myst III bugs:
Fixed a rare crash when walking near the magnetic rings in Amateria.
Fixed unresponsive controls for the holographic projectors in J'nanin.
That's not much, but Cyan and GOG.com are planning to use ResidualVM for the Myst III digital release and we want it to be as stable as possible.
In other news, progress on adding support for The Longest Journey to ResidualVM continues thanks to our GSoC student Liu "Douglas" Zhaosong. You can follow his work on his blog.
Star Trek’s second mission, “Hijacked”, is finished. It’s very short, only consisting of
4 rooms. Despite this, the devs didn’t fail to insert a number of bugs into the mission,
most of them in the final room.
Let’s start simple: if you talk to McCoy at a specific time, he has no text.
However, there is an audio file which fits perfectly for this situation. So, I whipped up
some corresponding text to go along with it, and inserted it appropriately. Honestly, how
could I let DeForest Kelley’s wonderfully snarky voice acting go to waste?
It’s not the only voice clip that’s unused, either. There are a surprising number of cases
where they clearly intended for some dialogue to be said, but there’s problems in the
implementation causing it to not occur; for example, when you try to shoot the Masada
crewman, he’s supposed to say something, but due to a bug(?), he didn’t in the original.
There’s also an oversight with the mission scoring system. You’re supposed to combine some
inventory items together to progress through the mission, but you only get points for
this if you do it in a specific room! This is the consequence of each room having entirely
separate codebases; the code needed to be copied for each room, and clearly, they must
have forgotten that the code was duplicated when they made some changes to it.
Lastly, the final room is pretty buggy. Don’t read any further if you want to avoid
spoilers.
So, there are three ways to end the mission; either you shoot the Elasi, they surrender,
or they attempt to deorbit the ship. But… these aren’t mutually exclusive. In fact, all
three can happen, and the devs clearly didn’t think this through properly.
Consider scenario #1: You shoot the Elasi. This aggros them, and they immediately kill
your redshirt, followed by Kirk. However, you can still talk to the Elasi if you’re fast
enough. If you convince them to surrender… they don’t actually surrender, despite
saying so, they just keep killing you. If they deorbit the ship, you can call Sulu,
stabilize the orbit, and beam out, WHILE THEY’RE STILL SHOOTING YOU.
Scenario #2: You talk to the Elasi, and you choose the worst option, causing them to both
deorbit the ship and start shooting you. This one’s funny, because there’s no way to
finish the mission in a way that makes sense. You can do one of two things:
Scenario #2a: Shoot the Elasi. This is sensible since you only have a few seconds before
they kill you. After this, the mission is “successful”, even though you never save the
Masada from deorbiting. I guess the security team that beams over dies when the ship goes
down…
Scenario #2b: Call Sulu to prevent the ship from deorbiting. This results in the same
issue as scenario #1.
It’s a mess. I’ve tried to fix things as best I can, by requiring that if the ship is
being deorbited, shooting the elasi doesn’t end the mission, and vice-versa; saving the
ship from deorbiting doesn’t end the mission if the Elasi are still a threat. Also, if the
Elasi say they surrender, they actually do surrender now.
There are many more minor bugs, such as the Elasi guards freezing up if you shoot one of
them at a specific time; but I’d be here all day if I listed them all. I’ve put down
“BUGFIX” tags in the source code whenever something like this comes up. I’ve no doubt I’ll
encounter more while working on the next mission.
As I mentioned in the previous post, we still have issues with Planet for some reasons, so I'll keep putting links here until it's fixed, or until the end of GSoC, whatever comes first.
You could also prefer to directly check Drenn's blog. It's There!
We had, and still have, some issues to synchronize Drenn's blog with planet (in the scope of GSoC 2018). I therefore decided to post links to his blog posts so it's visible on Planet.
Here is the first blog entry posted by Drenn more than a Week ago. He made impressive progress on his Star Trek engine. Enjoy!
After all this time, support for the Might & Magic Xeen games is finally close to being finished. It's currently the official testing period for all the Xeen games, and I've already had a whole bunch of useful bug reports, which I've all taken care of. At this point, it's almost certain that the Xeen engine, and games, will be supported in the next official ScummVM release. It's a nice feeling. More than any other game engine I've worked on, the Xeen engine was the one that got put on hiatus the most number of times. It's very satisfying to finally complete it.
What's in store for me now? Well, I do want to spend some time properly unwinding and actually playing some games. They've only just come out with the sequel to Pillars of Eternity, which I'm now getting into, and plus I have a backlog of other RPG and adventure games that I want to have a chance to properly play. So that will probably keep me occupied for a few months at least. I've also got a move from the US East Coast to West Coast coming up next month. So various preparations are consuming my time, and it's all to the good that work on Xeen has already finished.
Not that I'm totally stopping work on ScummVM stuff. As with the early days of work on Starship Titanic, I started doing some casual reverse engineering during my lunch breaks. By happenstance, I experimented with the Ultima 1 executables, and become enamored by how simple they were to reverse. Both due to the simplicity of the game (considering how old it was), and the way the code was implemented. It was just the thing for doing casual reversing work in chunks of an hour at a time.
As a result of this, I've already implemented a great deal of foundation code in ScummVM for supporting the game, with a long term eye for having a clean engine structure that could support all the early Ultima games. Here's some screenshots of the player within the overworld, a city, and within the dungeons:
At this point, I've got basic movement functionality working. The player can roam around the map, entering cities, castles, and dungeons. Dungeon movement also works, as does going up and down ladders, so the dungeon can be further ventured down into, or left entirely. That's it at the moment, though; it doesn't yet have any combat, magic, inventory, character interaction, etc. See this Youtube video for an example walk-about that covers venturing into a dungeon, and then heading over to the city of Britian.
Don't expect too much more work on this in the near term, though. As I said, this is more a side thing, and I definitely do want to spend some time unwinding and playing some games. When I get my fill, later in the year, then we'll see. At the moment, it's looking strongly like I'll keep working further on this. I've some crazy ideas about not only adding support for the game, but also reusing the Ultima VI tileset to create an "enhanced" Ultima 1 experience that's more visually pleasing, and rework the controls and user interface to be likewise more player friendly. I know that oh, pretty much everyone who ever played their way through the space simulation section would much appreciate mouse control for aiming at enemy vessels. :)
I kind of consider the RPG series of Ultima, Might & Magic, and Wizardry the triumvirate of major RPGs from the last century and, now that ScummVM is embracing RPGs in addition to adventures, I'd love to see them all under the ScummVM banner. Now that at least some of the earlier Might & Magic games are supported, it may well be time to start in on Ultima support :) Of course, I'm not entirely abandoning working on adventure games. I'd previously made some promising initial progress in reversing some of the Legend Entertainment games.. Companions of Xanth and Gateway. Irrespective of how much time I end up devoting to working on Ultima I, I may end up also devoting some of my time to those games as well.
Anyway, that's probably enough for now. I still want to get in some Pillars of Eternity 2 before it's time for bed. :)
ResidualVM 0.3.0 is released. Go grab it. Let's get it over with, this new version does not have any new supported games to offer. But that does not mean there is nothing interesting in there. It's packed with almost three years worth of fixes and improvements. Most notably:
Better compatibility with newer operating systems thanks to switching from SDL 1 to SDL 2.
A multilingual user interface.
A widescreen mode for Myst III.
Both Grim Fandango and Myst III have received a lot of awesome user play testing, and as a result many bugs have been fixed.
Detailed release notes can be browsed in our NEWS file.
What's next? The development version of ResidualVM contains two additional work in progress games, Escape from Monkey Island and The Longest Journey. A lot of work remains before they can make their way into a release. But if you are interested in helping, contact us!
With all the progress made on the supernova engine in ScummVM since the last post, I decided it was time to post an update. And in case you can't wait to know the answer to the question asked above, watch the video at the end of this post. But it would be a shame not to read first about the amazing work done lately ;)
Translations
Thanks to the many suggestions from JenniBee, and some help from Joefish who reviewed all the translations for which I had a doubt, the first pass of the translation for the first Mission Supernova game is now completed. And I even had time to do a second pass for the first half of the game while testing the engine, and so improve the translation with the help of the context.
Now what would help the most to get the translation completed is for native English speakers to play the first half of the game and suggest improvements on our translations web site.
We also already have a lot of suggestions for the second game, but it is a bit early for me to review those as I would like to focus on completing the engine for the first game.
As a reminder, the game is available for free on the original developer web site and the source code for the scummvm engine is on GitHub. To play the game you will also need to get the supernova.dat file that contains the game text (both German and English). I have uploaded a version on my dropbox, but you can also compile the devtools/create_supernova tool yourself (available from the same repository as the game engine) and execute it to generate the data file.
Engine
With Strangerke we made a lot of progress on the engine, getting to a point where more than half of the game is fully playable without any major issues, and this includes the supernova explosion! To get to this point, we needed to:
Implement the dialog system. In the early game there is no dialog (you will understand why when you play the game), so the dialog system had not been implemented yet. But it was required to progress beyond the first scene where the protagonist meets a fellow creature capable of speech.
Implement the event callback mechanism. There are a several events in the game which are on a timer, the supernova explosion being the first of those. I am not a fan of timed event, but it kind of make sense in the context of the game. And we are not going to change the game anyway ;)
Fix the graphics pipeline. There was some minor graphic issues, so major ones, and some catastrophic ones (causing a random crash due to out of bound access when trying to access the image at index -1 in the image array).
Fix a palette brightness glitch that caused some rooms to be entirely black. This made it quite hard to progress.
The graphics is an interesting aspect, so I will now give a bit more details about that point. It will also give me a good opportunity to add some much needed illustrations to this post. In the supernova engine, images contain multiple layers. The first layer always contain an image for the full screen (minus the inventory area, except for a few images that really are fullscreen) and then additional layers can be displayed or not for example to make animations, or depending on the context (door open or closed, object taken on not by player). Each room has an associated image and an array of boolean that indicates which layers are visible. When we render the room we thus iterates on all the layers and render the visible ones.
During animations though, we only render a layer and all those above it or "unrender them". To unrender a layer, we simply render the room starting from the first layer, for the area previously occupied by the now hidden layer.
The first graphical glitch found was due to setting the layer visibility flag in the wrong place when showing a layer, which caused the flag to be set not only for this layer, but also for all the layers above it. And we ended up with layers drawn in cases where they should not be.
What is this brown thing in the toilets?
Cleaned version.
Another minor issue with similar circumstances is that there was a logic bug that caused a single layer to be drawn in some places without redrawing layers above it.
I also introduced graphic glitches myself when rewriting the code to reduce memory usage. Initially the code had been implemented by using a surface of the full screen size for each layer. But most layers are a lot smaller and I rewrote the code to only use a surface only as big as needed for each layer. I got the surface pitch wrong however in the code bitting the background layer to screen when "unrendering" a layer. That resulted in some interesting animations...
I know he is ugly, but was that a reason to censor his face?
Also, two more hours before what?
Then more recently I stumbled upon a more serious issue that required rewriting a big chunk of the way images were rendered. The issue was that the wrong image was being rendered, and with the way the engine had initially been implemented in ScummVM there was no way to tell the rendering code to use the correct image. This caused the random crash mentioned above (when trying to render image -1), and when the crash did not occur (which happened more often than you might think), an interesting cutscene mixing layers from two images.
Flight cutscene, before and after fix
Rewriting the graphics code to fix this last bug gave me the opportunity to also easily change the way the graphics data are loaded. The initial engine implementation in ScummVM was loading all the graphics data when starting the game. Now they are loaded on demand, which should reduce considerably the memory usage (and speed up a bit engine startup on slow platforms).
Conclusion
And now, finally, enjoy the supernova explosion!
But I won't tell you when it occurs so that you have to watch the full video of me playing the game (but at least I recorded it with the English text and not the original German one, and I slightly edited it to make it a couple of minutes shorter).
As you may be aware I was a mentor for the ScummVM organisation for the Google Summer of Code this year. I was supervising the work by Joefish on the Mission Supernova game. The engine has not yet been merged into the main repository, but work is continuing in Joefish's GitHub fork.
At the end of the GSoC coding period the first chapter of the game was completable. However the game is only available in German. One of our aim is not only to make it possible to play this game again on all platforms supported by ScummVM, but also to play it in English. And this is what I have been focussing on in the past week.
I have actually been working on the translation for a while with the help of Joefish. We are using the ScummVM translations website to handle the translation. With previous games when working on a translation we would use more rudimentary tools, such as a shared Google Doc spreadsheet. Thus using our translations website to translate a game is an experiment, but so far I have found it a positive one. And I expect it will also allow participation from the community to complete or improve the translation (this has not been the case yet as we did not advertise this translation until now).
What I was focussing on this week is writing a tool to generate an engine data file, as we do with other engines. Until a few days ago all the text was hardcoded in the engine. But the idea is to have the text in this engine data file instead and to have the engine load all the strings when it starts. This has two main advantages:
The strings being into a separate file instead of being in the engine, the ScummVM executable ends up being smaller and using less memory when the engine is not running (for example when playing another game).
We can store the strings in several languages in this engine data file and the engine will load the strings for the language selected by the user when the game was added to ScummVM. This means we can provide translations for the game.
Format of the engine data file
The data file starts with a 3 byte identifier ('MSN') that can be used to identify the file, and a 1 byte version number so that we can change the format in the future without breaking everything.
The rest of the file is a collection of blocks, with each block having a 12 byte header and a variable size data section: byte 1-4 marker, for example 'TEXT' for the game text byte 5-8 language code, for example 'de' for German byte 9-12 block data size in byte byte 13-N block data
This file structure means that we can easily add other types of data in the future if needed, such as the font data (that is currently hardcoded in the engine) or the mouse cursors (also currently hardcoded in the engine). The marker tells us what type of data we will find in a block. Currently the markers used are 'TEXT' (game strings), 'IMG1' and 'IMG2' (for newspaper bitmaps).
With this file format we can also easily add translations to more languages. Currently we have data for German and English.
Generating the engine data file
The German text is hardcoded in the tool that generates the engine data file. All other data however are provided with supporting files. The tool has a list of languages for which it will look for those supporting files. Currently for each language we can have 3 files:
strings-xx.po
img1-xx.pbm
img2-xx.pbm
(where 'xx' is a language code)
For each file found a corresponding data block will be added in the engine data file.
The strings-xx.po file is typically generated and updated by our translations web site and contains the translation in a given language for each German string. The German strings are defined in a specific order, and when generating the TEXT block for another language the tool iterates on the German strings, for each string looks for a translation in the po file for this language, and if found write this translation and otherwise write the original German string. This means that untranslated text will appear in German when playing.
The ppm files are bitmap images in portable bitmap format for the two newspaper article images in the game. They can for example be generated with gimp. Other than the header, the ppm format happens to store the data in exactly the same format as the game does for the original bitmaps in German. So the tool simply reads the header to do some sanity checks on the format and image size and write the rest of the file to the engine data file.
So adding a new language simply means adding its code to the language array in the tool source code, and dropping some pbm and po files in the directory where we execute the tool.
The game contains 655 strings. Currently 281 of those have been moved to the engine data file, and this includes the name and description for all the objects. A lot of strings are still hardcoded in the game engine though, such as all the dialogs, and they will be moved to the engine data file as well eventually.
The progress on the translation to English is similar, as 279 strings are currently translated, although some of those are for strings that have not yet been moved to the engine data file, and can thus not yet been seen translated in the game.
Earlier this month we announced two projects for this year Google Summer of Code to add support for the Sludge engine and for the Mission Supernova games in ScummVM. I am a co-mentor for the Mission Supernova project (the other mentor being Strangerke). We were lucky enough to be provided with the original source code for Mission Supernova (for which we have to thank the rights owner). With the coding period for GSoC starting officially tomorrow we spent the last month looking at this original source code. Interestingly, in addition to the source code for the game we were also given the source code for some tools. One of those converts a MOD music file to a game data file. And I though it would be interesting as a side project to reimplement it so that it works on modern computers, and to then extend it to perform the reverse conversion from game file to the original MOD file (which we don't have).
I thus spent a few days working on this in the past two weeks.
We have been asked not to share the original source code (and anyway you would have to be a bit of a masochist if you want to see C code from over 20 years ago). But I will show a small extract to give you an idea of the work involved. The original source code is in C and for DOS.
The source code for the mod conversion tool is very compact and starts with these two functions:
For anyone familiar with supporting both big endian and little endian platforms, what they do is obvious. They swap bytes to convert between big endian and little endian representations.
Conversion between big endian and little endian should come as no surprise. The MOD format was originally developed for Amiga, which are (or at least were at the time) big endians computers. Looking at the specifications of the MOD format shows that it is indeed using the big endian convention. On the other hand the DOS operating system was working on little endian computers. Using a little endian format for the music in Mission Supernova thus made sense to avoid having to swap bytes during runtime. Every little helped at the time to get good performances...
Another thing visible in the code above and that should come as no surprise (at least for developers dealing with old platforms) is that an unsigned int is coded on two bytes (and not on 4 bytes as you would expect nowadays) and a long int uses 4 bytes.
The last point we can note is that functions and variables have German names. Fortunately for me I did study German at school and could understand most of the code straightaway without having to ask Google translate (or my German sister in law) for help.
The first step of my work was to rewrite the code of that tools so that it works on modern computers, whether they are using big endian or little endian conventions, and can be understandable by others.
I replaced the byte swap function from the original source code with code we already have in ScummVM that handles byte swapping depending on the platform on which the code is run (so that for example reading a MOD file would only swap bytes when the code is run on a little endian computer).
I replaced data types such as unsigned and long with types provided by ScummVM such as uint16 and int32.
I rewrote the code to use ScummVM Common::File API instead of the low level DOS file access code.
I translated variable and function names to English.
I objectified the code a bit adding a ModReader class.
At this point, without the original MOD file, I had no way to know if the code I wrote was correct. Writing this code however helped me understand the differences between the MOD format and the format used by the Mission Supernova game.
The two formats are very similar, but besides the different endianness, there are a few other differences. Actually the format for the two parts of Mission Supernova is slightly different.
Here is a description of the MOD file header:
And one of the Mission Supernova part 1 data file header:
For the Mission Supernova part 2, there are only 15 instruments stored and not 22.
Note how some information is missing in the Mission Supernova data file. That means that we have to guess what that information should be when converting that data file back to a MOD file. Fortunately none of that missing information is really important. For example for the song name I just decided to use the name of the MOD file that was hardcoded in the original source code.
Some other information is just formatted in a different way, such as the Mission Supernova instruments data having a loop start and loop end instead of a loop start and loop length.
Also the Mission Supernova data file stores explicitly the number of patterns and the offsets of the samples data. Those have to be computed from other informations in the MOD format.
The other difference not seen above between the two formats is in the pattern data. Both are using 32 bit values, but they are not coded in exactly the same way. For details on the differences just look at the source code and comments in the rewritten tool.
This knowledge of the MSN music data file might be useful when we have to work on supporting the music in the game engine reimplementation. For now I used it to write some code to do the conversion the other way around: from the game data file to a MOD file.
This allowed me to check that the code is correct:
By checking that the converted MOD file I am getting is played correctly in a player supporting that format.
By doing a round trip conversion: converting from MSN data file to MOD and then back to MSN data and checking that I get back the original file.
My first round trip test actually resulted in the original and converted MSN data file having a one byte difference (every bytes were identical except one). The offset of that bytes indicated it was the second byte of the order list length value, coded on two bytes in the Mission Supernova format. And then I realised that I was using a char variable (that uses one byte) since in the MOD format the order list length is coded on one byte. Writing that variable on two bytes meant the second byte was garbage.
Implementing this reverse conversion also allowed me to listen to the music without waiting for the games to be supported in ScummVM. And to let you enjoyed that music as well, here are recordings for the music of the first and second parts of Mission Supernova converted to MOD and played back in an OpenSource ProTracker clone.
Time flies so fast! It's been 10 years since we introduced the "Modern Theme" to ScummVM. 
This mission has some rather pretty screens.
A personal favourite of mine.
October 01, 2017
