January 23, 2017 04:00 PM All times are UTC.
Welcome to the ScummVM planet - This aggregates the personal blogs of developers, teams members and active participants from all around the ScummVM community. If you wish to subscribe to updates to the planet or individual blogs please use the links on the right hand side. To add your blog to the planet contact DJWillis.
The Dude... the Pipe... the Dark-Burrow-Dweller... and you.
After several years of development, the ScummVM Team is pleased to
announce full support for the obscure, fancy, and engaging adventure
game The Full Pipe (Полная Труба in Russian). Grab your
Game CD (Android and iOS versions are not supported),
the daily build, and help us
test Dude's adventures in the world of Small Fries.
The game is not available in English, but doesn't have any text other than
in the menus. Thus if you don't have the game already you can buy the German
version published by Daedalic Entertainment.
Until now, translating the ScummVM software was not straightforward. Contributors had
to get the translations template file to their computer, update the translation using a
dedicated application, and then propose the updated translation to the ScummVM team
using a Pull Request
or our bug tracker. The ScummVM team
itself also had some manual tasks to perform, such as updating the translation templates
from the source code when needed or rebuilding the binary translation file after receiving a
All this belongs to the past. Now, thanks to our new
translations website, powered by
Weblate, updating the translations is easier and more
comfortable than ever before. Translation templates are updated automatically when needed,
and so is the binary translation data file. Translators now just have to translate. That's it. The
website also helps translators by performing automated quality checks and suggesting
Finally, the new site also adds a community aspect to translations, enabling users to vote for
suggested translations and even make their own suggestions. Anybody can view the
translations, but you will need to log in to make suggestions, comment, and vote. You can
log in using your GitHub or Facebook account, or by creating an account on the website. You
will still need to contact us if you want to contribute further (for example to start a new
translation or help on an existing one), as we will need to grant you access to the Translators
group. Unlike regular users, translators can accept or reject suggestions and modify
translations directly. Contact us by visiting the #scummvm IRC channel on irc.freenode.net or
by sending an email to
firstname.lastname@example.org if you are
interested in becoming an official translator. Otherwise, head over to
and help us make ScummVM accessible to everyone.
A lot of things happened with xoreos this past year, albeit most of them hidden and “under the hood”:
I wrote about disassembling NWScript bytecode. The tasks I mentioned there are still open, too. If anybody wants to take them up, I’d be happy to explain them in more detail :).
We released xoreos 0.0.4, nicknamed “Chodo”. That was the only release of xoreos in 2016. xoreos 0.0.4 included some minor fixes and features for Neverwinter Nights, and the xoreos-tools package included the new NWScript disassembler.
In April, I reached a streak of a full year of daily xoreos commits. Due to some real life things, I had to take a break there, though. I’m now again at three months of daily commits, but there is a three-month “hole” between April and August.
GitHub contribution graph in April
GitHub contribution graph in November
Farmboy0 fleshed out the Jade Empire engine a bit, mostly in the scripts department.
I then implemented a few more animation script functions, too, which is especially noticeable in the intro animation for Hordes of the Underdark. I also fixed a mistake in the keyframe interpolation. This takes care of another glitch in Neverwinter Nights: model nodes rotating the wrong way around.
I fundamentally restructured our build system, or at least the autotools part of it (xoreos can be built using either autotools or CMake). Previously, we used a recursive autotools setup, where make recurses into each subdirectory. This is, unfortunately, pretty slow, among other drawbacks. I changed it to be non-recursive now, with the top-level Makefile instead being created using (recursive) includes.
I then introduced various smart pointer templates into the codebase, making it easier to read and easier to keep track of memory allocations.
berenm added AppVeyor integration. Like Travis CI (which we already use as well), AppVeyor is a continuous integration service. This means that every single commit to the public xoreos repository will now be built on Microsoft Windows, using Microsoft Visual Studio 2015, in addition to gcc and clang on GNU/Linux (via Travis CI). This ensures that any compilation breakage on these systems is immediately visible and can be fixed at once.
mirv is still working on rewriting the OpenGL renderer.
I am currently writing unit tests for the xoreos codebase, using Google Test. I already found multiple issues, bugs, and corner cases while adding them.
From my side of things, my current plan is to make my unit tests branch public some time in December. I’ll write a small announcement here about it then. A new release of xoreos, 0.0.5, should follow early next year.
As always, this all wouldn’t have been possible without a lot of people. For them I am thankful.
Farmboy0, for various fixes, implementations and file format spelunking.
Supermanu, for his character generator work and pathfinding research.
mirv, for continuing to work on the OpenGL rewrite.
Since my prior posting earlier in the year, there's been a great deal of progress in Starship Titanic. I decided to put aside the problem of reverse engineering all the Star Map classes until I had the rest of the game working better. In that respect, I've made great strides since, as of last weekend, I was able to complete the entire "prologue" of the game That included using the computer, experience the crash, talking to the Doorbot, entering the ship, and viewing the Credits. Huzzah. \o/
I was going to prepare a video showing the intro, but with the most recent changes, there seems to be some instability showing up. It seems like something that was already present, just coincidental that the newer changes result in more frequent crashes. It's kind of hard to narrow down the cause, as there's also a problem with the implementation of the Indeo video decoder we're using for NPC videos like for the Doorbot, where it's reading past the end of the frame data. So it's difficult to track down the memory corruption, as warnings about the decoder are completely overwhelming everything else.
So for now, I'll present a screenshot of the amazing multi-color Doorbot :)
After thinking over matters, I've decided to keep progressing into the game, and come back to look at the problem later on. Part of the trouble I'd been having with the code was the sheer length of the intro as I got further and further into it.. requiring me to wait through several minutes of cutscene & conversation every time I made any changes or bugfixes. Even if the intro has suddenly become unstable, I still have savegames I made from beyond it, so I'm using them as a starting point to make further progress testing into the game.
Speaking of testing, I've had a major boon to my efforts to track down bugs in the code. I was previously stymied trying to test the original Windows executable in the IDA debugger, since it kept crashing on me. Plus running in compatibility mode full-screen didn't help either. And without the ability to see "valid" values in the original executable, I anticipated it would be difficult to track down errors in my code, since I wouldn't know whether values/state at any point in time were already wrong on not.
Luckily, though, I stumbled on a solution. Using the Visual Studio "Attach to Process" allowed me to attach to the game executable without it crashing, unlike IDA. At least, for the majority of the time. Though switching from the game to the debugger and back again caused severe corruption of the full-screen display. Luckily, though, there had been some previous discussions about running the game in a window - I was able to use a utility called DXWnd that intercepted the game's DirectX calls and forced it to run a window. The result wasn't perfect, in my opinion, for anyone wanting to play through the game, but it's worked well enough for my purposes, in conjunction with Visual Studio.
As a result, I'm now making much better progress than I had anticipated, and hunting down bugs is in general much easier than I'd anticipated. Let's hope that stays the case.. my next major gameplay milestone is to complete more extensive conversation with the Deskbot to get myself a room. The basic yes/no detection for the Doorbot worked pretty smoothly first time I tried it. The Deskbot, though, is using more of the conversation parser - I've already located and fixed some problems with it. Let's hope that there won't end up being too many.
On a final note, the one downside of my surging progress with Titanic is that I'm currently spending less time working on finishing my Xeen engine. I'd originally anticipated the frequent roadblocks trying to hunt down bugs in Titanic would have me growling in frustration, and switching to Xeen for awhile to unwind a bit. Now with the ability to debug the original executable, that hasn't really happened so far, and hopefully won't happen. I'll probably end up spending more time right now focused solely on Titanic, and see if I can't get the bulk of the game with the exception of the final starmap working by the end of the year. Then I'll be in a better position to alternate between working on Xeen and trying to disassemble the remainder of the Starmap classes.
Our one month testing period is over. Thanks to a lot of help from you, our fellow
users and testers, we are now ready to ship our next release.
As always, we have been busy adding more games and enhancing old ones. Now,
after many years in production, we are proud to announce
support for Myst and Myst: Masterpiece Edition.
Besides that, we now support U.F.O.s and the early Sierra
Hi-Res adventures, namely Hi-Res Adventure #0: Mission Asteroid,
Hi-Res Adventure #1: Mystery House and Hi-Res Adventure #2:
The Wizard and the Princess, for a total of six newly supported games.
With this release we are switching to SDL 2, so please,
report any bugs you encounter. Last time
we introduced Sparkle-based updates for Mac, and now we're trying to launch it
also for Windows. Hopefully, you will see it in action once we release our
next version. Once again, we would like to mention that auto-updating is selected as an
opt-in during your first start up. By default, if you enable it, it will check
weekly for new releases. Screenshots of this in action can be seen at
the WinSparkle website.
Notable bugfixes and enhancements include Hercules rendering and font
for the AGI games, numerous bugfixes to SCI games, enhanced Sherlock
games experience, and AmiUpdate on the Amiga port.
"You are in the front yard of a large, abandoned Victorian house."
You can now experience adventure game history, and play Sierra's early
Hi-Res Adventure #1: Mystery House in ScummVM. Dare you leap back to
1980 and take the steps up to the porch? Grab the game and a
daily build, and join our testing by stepping back in time.
We're proud to announce that we now also have support for several other
games using the same ADL system — which feature advanced technologies
such as color graphics! Perhaps you'd rather save the planet (Hi-Res
Adventure #0: Mission Asteroid), or save the princess (
Hi-Res Adventure #2: The Wizard and the Princess)?
We're working on the rest of the Hi-Res Adventure games, but in the
meantime, we're releasing support for the Apple II versions of these
three games for testing, and look forward to seeing any bug reports for
problems you might find. We'd like to include these in our next release,
so be sure to read through our
and don't forget to take some
Mystery House was released into the public
domain by Sierra On-Line in 1987, to celebrate their 7th anniversary.
You can download it for free. For more details about the
other games and where you might be able to buy a copy, take a peek
at our wiki pages.
If you happen to have original floppies for any of the Hi-Res Adventure
games (for any platform), we want to hear from you. In particular, if
you have an Apple II version of Hi-Res Adventure #3: Cranston Manor,
please get in touch. Many of these games are rare, and we cannot add
support for all the different releases without your help!
And so it has come to this, all things must end. But it is nothing to be sad about for me, this has been a great addition to my experiences and I welcome the change for it has been something short of 3 and a half months that I've been working on DM. Not that I won't be working on it from now on, but I'm definitely taking a few days off, lest I come to dislike it for looking at the codebase for too long.
The pull request to merge the engine is due next week as it still doesn't compile with GCC (strangerke has been working on it relentlessly (think 127kbs of error log reduced to 9)). Relatedly, the code is not particularly compliant with the coding conventions at ScummVM (strangerke is working on it, I'll soon start feeling ashamed and will have no choice but to help him).
As for the future of this blog, it is possible that I will post updates for when something major gets incorporated into the engine (think support for other versions).
Almost forgot about the new stuff, if I remember correctly it's convenient loading/saving from the launcher and from the inventory. And also there are debugger commands like godmode, noclip, set pos/map and the aptly named, gimme, which spawns items. Entering the commands without arguments will output their usage, call 'help' for a list of them.
Gimme can be used like this: call 'listItems', if you are looking for something enter any part of it in caps like this 'listItems OF FEAR'. Once you found what you are looking for, call gimme with its name: 'gimme HORN OF FEAR'.
Setting the map is slightly broken. For best results, teleport to an adjacent map, then use pos to set your position next to some stairs and off you go. Avoid using noclip.
I am sorry for not writing in a while, but as the finish date for GSoC comes closer, everything has to be ready for the end. My mentor and I agreed that the PR sould be submitted around August 14th, so these last days my intention was to have as much functionality as possible in before the PR came, as I was sure that after that I would have to spend all the time I had fixing things that I either forgot or didn’t know were broken in the first place.
However, I didn’t get to implement much, since a pesky error on my part had me using valgrind for three days, looking for the source of a heap corruption. Luckily I found it and fixed it right in time, and the engine was in at least a playable (albeit not complete) when the PR happened.
On the morning of the 15th I submitted the PR, and by now I have received tons of excellent feedback, and have compiled a list of fixes that need to happen. Thanks to everybody for pointing these things out that fast, so I can get to work right away :)
On a side note, I am planning on writing a postmortem once the GSoC is done, with some important lessons I learned during this experience, which will hopefuly be of some use to future me and anybody that comes after.
For the rest of GSoC I’ll be working on the PR, which you can check out here. Please feel free to pull and test it, I’d be very thankful for any feedback to make the project as good as it can be.
What I was working on during GSoC is Cloud storages support in ScummVM. Describing this feature in my proposal (mirror), I mentioned that it would include an API to interact with supported storage providers (which are Dropbox, OneDrive, Google Drive and Box), saves syncing mechanism, functionality to upload and download games data, and, of course, GUI for all of these. Proposal also has some extra tasks mostly making user experience better.
Some things were rediscussed during my work, but the main idea remained intact. The work is done and pull request already awaits final review before getting merged. API for all four announced storage providers works fine, saves are syncing and games data could be easily downloaded. Not only described extra tasks were complete, but also some functionality not mentioned in the proposal was added.
Saves sync is probably the main reason why Cloud storages support is needed at all. It allows users to easily continue playing the game on another device by simply connecting both to the same storage and doing the sync. It’s automatically started on ScummVM launch, on games saving (including autosaves) and when user opens Save/Load dialog. This dialog was updated to show a progress bar while syncing and also to «lock» slots which are being synced. To indicate that there is a sync in progress, small Cloud icon is shown in the corner.
To use the feature, users must connect a storage first. To do so, they should navigate into Cloud tab of Options dialog, select a storage provider and press «Connect» button. It opens a special Storage Connection Wizard, which provides the instructions on connecting. It has different variations depending on set of libraries ScummVM was compiled with. In the most simple case it says users should navigate to a special short link (to scummvm.org), which redirects them to provider’s page. When they allow ScummVM to use their storage on that page, they are redirected back to scummvm.org, where the code is shown. This code should be typed in the wizard dialog. It’s used by ScummVM to connect to the storage and use provider’s REST API then.
ScummVM page makes the code that way so wizard could check that code has no mistakes in it. If there is a mistake, it notifies user where it probably is. If ScummVM was built with SDL2, pasting from clipboard is supported. Wizard also has «Open URL» button, which makes it easier to navigate to provider’s page on platforms where URL opening was implemtented (these are Windows, Linux, Mac OS X, Android — iOS and Symbian are coming).
But it’s much easier to connect a storage when ScummVM’s built with SDL_net support, because then ScummVM runs a local webserver. In this case users are not redirected to ScummVM site from provider’s page — instead, they navigate directly to webserver’s page. No code typing is needed then, because ScummVM automatically gets it from user browser’s HTTP request. This webserver makes connecting a storage really fast and simple.
Another thing Cloud storage might be used for is games data download. Users can put their games into storage and then easily download on all their devices. A special «Download» button in Cloud tab opens Download Dialog, where users can select a remote directory to download and a local directory to download into. It shows a progress bar there and automatically tries to detect a game when download is complete. Users are also free to run download in background: no detection will happen, but a message will appear on the screen to notify them of finished download.
Both storage connecting and game downloading are shown in a video I’ve recorded. I’ve also posted information about my progress in the blog every week. Feature is documented on the wiki pages, with some diagrams included.
And, finally, we’ve decided that I should do a big extra task. Local webserver, which I originally proposed to simplify storage connecting process only, has been extended to be used for «Wi-Fi Sharing» feature. It means that while ScummVM’s local webserver is running, one can use browser on another device to navigate through directories, download files, create new directories or upload files!
Users can specify server’s port and their ’/root/’ directory within the same Cloud tab. Only files under specified directory and ScummVM’s saves directory are available, so users secure data is safe.
Yay! The color palette is FIXED once and for all! No more radioactive green snot on every object and champion and door and what not! The GUI looks fabulous with it's newfound colours, it's just so much more pleasant to click on it now. Now, the original game uses copper to stretch those few colours the Amiga platform could offer at any given moment and given that ScummVM has no support for something like that (and why would it?) I shied away from even coming close to trying to fix it. Now as the end is nigh, it had to be fixed, and so it was! Phase one, double the palette and offset the pixel colours. Phase two ???. Phase three, profit! Relatedly the engine is almost complete. What's missing is correcting some display functions, hunting down a few annoying bugs and making sure the game can be finished and then all will be well. The later, so I gather, is not yet achievable. The fluxcages spawned to trap Mr Chaos cannot be seen, because the function drawing them is missing, and that particular function happens to be in assembly (... ehh) and so I can't really test if anything is happening at all, gonna have to make it draw some dummy image. As always, here's a nice GIF, basking in it glory:
Cursor got fixed, seems like googling Amiga hardware sprites is all it needed. Also found the bug with the spell symbols, a petty toUpper call in the text drawing method, dunno what is was doing there in the first place. Long story short, dungeon looks a lot nicer now, palette needs a fix and then it will truly look splendid (sans some scrabbled textures). Here, have nice GIFshot:
This week I’ve worked on some security-related updates for LocalWebserver. It now has a «minimal mode» — no Files Manager features are available, only redirect_uri. It’s used automatically in Storage Connection Wizard, and server goes offline the moment it gets the code.
Paths are now dealt with more strictly: «../» is forbidden to use, ScummVM has some «blacklisted» directories and user can define where the «/root/» is. Files outside that folder are not available through Files Manager. Plus, if no rootpath specified, «/root/» is not even listed — only «/saves/» is available then. GUI for changing rootpath is added to the same Cloud tab.
One other thing I did was rebasing. Travis checks PRs by merging with master and then building. Titanic, which wasn’t there when I started, was using OutSaveFile as if it’s typedefed WriteStream. But in my Cloud PR OutSaveFile is a real class (which starts saves sync when finalized). So, I had to rebase and add a simple fix in Titanic, and now Travis checks are passed again.
In order to update WAGE saves, I had to add pos() into WriteStream class. Managed to break a few builds that way.
Also, fixed a few crashes in WAGE games. One was because code tried to copy pixels outside of the Surface, and I added checks, so copied rectangle is clipped to always be within surface area. The other was because operator was used on an empty String, and now that code uses it only when String it not empty.
The title says it all. There are now sounds in the game, it is the first thing that worked on the first try, or so I hope. One can now indulge in the angry grunts the champions make when you bump them into the walls. Makes one wonder if they want to be there in the first place. Why would they follow an order to go headfirst into the wall? Also, blitShrink function has been dusted and adjusted and thus the dungeon looks a lot nicer. It seems that after hours of searching for broken code, I realized that nothing is broken, but the original blitShrink method works a bit differently than what I tried to simulate it with. So it seems I can't avoid the assembly after all.
kacIt has been too long, and I apologize for that, but like a Protoss player taking a base, many things got ready at once in a short short time (I just bought Starcraft II). So, without further ado, here are the occurrences and features of this past week.
Please ignore the graphical noise in the following screenshots, since it’s due to an issue with my display, not the rendering.
The MacGUI PR got merged
The very first thing I did on GSoC was to take the GUI system from WAGE and extract it to graphics, so that I (and hopefully some other devs) could use it for our engines. Over the course of the development I added some things to it, the most important being the support for custom bitmap-based 9patch borders (using _sev’s code).
Just today, the PR issued to merge this extraction to graphics has been merged, so hopefully somebody will find it useful. I know I certainly did.
The save/load system
I had the saving and loading functionality implemented already (luckily in MacVenture it’s pretty straightforward), but it was a pile of hacks that only worked in one savefile and just for Shadowgate. After some work, and much inspiration from WAGE and Drascula, I added support for ScummVM’s saving dialogs and metadata, which is included at the end of the file so as to try and make the savefiles compatible. However, I have no way to test this, since I do not own a Macintosh.
You can see the results of the new save system here:
A glitch in the matrix…
I added detection for Dèjá Vu and Dèjá Vu II, which would be fairly straightforward were not for the fact that all the Dèjá Vu game files had special characters in their names (D?j? Game, I’m looking at you), so I just couldn’t open them. I have a couple of proper solutions in mind, and I have to look at how Dráscula deals with Spanish tildes, but for now it rests as a dirty hack that takes the problem characters (è and á) and just transforms them to ascii.
Here is a screenshot of Dèjá Vu…
And here is a screenshot of Dèjá Vu II: Lost in Las Vegas. Note the recurring theme of the coat.
That is one unfortunate thumbnail, but it is actually the first sound I could find in Dèjá Vu. I implemented the sound system, which has about half a dozen formats, so that now we have sound. I haven’t tested all the formats, only the ones checked here.
Lastly, I implemented the system to read the border files from a file called macventure.zip. Eventually, I’d like to have this file bundled with ScummVM, like Dráscula or Wintermute, but for now just downloading it from here and placing it in a folder findable from ScummVM (the game folder or the extra path) should suffice.
In addition, if you have the games and would like to play, I wrote a short guide here.
As always, I try to keep my Trello updated, and I actually sometimes stream development too, on Twitch.
Well, this week I was mostly working on some fixes for Cloud PR, some minor warnings, TODOs. Updated GUI a little bit: added a «Paste» button in Storage Connection Dialog, so user’s clipboard could be automatically put in the 8 code fields. Dialog also contains a Container now, because buttons couldn’t fit in 320×200. Storage providers logos are now shown, which is just a nice thing to have.
Apart from that, I’m waiting for PR feedback, and thus have to do something while waiting. I’ve joined the ScummVM team and now working on WAGE engine. Well, the only thing I did for now was «porting» save/load code from existing Java implementation of WAGE engine into our ScummVM one.
It was a lovely surprise today to notice that writer Rebecca McCarthy mentioned my enhanced music for classic point and click adventure game Beneath A Steel Sky, which I spotted on the Giant Bomb forum under part 3 of her Adventure Afterlife series.
One of my all time favorite games, if cyberpunk or science fiction interest you I encourage you to play the game. It is available legally for free from SCUMVM or GoG as is James Woodcock’s ( Fan made) enhanced soundtrack, which I recommend you use. Mobile versions are also available if you want to support the dev. Rebecca McCarthy – Giant Bomb
I mentioned it in Adventure Afterlife but the Beneath a Steel Sky remix by James Woodcock is awesome https://t.co/reNMz79fXG
(I don’t have any excuse, I just forgot to post that in time.)
This week I’ve sent my Cloud storage support as pull request to the main ScummVM Github repository, and thus started fixing everything what is not fixed enough, adding new minor features, etc.
Only pasting from clipboard support was added for EditableWidgets (I’d have to fix that too, btw).
I’ve added two new test suites in the Testbed: Cloud and Webserver. I’ve also added one test for openUrl in MiscTests. Almost all my tests are interactive, meaning they require user to check whether the feature works as it should, and press the right button.
Peter Bozsó (uruk-hai), my mentor, added OS X backend for openUrl. iOS backend it not there yet.
Another small feature which lacks iOS backend is the Networking::Connection::isLimited() function, which is used in DownloadDialog to notify users to think whether they want to download game data while using limited connection. For now only Android supports this, other platforms are using default backend, which always returns that connection is not limited.
AJAX version of local webserver’s Files Manager was added this week. One can navigate through directories without page refreshing all the time. This version also contains «breadcrumbs» navigation feature, so one can get up more than one level with one click at the path.
Users can override webserver’s port through the Options dialog now. Yet, if they want to use webserver for auth, they have to use the default value.
Some refactoring/cleanup in the code and different minor fixes were made. More OSD messages added to notify user of success or failure. I’ve also updated Dropbox to use their API v2 everywhere, as v1 is deprecated now.
I’ve added detailed information about the cloud storage support and local webserver to the ScummVM wiki. These should help people understand PR’s code. Some ideas are coming out in the discussion. For example, webserver is being stopped now when users close the Options dialog to prevent it being used by someone else.
At last, the game of Shadowgate is winnable in the ScummVM platform!
There is a screenshot of the final battle with the wizard at the end (spoiler warning).
It took somewhat longer than expected, and it’s in no way as complete as I would like it to be (just look how the interface is broken), but still I’m very proud of that.
To play the game, you can clone and compile my github fork, get the game files, and unpack them using some HSV extracting tool. If anyone is interested, I have a bunch of savefiles to various rooms (more on that later).
Now, what’s next? After talking with my mentor, we have settled in the following order of prioirities to spend the rest of the time:
Fixing the GUI > Adding support for other Macventure games > Add sound > Add Apple II graphics.
I will probably be dancing around the first two, since they are not mutually exclusive and when I get frustrated with one I can work on the other.
Other than that, I try to keep my Trello updated, and the spirits up. Again, I am sorry for falling so behind on schedule, I am suffering the consequences of not doing things properly on the first try, during the June burndown inferno.
Hi all. Been a while since my last posting; and a hectic last couple of weeks. After flying down to Florida for a 4th July long weekend getaway, I was barely back before I had to get ready for a work trip to the west coast. Then I get back to two days of company photo-shoots, which kind of disrupted getting back to normal work; but no sooner than they were done, I come down with a cold and had to stay home for several days. And was too sick to even really use my computer until now. :(
Nevertheless, despite all of that, work has been progressing on Starship Titanic since my last posting, and I've made a lot of progress. Firstly, there's the PET User Interface. This was one of the major areas that still had to be investigated last time. Since then, I've spent a lot of time figuring out how everything works, and re-implementing it all in my ScummVM engine. Behold the fruits of my labor:
Code for all the various areas has been pretty much all implemented. You can enter text in the conversation tab, and click stuff in all the others. There's just a few minor visual and functionality glitches here and there, such as the dials in the above image. Some of the tabs, like the Remote and Rooms tabs, are mostly untested at the moment.. it will be easier to test when I have more of the game logic implemented, and can properly test them with live data. Saving and Loading is also waiting on me to finish the saving and loading code.
As before, I was aided somewhat in implementing the PET by how clean the class hierarchy was designed, and the fact there were various internal error messages scattered about the code to give an indication of the classes' original names and purposes. For example, the code has a single common class CPetGlyph for the square glyphs used in all the different PET tabs, and CPetGlyphs for the collection of glyphs, with all the logic for scrolling and rendering. Individual PET areas simply derived from that, and instantiated whatever glyphs were needed, be they action icons like for Save, Load, and Quit, or inventory items that can be selected.
Once the PET was pretty much done, I started looking into the conversation handling and text parsing. To begin with, I worked one of the easier, self contained areas of vocabulary loading. All the vocab data for Starship Titanic is held in a resource inside the game executable called STVOCAB.TXT. It holds the data for an impressive 4700 words! A lot of these come down to synonyms for common words, and in fact even includes foreign language words, like "ein" as a synonym for "a". Each word entry in the file also contains data for what kind of word it is (such as action, adjective, etc.), as well extra information that is used by the parser to uniquely identify the words. The vocabulary loader builds up the data for each word in turn, building a master list of overall words support by the game, and a linked list of synonyms for each word.
Once I did that, I started looking into the input processing code that gets called when a line of text is entered in the PET. First of all, the game calls some pre-parse methods. These take care of a bunch of standard processing of the line, including:
lowercasing everything and removing extra spaces between words
Conversion of numbers from textual formats into decimal. That was surprising.. the game has special code in place so you can enter numbers like '42', 'forty two', or even 'LXXIII'. That's right.. the parser is so sophisticated, it even handles roman numerals. :)
Expanding common contractions like "it's" to "it is".
Once preprocessing is done, it then breaks down the entered sentence into a series of words. Each word is checked against the word synonym lists to identify the known word for each entered word. Here again, the parser has extra complexities that surprised me. It can not only find word matches based on exact word matches, it has two fairly complicated routines that can identify various forms of pluralization, as well as common word prefixes like 'super', 'anti', 'counter', etc. and find matches on the word without the prefix. At the end of this process, the entire input has been broken done into a linked chain of recognized words, and the real handling of figuring out the player's input can be done using custom logic for each NPC script.
At this point, whilst I had made some inroads on NPC script handling, I somewhat started drifting on what area I worked on. Strangerke had been looking at some of the core game scripts to add me in writing game logic, so to make things simpler in the long-run I went back to the main CGameObject base class, and spent some time figuring out all the remaining unknown methods. Quite a few of the methods were basically stubs calling PET Control methods, so my prior work on the PET was a great help.
Next, there was a bunch of CGameObject methods calling movie code, so I decided it was finally time to figure out what it was doing. I was able to properly implement the game's OSMovie class which handles movies, and it's AVISurface which handles the low level AVI files. It was actually interesting in that some videos have a second video track - I haven't 100% disassembled the movie drawing code, but my impression is that pixels in one are transparent, and in the other are not. This required me to add an extension to our AVIDecoder to allow a callback method for selecting which streams the AVIDecoder would use. That way, I could set up two decoders in the AVISurface class, one with the audio and first video track, and the second with the second video track, if present. This neatly avoids the limitations of one video track per decoder. The replacement movie code is all implemented now, but apart from cursor loading working (the cursors are stored frame by frame as a video), movie playback is still crashing, and will have to be debugged further. But at least I know what the bulk of the movie methods are now.
Finally, I moved onto the Star Control class. This is what implements the game's starmap control.. with PET mostly implemented, NPC scripts partially implemented, and movie handling more or less done, only it and sound are left unlooked at. And as I've mentioned previously, sound handling has a lot of spatial processing for sounds that I may simply not implement - the AVIDecoder already handles video sound, and I'm hoping to hook up the other sound methods to simply use standard ScummVM sound decoders. So that makes the star control the only remaining large area to look into.
Oh boy, and what a large area it is. Even basic identification of classes was already bumping up to nearly 30 classes all specific to just the star control. I felt greater and greater dismay the further I looked into it. Honestly, it was like Hopkins FBI all over again, where they included a full Wolfeinstein 3D style minigame in the PC release. In that case we were lucky, and could simply ignore it. Not so much in this case - all the classes will have to be implemented.
As I've looked into it further, and worked on classes that don't depend on others (so are self contained, and in theory easier to figure out), some of them have started to make sense. There's matrix and vector classes.. somewhat surprisingly, two versions of each, one for floating point numbers and one for doubles. I can only assume back in the day there was space vs speed considerations for having both types; I've not yet reversed all the methods, so I don't know if they can be merged in the future. I've also identified two "star points" classes that use large lists of spatial data that was hard-coded in the executable. Hopefully, the other remaining classes won't be too hard to figure out likewise.
So, all in all, things are going good. I'm starting to feel better, and I'm making inroads on the last major area of the game. Once I get that out of the way, it'll be a matter of finishing off the NPC scripts, implementing miscellaneous methods here and there such as sound handling, and then implement the game logic. Which, with all the core engine implemented, will hopefully be a straightforward, if slow process.
This last week has been, for the most part, incredibly frustrating. The inventory system had so many little bugs in it that it seemed that when I fixed something (often after most of the day looking), something else broke. However, somehow I managed to get it working, and thus, with the skeleton key in hand, I can go on to the fun part of the project: QA! In all seriousness, in the upcoming week I will be focusing on just following a walkthrough of the game to detect and solve the little bugs in the way of winnability. I had a milestone in my planning to have Shadowgate winnable for today, with which I meant a much more polished state that it currently is in. However, several factors and my frustration (no-one but me to blame here) have made it impossible to acomplish.
Well, actually, I wouldn’t know currently whether Shadowgate is winnable or not, because there is a little bug that prevents me from lighting more torches, and that makes it near-impossible to advance more than a few rooms. However, I am really please to see that at least there is nothing major preventing me from playing the game, and that the systems I put in place (read: “copied”) are more or less working.
So, without further ado, here are some screenshots of the first rooms of the game:
Again, I apologize for the short clips, but it is a combination of my switching to Arch Linux (and the limitations of the software I use to screencast) and my general slowness while playing.
I am hopeful that, by the end, I will have a product that I can be proud of, and hopefully somebody can enjoy (I for one am having a blast playing through this game).
My exams were over, I now have to turn my attention back to Director engine. I continued to work 5 days ago. And next things have emerged:
Starting from Director 4, RIFX containers can have little-endian byte order, specifically for this, regular ReadStream was amended to ReadStreamEndian and now engine can work with both orders. In so doing, byte order is determined by first tag (“RIFX” or “XFIR”)
In the majority of the movies, casts not stored directly in the movie. There is a special file, with a special name (surprising!) that stores, common to all movies in folder, sounds, images, texts. For this situation, renew shared casts immediately after jumping to new folder was written. Shared may be some regular buttons for game.
For example: (Spaceship Warlock in our engine)
But determine shared cast movie name still a problem.
Significant progress has been made by sev : storing local/global variables, Lingo macros, built-in-functions and more… On my part, were introduced execution go to loop, go to next/previous commands.
Reading text resource realized a long time ago, and it time to deal with rendering issues. Director font map has been reviewed by sev. And it was decided to use classic mac fonts data file for drawing our text. Future target: rendering with box shadow, borders (if needed)
As it happens, I needs to complete some work related to my student status. It’s about my exams. In this regard, in the coming week my time should be given to questions about tropical geometry and information models/processes.
At the end of week, I will continue to work on Director engine.
And once again I’m writing about my GSOC progress.
Loading transition info and implementation of some movings (8/52).
Imitation 2 track sound system (such as in Director).
Rendering trail sprites (it works like background images)
Parsing new kind of scripts – movie scripts.
Speaking of scripts, there is a work on the Lingo compiler by sev. There is, at present execution some media interface commands like play part of sound files and the processing of alias map. Also put/set commands have been added.
Were reviewed, Director ‘go to movie (label)‘ commands. In many games this is main command, with this you can simulate walking across levels. But this transition using special name of movie, and because of that movie loading was redesigned.
I know I promised to show The Journeyman Project movie files. But right palette still got trouble.
Specifically for this post, I’ve copied in my local code, palette, which contuinity uses as fallback. And some of JMP movies seems more or less pretty.
A week went by, engine changes, but slowly. Now is the moment when differences in Director versions are important.
Under the supervision of sev, I started to learn life cycle of the playback/draw system, it’s important because it can tell about when causes local events, such as frameEnter, frameExit, idle. In other words, we want to achieve event order, same as in Director. For me personally, it was hard to figure out what order is correct, sev advised me to check it right in Director. Because our target game is Journeyman Project, I looked in 4.0 version. By writing a simple script, I have found right positions in my code for calling this events.
It is worth noting, that in other versions the order differs, part of the problem is new events (D6: stepFrame, prepareFrame, beginSprite …).
Maybe you have already guessed that, this events are necessary for our feature Lingo interpreter. In addition to the system events, I’ve also added mouse events, this event are sent to clicked sprite on top layer.
Simple sound wrapper had been written, in JMP we’ll see .wav and .aiff formats. ScummVM audio decoders deal with this.
Decoder for 1bpp pictures was also added. This pictures are sometimes used instead text.
There is now some issues with 256 color palette loading. I hope problems will be solved, and in the next report I can show movies from The Journeyman Project.
Although I have never officially announced my work for classic point and click adventure game The Gene Machine, I have already finished the soundtrack… Just one problem, before I release it in full, I need permission. I have always sought permission to release my enhanced soundtracks, however tracking down those who have the rights to allow me to do just that has proven… well… problematic!
To whet your whistle however, I have decided to release a single track as a preview in the hope that one day, the powers that be will contact either myself or the ScummVM team allowing this to happen.
Until then, enjoy this little taster as we all keep our fingers crossed.