Planet ScummVM

Since it's been a month since my last posting, I thought it was time to post a status update. The work on disassembly of the MADS (Rex Nebular) engine has been going well.. I've made a lot of progress on various areas of the game logic.

I haven't actually added much code to the ScummVM engine yet, since I decided it would be better to fully understand a given area before I implement it. At the moment, my reverse engineering has been some undirected - it's obviously more effective to work on whatever areas you can find that are easiest to disassemble, which in turn may make other areas easy to disassemble later on. However, it does mean that there are bits and pieces across the entire engine that are now understood better.

One area of the game engine that's going to prove interesting is all the hard-coded game logic - each scene has several routines associated with it for scene loading, scene animations, and handling interaction with elements. Obviously it would be preferable not to completely reimplement these in code, since it would take a lot of code space, and there are two other games whose code would also need to be reimplemented.

What will probably need to be done is to create some kind of basic script interpreter engine, and convert the various scene routines to an interpretable script. Luckily, most of the routines seem to use a fairly simple subset of x86 instructions, so it shouldn't prove an impossible task. In any case, that's something for the future, when I fully understand all the various routines the methods call. For now, I've set a short term goal of manually implementing the logic for the first game room, and will return to the idea of scripts at a later date.

On a final note, for those of you who like pictures, see below for an example of the scene walkable areas for the fist scene of Rex Nebular. It also has a slightly weird sprite frame from Rex inserted into the picture - Rex isn't movable yet, though.. it's just a test from some of the loaded sprites:

It's time once again to go out and test all your favorite ScummVM games as we gear up for another release of our humble little software. We have several new games announced as supported:

• Blue's Art Time Activities
• Blue's Reading Time Activities
• Dragon History
• Freddi Fish 5: The Case of the Creature of Coral Cove
• Pajama Sam: Games to Play on Any Day
• Spy Fox 3: Operation Ozone
• TeenAgent

So, go grab your favorite games, and play through them using a daily build. If you find any bugs, report them on the bug tracker. Once you've completed a game, please report it on the forums so it can be added to the release testing wiki page. When reporting, make sure you provide the version, language, and platform of the game you're testing. Any other questions? Go see the Release Testing guidelines.

As you may or may not know, Uru Live opened to the public again on February 11 for the very high price of $0. It felt weird playing Uru again (not that I played Uru Live in the first place, except for a little bit :P). The cleft, and the main for ages of Uru were pretty much the same as the original. A lot came back to me as I played through them.

However, the Path of the Shell ages (Er'cana and Ahnonay for those keeping score) were a bit different. I ran through Ahnonay the same way as I did in the original... only to find myself not being able to rotate pods with the handle in the "hole in the wall." Turns out, they changed the way it's played to require two people...

A fellow ScummVM dev and myself set forth to help each other through the Path of the Shell here. We beat Ahnonay together and then helped each other yet again to find the way to K'veer Island (the island that Atrus is trapped in in Myst/Riven) and by extension, Myst. Here's Scott's and my avatars in Myst:


For what it's worth, my KI# is 590130 and my nick is, of course, clone2727.

Also, for those still keeping tabs on Riven progress, you can now go free Catherine (not that it's possible to get to the 233rd age yet anyway). Interested in trying? Use these console commands to plant you in Gehn's bedroom.

var agehn 4
changeStack ospit 81

Of course, you still have to make your way to Prison Island afterwards ;)

Today I took out the Snow Leopard upgrade DVD that I've had for several months and installed it. I figured it was time for an upgrade.

Positives, positives... It didn't crash when installing... It probably is faster in some way. Ok, now for the negatives:

First thing I noticed: Had to recompile all the libraries. There was libSDL and libMAD present, but broken; or at least compiled for the wrong processor. Not to mention libFLAC and libMAD didn't recognize the host, so I had to force it to "x86_64-darwin". Other than that, the 4.2.1 gcc seems to be an improvement over 4.0.1 from the XCode that comes with 10.5.

Second thing: The definition of KB/MB/GB. I hate the 1000 bytes = 1 KB definition and the 1024 bytes = 1 KiB definition. I'm quite disappointed 10.6 now uses this definition.

And finally, the biggest problem: QuickTime X. To put it plainly, QuickTime X is complete crap. I can't believe Apple would release something like this. It is total trash. As when testing any video in a video player, I like to start with Myst's (the original!) Cyan logo video. First off, QuickTime X has almost no codec support; it has to call upon QuickTime 7 to play the videos. This already took about 10 seconds to load the video. Next problem: The video played at some ridiculously high resolution and wouldn't let me play it at normal size. In addition, the whole interface was ugly.

So, I'm looking around, and alas! QuickTime 7 has been pushed back to the Utilities folder. Following some other guys' advice in this post (of people who agree with me), I switched all QuickTime related files to be played with QuickTime 7. That's much better...

On a completely unrelated note: Background videos in Riven work now! Go watch the awesomeness that is the Tay linking book videos! So epic!

Hi, it has been a long time since my last update.
TeenAgent has been implemented into the main trunk of ScummVM, however it is not based upon my engine, instead Vladimir approached the ScummVM team a few months back with a rather complete engine, the engine itself is very tidy and the game is in a completable state.
Congratulations are in order to Vladimir for his incredible work on the game, surpassing any progress I would have been able to make with my poor reversing and coding skills.
The project was great fun and an incredible learning experience while working on it. I would like to thank _sev, Buddha, salty_horse and jvprat for their patience with me while I was learning.
I would also like to thank john_doe for his commits for the music support, as TeenAgent used a strange and new module format for music. I have great respect for everyone who helped me learn. It will not be my last attempt.

I have recently been looking into the code used for Maniac Mansion sound effects on the Apple II.

A little history on the Apple II:
When Maniac Mansion was ported to the Apple II(around the same time as the C64 port) it was seen as a very ugly version, in comparison to the C64 version. It had some graphical issues and didnt include scrolling. Unfortunately the port also suffered in a worse way. While the C64 had the SID chip, an incredible sound chip allowing for some really interesting sound effects, the Apple II port had to make do with a REALLY basic speaker. The Apple II port used the speaker to generate some sound effects such as the character selection 'click'. Unfortunately the speaker could only output a square wave 'click' when the port was addressed(using a 'bit $C030 command) The speaker itself had not options for changing the output or frequency of the output.
This of course left games programmers with very little to play with, many of them came up with intelligent code that would call the speaker multiple times using various loops to 'click' their way into something vaguely audible. Maniac Mansion was no different. Lucasarts created a resource format which consisted of a collection of various bytes which told the code how it should 'click' the speaker.

The resource format is incredibly simple and the resource is simply read in by the code which judges how often and when it should click the speaker.

I am currently about half way through reversing the code(6502 ASM), I have converted about half of it to c++ and am planning on finishing the rest within the next few months(there isnt much, I just do not have much time with university commitments, as it is my final year any external work has to sit on the backburner until I get stuck with my university work :p)

I would like to thank Tobias for his patience with me, he has been especially helpful as he has not ever given me the answer, rather he provides ways for me to find the answer and makes sure I understand it. He has been so helpful in explaining some of the 6502 opcodes and explaining how they were used. I am greatful for his friendship and advice.

Though this code is not very complicated, I am still struggling to understand completely what it does. I am sure this will come to me.

EDIT:
I have documented all I know about the resource format on the ScummVM wiki here : http://wiki.scummvm.org/index.php/SCUMM/V0/Maniac_Mansion_Apple_II

Though this information is pretty irrelevant, since the code is what actually makes sense. The resource is simply a loop controller.

Finally, the support of Fascination is showing some interesting progress! These last two weeks, the window behavior has been fixed, the hotspots correctly handled in them, and it’s now even possible to play through some levels. DrMcCoy even added a save/load function that works perfectly! There are still a lot of glitches in the floppy versions (almost none in the CD version) and it’s not completable for various reasons, depending on the platform and language concerned.

Here are some screenshots of the CD version.

Stay tuned for further news! I promise I’ll try to give some before this summer

Every once in a while, I check on eBay for a few games that I either find interesting, might help ScummVM in some way, or because I'm trying to track one down. I was looking for Mac versions of Sierra games, and at one point I happened to scroll down to the bottom of the screen and something caught my eye. "Leisure Suit Larry" (along with "Lego Batman") was mentioned on the bottom of the screen:


Bewildered as to what that link could possibly mean, I clicked on it and it lead me to here. Apparently Leisure Suit Larry is a popular item on eBay. Or something. I don't really get it. Not to mention none of the products listed below the description are even related to LSL.

</random blog post>

EDIT (2/3/10): My continued eBay searching has left me somewhat confused. Sometimes the item is just ridiculous, sometimes the price is just ridiculous. Since I don't want to just create another post, here are a few oddities I have found:

- King's Quest I CD: In addition to the high price... "replaced with the CD version"
- King's Quest V FM-Towns: High price. I would love to have it to be able to support it though! (Will accept a donation :P)
- Indiana Jones FM-Towns... Phone Card: Self-explanatory
- LucasArts Japanese Mac Pack: I would love to have this too (not as much as KQ5 FM-Towns), but I don't want to spend this kind of money...
- Indiana Jones and the Last Crusade: Seems normal... until you see what it comes with.

And, the icing on the cake:

- King's Quest VI "Girl in the Tower" CD: My jaw dropped when I saw this one. Yours probably will too. The price pretty much tells all. But it has free shipping!!!!


In other news, I've been tinkering with the Mohawk QuickTime code and working on adapting it to the Graphics::VideoDecoder interface in ScummVM. The more I do this, the more it feels like shoehorning the features that QTPlayer has into VideoDecoder isn't working. In addition, md5 mentioned that IHNM Mac uses QuickTime MIDI files. Curious as to what that involved, I looked up some samples on the internet. They use the good bad ol' QuickTime format to hold its own MIDI format (I had hoped for SMF in a QuickTime wrapper). Here's what Apple's QuickTime specs said about it. Anyone out there have better info than that?

Those of you who have been following the IRC channels will know that I made some real progress in the disassembly of the tSage engine, used by the Ringworld - Revenge of the Patriarch game, as well as several others. I hadn't really gotten into any of the game logic, but I'd made great strides in decoding the image formats and graphics display, so much so that my engine on Scummvm-misc could display test dialogs, as well as the right-click game dialog with proper highlighting and everything. Sample screenshots below:





However, given the perversity of life, I should have known better than to announce on the channel that I was making real progress. :(. Because as I was getting into the game execution logic, I began to realise just how complicated they made the engine. Whilst using C++ has it's advantages from the perspective of RE, since it does group related functions close together with the variables they use, in this case, they seemed to go overboard with it.

A preliminary look at the main event loop of the game shows they implemented a stack of "action objects" - multiple independent event handlers can be registered and control swapped form one to the other. I'm not sure, but I presume that this is to allow for things like cutscenes, where standard game operation is superseded.

For even more complexity, they went with a class hierarchy for the action objects which is at least four levels deep!.
The classes, as far as I've been able to determine, are as follows:

• They have a core 'SavedObject' class, which has some internal logic for registering each object instance in a central list. I presume this is for automating saving and loading.
• A child class which I'll call "SmartPointer". This class seems to encapsulate a pointer to a secondary object, and provides core logic for 'fixing' up the pointer reference. I can't be sure yet, but there's a lot of 'fixup' code in the engine, which I presume handles restoring pointer references between objects when a savegame is loaded, and all the objects are re-constructed. This class also implement a basic virtual table of processing and dispatch methods that automatically defer to the foreign object if one is set
• Yet another child class which introduces basic event manager functionality amongst other things
• A concrete fourth child class which seems to further override some of the event manager functionality, and is what gets added to the game's "action list".
  

As you can imagine, all this makes it somewhat different to understand what the blazes is going on. I've also examined the engine from some other points, such as scene loading, and even that seems to be overly complex, with method calling method - they really went in the direction of generalised/generic functionality, with the result that the code seems to be much more complex than it should otherwise need to be.

</whinge>

In any case, the tSage engine was only meant to be a part-time RE effort for when I needed a break from the MADS engine. I did somewhat get carried away with it for the last few weeks, but that was only because the areas I was looking at proved so easy to reverse engineer. Now that I'm running into more complex code, I've decided that the break's over, and I'm returning to work on the MADS engine again. For that, at least, the code seems somewhat simpler, even if it can be a pain to run in the DOSBOX debugger because the segments keep getting shifted around in memory. ;)

For those of you who are Ringworld fans, don't worry, I'll still be working on the engine. It's just there won't be as rapid a progress on it as I'd recently hoped.

Today I am once again writing on something about SourceForge.net that bothers me. Just to get this straight first: I use SF.net and a lot, and in general it works fine for me; but in the past year or so, more and more things keep piling up that really annoy me about it. And so I feel the need to vent some of my frustration by telling the world (or at least the handful people reading this blog ) .

Now before you yell at me “Don’t blog about it, dude, report it to them!”: Of course I report all issues I find via their tracking system or their IdeaTorrent. All right and proper. It just is that it recently most of the time feels that all the reporting is for nothing… I report things, and they get ignored; or they get lots of attention and in the end are still closed with “Won’t fix, this is as intended” or “We currently decided not to fix this”…  To give another example, just today, I went to their IdeaTorrent, and noticed some duplicate ideas. IdeaTorrent allows you to report duplicates. Sweet! Until you realize that lots of people report duplicates, but they never get resolved… I am not sure whether to cry or laugh when I read this on the page where you report duplicates: ”The admins will accept or reject the merge shortly.” (Emphasis mine). “Shortly”, right…

Anyway, back to the subject: Some months ago, SF.net added a “project review system”. It allows anybody to give any project a thumbs up or down, and optionally a review. At first my thought was, neat idea, this could help filter out all the junk, and let the few real gems shine.

That is, until I realized that anybody can vote, even people not logged in. So you can quickly abuse it to generate many thumbs. Moreover there is nothing a project admin can do to delete inflamatory, off-topic or simple insulting “reviews”. Don’t get me wrong, of course negative comments must be allowed. If you try a product and it doesn’t work well for you, you should of course be allowed to give a negative review. But a constructive one, please! Take libpng for example: At the time I am writing this, they had 26 “thumbs up” and 21 “thumbs down”. With “reviews” like “YOU SUCK!!!!!!!!!!” or “that sucks man”. I kind of doubt that the reviewer, “anonymous” (who would have guessed…) is basing this on actual user experience. Well, or if he does, after this “review” I sadly don’t know what the experience actually was. To me it looks more as if it has beome a sport for various people to thumb down random projects. Man, I am glad that ScummVM (444 thumbs up, 92 down) and Fink (37 up, 6 down) are not hit as badly. Although I really wish I knew what those 92 thumbs down guys were complaining about. Maybe some of them even left reviews explaining their gripes, but so far, I haven’t found a way to read any reviews except for the last 25, so I’ll probably never know…

To sum it up, I just wish this project review system wasn’t so horribly broken and… naive? unprofessional? incomplete? useless?

There are certainly many ways to improve the situation, and luckily not all of them involve removing this feature. Instead of writing about those in detail, I’d like to point you to the relevant IdeaTorrent page, and ask you to vote on the various proposed solutions there. Yeah, I started this post with a complaint on how ideas on this IdeaTorrent may not be read at all… but I haven’t lost hope yet, and one should at least try, right? . So, let’s all try together, maybe we can move this unmovable rock a little bit.

P.S.: In case you are wondering which IdeaTorrent idea I found duplicates for, it was of course the one I linked to above… Seems quite some people are annoyed by these reviews and post new ideas suggesting improvements. If you wonder why they keep posting new proposals instead of voting on or adding to existing ones, well, my guess would be that it’s because the IdeaTorrent’s “Search” button, quite surprisingly, doesn’t do what it says. My request to rename it to “Filter by tags” (that’s what it actually does… not quite the same, nor quite as useful, don’t you agree?) was, of course, rejected long ago .

To coincide with the updated ScummVM logo, ScummVM t-shirts have also received a touch up. The shirts sport the verb list from Maniac Mansion on the chest, the updated logo on the upper back, and the scummvm.org url on the left sleeve. Shirts are now available in sizes small through 3XL. If you need a larger size, please contact sales[at]combobreaker[dot]com with a request.

Shipping to the USA and Canada is free, and shipping elsewhere is offered at a discounted rate. Head over to Combobreaker.com to pick them up!

Hey, here’s another small video, this time with Mike walking in-game (without pathfinding) and talking to Hank in the diner. Also a nice showcase for most of the remaining graphical glitches (highlighted in the annotations) :

And so, 2009 comes to a close. For those of you who have a good memory, you might remember a post I made last year about what I had hoped to accomplish this year. Let's take a look:

2009 Goals: (In order of probable completion)
- Fix Cinepak decoder!
- Fix QuickTime IMA ADPCM!
- Get background videos working!
- Figure out image format for other Mohawk games (Current hunch: LZSS similar to Myst format)!
- Finish Riven support (aka all the hardcoded puzzles :P)!

I have crossed out what was accomplished. Technically, I have a WIP patch that allows for background videos to work in Riven and background videos already work in Myst. So, we might consider that one done, even on a bit of a stretch. :P And now for next year:

2010 Goals: (In order of probable completion)
- Finish Riven support (external commands, mostly)
- Finish Myst support (some resource fixes, and then lots of the hardcoded opcodes)
- A look into other games more

Again setting lofty goals! We may even consider them monolithic in size! (Pun intended!) And for one last treat, to see the "good" ending of Riven, open the debug console in ScummVM (ctrl+d) and use these commands:

var ttelevalve 1
var ttelepin 1
var ttelecover 1
var pcage 2
changeStack tspit 137

Turn the lever down and press the button and voila! (Note: The DVD version may have to use a different card number, but just walk over to the telescope if you end up elsewhere :P)

Happy New Year! :)

Instead of continuing working, I’ll rather goof and play around with video capturing . So you can now gaze at some working Mike walking animations:

Just a minor update: Thanks to sev, who pointed me to the right place in the wine sources, I managed to solve the cursor inconvenience .

The problem was, of course, my own damn fault; I misunderstood the nature of cursor resources. They’re not, as I assumed, plain cursors, one by one each taking up a resource entry. Instead, there are cursor groups, each capable of possessing a variable amount of cursor images. That isn’t even a quirk of NE files, it’s like that in PE files as we well, so my complaining about the executables being in that old format doesn’t apply either. Mea culpa.

Initial reports claim the gold held in the MEGA-BANK vault just began to vanish! MEGA-BANK Security were asked to review the CCTV footage, however they were unable to see anything.

Do you think you can solve this mystery? Are you willing to take the role of Mark Hopper? A teenager hired by the RGB to infiltrate the mansion of John Noty and discover why and how he is becoming so rich.

That's right ladies and gentlemen, TeenAgent is now ready for testing! Why not take this opportunity to join Good Old Games where you can download TeenAgent for free. You will also need a Subversion build of ScummVM. As usual any bugs should be reported to our bug tracker. For further information, check out the bug submission guidelines.

The usual direction when adding support for T7G and 11H to ScummVM is to take the data files, decode them, and get the video and/or audio out of them. Last week in a moment of boredom, curiosity, and 'how hard can it be?', I decided to work on working on going the opposite direction and have a look at encoding instead.

As it turns out, still frames were fairly straight-forward to implement due to the simple structure.

Animation frames were a little harder. Most of the issues resolved to be related to how I was attempting to keep track of palette changes between frames (and the rewriting of the new frame to line up colours common between both palettes but at differing indices), but I also got caught up in some minor logic errors in the encoder (in particular, encoding the same tile using two different methods, and requiring a "new line" opcode at the end of *every* line - not just those that end early). The final result can be found at this YouTube video (sorry - no embedding, since my blog doesn't support it).

Encode speeds currently aren't the greatest yet, the output VDX files are not compressed, and the encoder requires 8bpp Indexed BMPs for use as input, but it's certainly a start for proof-of-concept purposes :)

The year is 1995. The place is the Czech Republic. You find yourself surrounded by inventive minds producing the latest Czech point-and-click adventure, and historically the first Czech CD-ROM-based adventure game. You are Bert, a young dragon with a taste for adventure. After being corrupted by an evil magic wand named Eveline, you are desperate to remedy the mischiefs you created. You decide to set out to find your father who disappeared a while ago.

Grab your copy of the Dragon History game and a Subversion build of ScummVM, and get playing! The game is available in Czech, English, and Polish so you have no excuses!

All bugs should be reported to our bug tracker. For further information, check out the bug submission guidelines.

The Scummvm-misc listing welcomes the new work in progress Dark Seed 2 engine by DrMcCoy. It looks very promising.

I've returned to spending a lot of my time working on the disassembly of Rex Nebular, and implementing it's functionality into the M4 engine. It's likely going to be an extended process, because the game engine is unbelievably complex. It likely comes from the MADS engine being designed as a generic game engine rather than just for single games specifically.

For example, I only just last night disassembled some of the code associated with displaying messages. Any given message is represented by a unique 'Message Id', which in turn loads a message resource that can contain an entire series of control elements for formatting dialogs. A sample message resource for looking at the curtains in the starting room is as follows:

[title25][sentence].Wincing at the "Drapes Of Wrath", you swear
to use them to light the fire when
you burn your ex-decorator at the stake

I'm dreading how complicated the actual script interpreter is going to be. :P

A new nice port is in the works:It is the very first screen shot. Hkz is working on it. ScummVM portability is demonstrated at its most.