Christmas 2023 holiday roundup

 Seems like it's been a full year since my last blog posting. I think that at this point, I'll just have to accept that I'm not very good at making regular postings. At least you all can take solace that my output of work on implementing support for new games in ScummVM remains unaffected. :)

So, what's been happening in the last year? Well, first of all, I finished implementing support for Might and Magic 1. The game itself was pretty primitive, but it was an excellent candidate for something I'd wanted to do for a while - not just reimplementing a game, but adding an enhanced mode that would breathe new life into it. In this case, I was able to use my experience with working on the Xeen games some years previously to add it's user interface into M&M1 as an option. I think it turned out pretty nicely:



After that, I decided to revisit another game that had been bit-rotting, in this case for nearly twenty years - the M4 engine, which supports the games Orion Burger and Ripley's Believe It Or Not - The Riddle of Master Lu. This engine had special meaning to me, as it was the first engine I worked on collaboratively with others, shortly after I first joined the group. Back then though, things got mixed up with trying to have the one engine support both those games as well as the earlier MADS games like Rex Nebular in a single engine, and work stalled on it. Since that time, I'd re-implemented the MADS engine separately (though currently not all MADS games are supported), so decided it was finally time to tackle the M4 engine. And, just in time for Christmas, I finally finished implementing it with Orion Burger fully playable.

I've just announced an official testing period for the game. All the logic of the game was hardcoded, and had to be manually implemented. As such, my implementation of the various game rooms' logic suffered from many minor errors that had to be identified and fixed. Hopefully this testing period will have others discover any other remaining issues I'd overlooked, and the game will finally be officially supported.

Other than that, on a final note, as with previous years I choose to muck around with something different over the Christmas holidays. In this case, rather than fall back on working on the early Legend games yet again, I decided on the Wasteland engine, in particular disassembling Fountain of Dreams. I'd done some previous examination, and it looked like the codebase was fairly straightforward and simple (at least for the small parts of the executables I investigated). My time was productive, and I was not only able to reverse engineer and reimplement the entirety of the character creation screens, but also get a basic map display going. I give you, Fountain of Dreams startup in ScummVM:

ScummVM Fountain of Dreams


As you'll see from the attached YouTube video, I even added basic mouse control support for the character creation screens, just for the heck of it. At the moment, what's shown is pretty much all there is, but it's a firm starting point that further work can be done on.

Will further work be done? At this point I'm not entirely sure. I haven't firmly committed to working on Fountain of Dreams and/or Wasteland. It may be that I'll just putter around with them between working on other games. After all, I still have the drudgery of implementing all the game scenes' logic for Riddle of Master Lu to look forward to :P.. maybe I can use working on this as a way to unwind as I work my way through it all.



Lets Do the Time Warp Again, Or Burger Chow Awaits

A poor test subject is doomed to repeat an hour over and over again. Trying desperately to prove his species worth as something better than merely meat to be ground into intergalactic fast food burgers.

The ScummVM Team is pleased to announce full support for Orion Burger, the classic game by Sanctuary Woods. The engine also adds a few niceties that the original didn't have, such as mouse wheel handling, and using the spacebar to skip walk animations - something the original didn't do properly.

Help us test the game by grabbing a daily build. Read through our testing guidelines. The game's logic was all hardcoded, so we really need people to try all sorts of weird actions, and make sure the game handles everything correctly and matches the original implementation. And please take some screenshots along the way.

Now, luck be with you. And remember, with the aliens cheating on the tests they set up for you, this will be no easy task.

ScummVM 2.8.0: Mysteries, Mammoths, and Muppets

We are proud to announce the last release of the year 2023. Please welcome ScummVM 2.8.0 – “Mysteries, Mammoths, and Muppets”.

New games

The team was quite busy working on new engines and enhancing existing ones. The list of supported games grew noticeably, and we now support these additions:

  • Adibou 1
  • Classical Cats
  • The Dark Eye
  • Dark Side
  • Escape From Hell
  • Gadget: Invention Travel and Adventure
  • Gobliiins 5
  • The Excavation of Hob's Barrow
  • Kingdom: The Far Reaches
  • Might and Magic Book One
  • Muppet Treasure Island
  • Nancy Drew: The Final Scene
  • Nancy Drew: Message in a Haunted Mansion
  • Nancy Drew: Secrets Can Kill
  • Nancy Drew: Stay Tuned for Danger
  • Nancy Drew: Treasure in the Royal Tower
  • Primordia
  • Reah: Face the Unknown
  • Schizm: Mysterious Journey
  • Shardlight
  • Strangeland
  • Syberia and Syberia II (macOS versions only)
  • Technobabylon
  • The Vampire Diaries
  • Whispers of a Machine
  • Wrath of the Gods and four other Director titles.
  • 14 AGS titles by Stranga and Cloak and Dagger

All together, we’ve introduced 50 new games and five new engines.

New platforms

We are happy to see the RetroArch port being properly rewritten, and the port is now part of our source code. The Atari port has also been redone from scratch and now talks natively to the hardware, skipping SDL as an intermediate layer. That made many more games playable on the platform. The Atari FireBee port is still using the SDL library, though.

Speed-optimized graphics

Thanks to the work of one of the GSoC participants this year, Wyatt Radkiewicz (a.k.a. eklipsed), we now use CPU-specific SIMD instructions such as SSE, AVX2, and NEON for drawing graphics in the AGS engine and in some generic routines. This led to 4-14x speedup in drawing for many cases.

Networking games

This year, we merged with the Backyard Sports Online project, which made it possible to play Backyard Football, Backyard Baseball 2001 and Backyard Football 2002 over the internet with other humans. Also, the Moonbase Commander support is in active playtesting mode, though not yet ready for prime time.

Notable engine enhancements

The AGS engine has been brought up to version 3.6.0.53 from upstream.

For many engines, we added support for numerous Chinese and Japanese game variants.

Believe it or not, we implemented a lot of native GUI dialogs for SCUMM games, bringing them closer to the original experience. We also rewrote the sound code for the SCUMM Humongous Entertainment games, making them flawless.

We performed a deep review of the Broken Sword 1 game engine, implementing some small, previously unnoticed things like scene transitions, in-game menu peculiarities, accurate fonts, idle animations, and more. Now, the game is absolutely faithful to the original.


The full Release Notes contain a comprehensive summary of the changes and enhancements; it is about 3 pages long with this release.

On our downloads page, you can find the downloads for various platforms. If you are using Windows, macOS, or either the Ubuntu Snap or Flatpack packages, the autoupdater will assist you in updating to ScummVM 2.8.0. Android port will follow up very shortly, as we are currently in the Open Beta phase.

Play the games, and enjoy ScummVM!

Ah, one more thing…

ScummVM is coming to the Apple App Store for iPhones and iPads. Please leave us with good reviews and report any problems on our Discord server, which includes an iOS support channel.

End of year Director roundup

The end of the year is almost upon us, along with a new release of ScummVM. So here's a quick update for one of the more ambitious sub-projects that has been percolating along in the background: Macromedia Director support! For those who aren't aware, Macromedia Director was a popular framework used for thousands of games, educational titles, catalogues, art pieces, and other multimedia works.

New Games For Testing

First off, the upcoming 2.8.0 release will have some new supported games, and we need your help testing them!

  • Alice: An Interactive Museum
  • Ape Odyssey
  • Classical Cats
  • Gadget: Invention, Travel and Adventure
  • The Cute Machine
  • The Dark Eye
  • Wrath of the Gods

You will need the data files from the games' original install media, and a daily build of ScummVM. For Macintosh editions, you will need to run the disc images through Dumper Companion in order to extract a complete copy of the files. In the case of The Dark Eye, you will also need the files created by the installer. As always, if you run into issues, please let us know on the issue tracker.

If your favorite isn't in the supported list, don't despair! ScummVM can open Director titles from 1995 and earlier, so grab the latest daily build and give it a try. Titles released after 1995 tend to use Director versions that aren't supported yet. If you get stuck, reach out on the issue tracker and we can try and help.

If you've encountered a Director title that isn't recognized by ScummVM, that's great! Please let us know on the issue tracker so the title can be added to the detection list.

Engine versions

As of the time of writing, ScummVM can detect 1595 different titles based on Director, with the following counts by Director version:

D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 D11
12 1 14 288 602 262 194 97 138 31 36 4

ScummVM 2.8.0 will be the first release to include preliminary support for the D5 version of Director! A lot of work has gone into overhauling the codebase to support multiple cast libraries, however there is still plenty of work to be done adding Lingo functions and Xtras.

Adding support for a new Director version is a tricky process. First we need to check for any changes to the movie file format, then add in any new engine and Lingo features, and finally start working through issues in a bunch of candidate games. Because of the difficulty of keeping track of facts vs. assumptions, we only enable features for an engine version once they have been confirmed with testing. This is why D6 and higher games will not open at all, even if the code we have would somewhat work.

Support for the D2, D3 and D4 versions of Director has steadily improved. We are building up a suite of unit tests, both for Lingo and for the various drawing modes in the renderer. Most of the remaining bugs are related to undefined or incidental behavior: consider how two types in Lingo are compared, or the precise order that events are processed in, or how functions respond when you pass them nonsense data? All of these things are relied upon by thousands of Director products! And ScummVM must support all of them as closely as possible.

XLibs

Macromedia provided a C API with Director, which proved to be very popular for adding new engine features. There are hundreds of XObject and Xtra libraries in circulation, and each has to be reverse engineered and added as code to ScummVM. As of the time of writing, we have added support for 63 of these libraries.

A common use case is video; Director includes the ability to play back QuickTime videos as a sprite, but computers of the day would struggle with the added overhead of Director's renderer if the video resolution was larger than a postage stamp. To work around this, several XLibs exist to play back video as an accelerated overlay. Quite a few D3 and D4 titles are missing video playback due to this type of XLib, and we plan on creating a common method of supporting these.

The good news is that most uncommon XLibs did only one or two things, and are pretty easy to add initial support for. Part of the XLib format is a list of docstrings explaining each of the functions; there is a script in /devtools which can turn these docstrings into stubbed C++ code, saving a lot of hassle.


That's about it! We have some very dedicated volunteers scouring every corner of the globe for obscure Director titles and testing them; if that sounds like your bag, you can visit them in the #engine-director channel of the ScummVM Discord. Until next time!

GOG.com Affiliation Links for ScummVM are no longer supported

For many years, the GOG.com affiliate program has been a great way to support the project. By using our affiliate links to buy your favorite games, you granted us a small commission from GOG.com – without having to spend more than the original price.

Earlier this year, GOG.com announced the discontinuation of its old affiliate program. Instead of using a simple referral link, they are now using two external affiliation networks that contain cross-site tracking, as considered by many ad blockers. Since we have been affiliated with GOG.com since 2009, we were able to use the legacy referral links for a couple of months after they initially discontinued them.

Unfortunately, our affiliate links were put to rest effective November 22nd, 2023.

Since we don't feel comfortable that you would have to use external affiliation networks just to support the project, and because tracking users is against our project policies, we decided not to join the new program.

We still value GOG.com as an excellent source for legally purchasing games supported by ScummVM. Even though we are not affiliated with them anymore, we'll continue to support their releases, e.g. by adding and updating the game data files' signatures for the ScummVM supported games that are available for purchase from GOG.com in our detection database and providing links to their releases in our list of available games.

Thank you GOG.com and everybody who used our affiliate link for your support over the last 14 years!

The teenage detective returns!

Put your sleuthing caps on, detectives—it’s time to solve some mysteries! We are proud to announce that a whole batch of HeR Interactive’s Nancy Drew games are now ready for public testing:

In Nancy Drew: Stay Tuned for Danger, a soap opera star needs your help in uncovering who is behind a series of death threats that sound a little too real for comfort.

Nancy Drew: Message in a Haunted Mansion has you helping out with repairs in a Victorian mansion. However, a series of strange accidents, plus some murmurings of a ghost haunting, quickly threaten to turn this renovation into an investigation.

In Nancy Drew: Treasure in the Royal Tower a failed ski vacation at an eccentric castle with even more eccentric inhabitants soon turns into a quest to uncover a lost piece of the history of the French Revolution.

And if you’re looking for something more thrilling, in Nancy Drew: The Final Scene, you will find yourself racing against the clock to find your kidnapped friend, while also saving a historical theater with more than a fair share of secrets from demolition.

The games are available for purchase through HeR Interactive’s website or Steam (Stay Tuned for Danger is not available on Steam, however). Or if you already own a CD copy of the games, their English and Russian variants will also work. To play them with ScummVM, you will need a daily development build. As always, if you encounter any issues, please submit a bug report to our issue tracker.

End of GSoC ’23: Final Submission

Google Summer of Code is coming to an end. Thanks to my mentors, it has been a pleasant experience.

Here are the things I have worked on during this Summer:

Universal Packaging Script

Code: https://github.com/scummvm/scummvm/pull/5300

Tasks Done

  1. Identify metadata requirement for DLC download support for various distribution stores. It has been compiled here.
  2. Create schema for metadata and JSON files like dlc-games.json and export-platforms.json to maintain all required metadatas for games and how to process those metadatas for different distribution stores.
  3. Write a python script that takes game location, platform type and the ScummVM binary location as parameter. It will determine the output based on the export-platforms.json and dlc-games.json. E.g. For Android Play Store, we will get a new AAB file bundled with games.
  4. Add entry for Android Play Store in export-platforms.json and all required dependencies (bundletool.jar, python scripts, etc.).

Possible Improvements

  1. It takes a lot of time to package games one by one. Create an option to mass package games from a folder.
  2. There are some dependencies that are required to be installed on the host computer. E.g. specific version of Android SDK, Java, Python, etc. Utilize container based approach for quick setup.

In-app DLC Downloader

Code: https://github.com/scummvm/scummvm/pull/5134

Tasks Done

  1. Create GUI dialogs and widgets to interact with the DLC Downloader and to show download progress, in-progress DLCs, etc.
  2. Create DLC Manager to manage downloading DLCs.
  3. Create an abstract class called Store. Different distribution store will need to implement this class using their store-specific API/SDK.
  4. Create a new path called dlcspath for storing the downloaded DLCs.
  5. Create a new compile time flag, USE_DLC for including DLC downloading support. This is disabled by default. To enable it, use --enable-dlc with ./configure. It will only be included on backends where it is supported (kFeatureDLC is true).

Possible Improvements

  1. GUI dialog for DLC Downloader could see some visual overhaul like list with game icons.
  2. Generalize DLC to allow other types of data like themes, icons, shaders, etc. Currently, only games is supported.

ScummVM Cloud Integration for DLC Downloader

Code: https://github.com/scummvm/scummvm/pull/5134

Tasks Done

  1. Implement ScummVM Cloud to download games from a URL (ScummVM’s server) by utilizing existing networking methods.
  2. Handle zip extraction and deletion of zip after extraction.
  3. Improve existing method Archive::dumpArchive().

Android Play Store Integration for DLC Downloader

Code: https://github.com/ankushdutt/scummvm/commit/c313a43cf1415598144c816474eb38ddbe56a8b1

Tasks Done

  1. Implement Play Store DLC support by using Play Core SDK for Java.
  2. Create JNI methods in C++ to interact with Java code for Play Store or vice-versa. Java code utilizes the SDK to implement functions necessary for handling download.

Possible Improvements

  1. Create script to generate the list of assets with required details like – id, name and detection keys for ScummVM configuration like engineid, gameid, extra, guioptions, language, platform and description directly by utilizing the dlc-games.json.

Documentation

Link: https://wiki.scummvm.org/index.php?title=Game_Packaging

GSOC ’23: Final Report

So originally, the goals for the project were to optimize the pixel blending code in the rendering code for the AGS game engine in ScummVM. The problem was: I completed that goal about half way through the coding period. So me and my mentors talked and what I did after was optimize the rendering code that most other engines in ScummVM use. I used SIMD cpu extensions to net a pretty huge performance gain.

Basically, in the AGS renderer, it got a 5x improvement all around and a 14x improvement in the best scenarios. In the global rendering code for all engines to use it got a 2x improvement all around. Here are the speed up results.

The most challenging part is knowing where to start. First, you must get to know your mentors really good, eg: calls, messaging, etc. If you don’t you’ll be left alone and not knowing what to do. And second, if a coding project seems big you should take three steps.
One: Figure out where you are and the actionable steps you can take to get there. What are the big milestones you have to hit along the way? Do you need to complete something else first to efficiently implement another feature? Should you write tests first? etc.
Two: Get the bare minimum code written. I know that sounds funny, but you should just get stuff working to start. This gives you plenty of time to look at your code and to step three.
Three: Make your code the best code anyones seen. Now that you have 90% of the code written, you can optimize it and make it cleaner and tie up any loose ends like updating the tests, making a PR, etc.

Once again, I’d like to thank Google Summer of Code 2023, ScummVM for the opportunity to work on a project like this and learn so much. And I’d like to thank my mentors for helping me when I was stuck, and teaching me how to work in a team.

So this is the end?

So, I’ve updated the AGS blending code to use the new SIMD cpu features detector and added AVX2 support for AGS. All I have left is to get my PR merged, tie up some loose ends, and submit my final submission for GSOC 23. I’ve had a great time coding with my mentors, @criezy, @lephilousophe, @ccawley2011, @somean, and @sev. They have been a great help and very responsive, and it’s been such a blast this past summer coding with them! I’m at college now and got plenty of work ahead of me, so bye!

GSoC summary and final report

Hi all, my name is Kartik Agarwala(hax0kartik) and in the past three months, I worked as a GSoC student under the mentorship of Eugene Sandulenko(sev) and Strangerke with ScummVM. The main focus of my project was to integrate the CRAB engine, developed by Pyrodactyl Games, into ScummVM. This blog serves as a final report of what I accomplished in this three-month period.

What’s been done?

The original milestones as per the project proposal were:

  1. The skeleton engine with some of the main classes is compilable within ScummVM.
  2. Skeleton engine is extended to fully include the first component. This includes loading and parsing XML files.
  3. Work on the graphics component.
  4. Get Unrest into a somewhat working state by integrating the essential components.
  5. All the components are integrated aside from audio.
  6. Integrate audio and ensure that the game is playable.

All of these milestones have been met.

CRAB has been fully integrated into ScummVM and Unrest is playable from start to end.

My changes were merged in the main ScummVM branch with PR #5229. You can also see the “feat-CRAB” branch in my fork to view the most recent changes I made.

Accepted changes

I created a number of PRs in the last three months, all of which were accepted. The following are some of the more important PRs that I created with a brief description.

  • PR #4753: Made text selectable in the input boxes.
  • PR #4766: Implement MKV decoding.
  • PR #4801: Added ProjectorArchive for the Director Engine.
  • PR #5229: Integrate CRAB engine into ScummVM. This was the main PR for this project.

What work is left?

The following list of things need to be improved or are missing:

  1. Filenames are inconsistent.
  2. Unimplemented “fade-in” effect of music.
  3. Saving/Loading is only available through GMM load/save menus.
  4. “Options” menu selectable from the main menu is broken.
  5. Keymapper needs improvement to make the game fully playable using only a keyboard.
  6. Unimplemented “ironman” saves.

Conclusion

My time as a GSoC student has been fantastic. In the last three months,  I have learned a lot from my mentors and the community. I want to note that the ScummVM community particularly has been very welcoming and helpful to me.

I would like to extend a special thanks to my mentors, Eugene Sandulenko(sev) and Strangerke who answered a lot of my questions and provided continuous feedback on the changes I was making. Their expertise helped me complete the project successfully.

Oh, and, I was given the offer to join the ScummVM team, which I have accepted 🙂 I hope to continue working on the CRAB engine and help maintain other engines in the time to come.

Thanks for reading!