Red Comrades in ScummVM (not GSoC*)

(* my site puts a post into RSS only if it has «GSoC» in it, so I kinda had to)

One of my favourite adventure games from my childhood are Red Comrades — these, actually, might even be the reason why I liked the genre. So, I’ve been keeping my eye on it after it was announced that the engine will be added by one of the GSoC students. Finally, there was a support announcement, and I’ve decided to play the games during the holidays.

This post is some kind of a less-formalized-than-bugtracker feedback. I’m going to actually submit some reports later, providing comparison with how the actual game worked, with screenshots, saves and probably even videos. For now it is only my experience compared with the impression of the game I remember.

Overall, the support is good. Both games are fully playable and there are no gamebreaking bugs — you can play it from the start to the end. Now onto my grumbling.

The Fonts

First thing that caught my eye: the fonts. I’m not sure why I’ve always played the game with subtitles, but now for me the game is incomplete if subtitles are off. I didn’t compare screenshots of the font yet, and IIRC it was just a usual Arial — but it looked a little bit off to me. To be more precise, there are multiple microissues with fonts/subtitles that I have.

First: I can see subtitles «jump» and I don’t remember that in the original game. Meaning that line height is different for different glyphs: for example, if line has «p», it will draw higher than a line without it. It can be easily seen in the dialogs how such lines go after each other and show up on different position. If the subtitles text is long, it will appear in two or three lines on the screen, and you could also see sometimes that these lines have different line height, as if one line is glued too close to another.

Second: there was a different font for inventory descriptions. ScummVM uses the same as in subtitles.

Third: when player is prompted to select an option in a dialog, these options have no outline or something. I don’t remember how exactly it looked in the original game, but for some reason I believe it was easier to read than in ScummVM.

The Walking

A really long time ago I and some other folks were making a fan continuation of the Red Comrades on Wintermute Engine. We were tinkering with original game resources, and ripped off all the backgrounds, music, sounds, animations, text lines and even walking regions. Now I don’t remember exactly if the original game also had some anchor points or did we add these in WME — but I remember that in original you could see heroes walking along some curved roads in the backgrounds pretty naturally. In ScummVM, however, this behaviour seems to be broken sometimes. I will prepare some save files to illustrate that in issues, but for now I will try to describe it. Again, there are two small issues that might (or might not) be independent.

First: a hero might ignore the walking region. Sometimes you could see how they are walking on the air or on the wall — obviously, outside the walking region. Sometimes I could see «Walk bug: Point doesn’t belong to any convex» message in the ScummVM terminal window, but I guess that only happened if heroes were spawned outside the walking region after fast travelling via the map. But when that happened, they just jump to the closest point which is in the region, so I’m not sure that’s the reason.

Second: pathfinding is a little bit strange. For example, we have three buildings on the screen: on the left, on the top and on the right. Say, our hero is at the top building entrance. We click at the right building. The hero starts walking towards the left building, and when they reach the left building entrance, they turn around and walk to the right building where we asked them to go. Not really critical, as they go where we want to in the end, but seems pretty strange, and, again, I don’t remember that in the original game.

The Rendering

I only remember that in the first game, and it’s not like that was happening all the time, but there were a few places where animations could «glitch» a little. What I mean is that a part of the sprite is not erased, so, for example, we could see two heads of the same character.

In some cutscenes there could be some «rainbow rubbish» pixels at the bottom. Not frequently, and I’m not sure that is easily reproduceable, but I’ve seen it happen.

The Cutscenes

Sometimes (mostly in Red Comrades 2?) there was just a lag instead of an animation/cutscene. For example, a hero walks towards an object and must play a unique animation of picking up or something, but instead your game freezes for a moment and then continues after (as if the animation was played and nothing happened).

Another variation: no animation is playing, but you can hear the sound effects, and the game continues after cutscene «has played». Most striking example is in the beginning of «Part 3» in Red Comrades 1, and I even have a save file for it. It shows a pretty long cutscene, but all you see is a static background.

In Red Comrades 2 there was another issue once: the game shows you that something has changed on the other screen, but in ScummVM this animation is very short, so you could even miss it.

Finally, there was an audio desync in the first cutscene in Red Comrades 2: you already seen the action on the screen, and only then you hear the corresponding sound effect.

The only almost-gamebreaking bug

In Red Comrades 1 I once had a strange dialog situation (maybe it activated twice or something): the dialog has stopped, as if it was waiting for me to choose an option, but there were no options on the screen, no cursor and no context menu available. So I couldn’t walk away or select an option to stop the dialog or pretty much do anything at all. But I think I just pressed Escape and dialog finished, so I didn’t lose any progress because of reloading.

The Speed

OK, another thing that was one of the first I’ve encountered, is walking speed. It is configured on the same screen you can turn subtitles on, so I’ve made it a little bit faster, and got a very fast walking heroes. I’ll have to compare that with the original, but I think I’ve got the max speed while it wasn’t on max in the options. I’ve just lowered it and played normally, so that’s not much of an issue.

The loading speed, however, is. It wasn’t painfully slow while I was playing, but when I’ve launched the original game, I was surprised how fluent it was. When switching screens, or loading animations, or loading some assets like inventory suitcase and options screen, you can easily see that ScummVM loads it, because it takes some time. In the original game it is barely noticeable. This might be the reason of some issues I’ve mentioned up there — such as audio desync or lags instead of the animations. TODO: I think the game was able to work with unpacked archives, and maybe ScummVM would work faster if I’d unpack its archives manually.

The Saves

Almost forgot about these! It was a little bit strange to see that you can’t use ScummVM own saving dialog. The cloud sync is not triggered when game is saved (or loaded via in-game menu), which is also a little bit unusual. And there are no autosaves, which could’ve been useful for me, because I didn’t save really frequently and thus I was lazy to play through the entire sequence again just to reproduce a bug I have just seen. Otherwise I would’ve have more saves to demonstrate these issues with.

Some technical information

I’ve used a daily build, ScummVM 2.3.0git11675-ga455b4c84a (Dec 31 2020 04:59:17).

I’m aware of two versions of the Red Comrades 1 game (which have different .exe files), but the only important difference between these is probably that one version has a compressed audio. I believe I’ve been playing the lossless version, and the game .exe version is 1.0.0.1.

I’ve been mostly playing on a laptop with Win 10 1909 (x64), and I also did a few quick runs on a desktop with Win 7 (x64).

You cannot Comprehend It

It's time to announce testing for a further two Glk subengines. The first is the Glulx interpreter, for which there are many free games available at the IF-Archive.

The second is the Comprehend engine games developed by Polarware. Here's where things get a bit more complicated. At the moment, only the first three Comprehend games are ready for testing - Transylvania, Crimson Crown, and OO-Topos. These three are also freely available at the IF-Archive. There is also, however, a re-release of Transylvania also available for free from the Graphics Magician website which isn't yet supported.

As for the other Comprehend games, Talisman will likely be supported in the future, but it requires significant further work. The other remaining games like Transylvania III use a completely different engine, so will remain out of scope for the foreseeable future.

So if any of you are interested, try downloading the relevant games and give them a try. You will need a daily development build. As always, please submit the bug reports to our issue tracker.

GSoD summary report


My GSoD task was to improve the Doxygen-generated documentation of ScummVM in terms of its structure, completeness, language, style, coherence, and overall usability.

Completed tasks


This is my final GSoD deliverable:

https://doxygen.scummvm.org/modules.html


To be clear, I have not created all of this from scratch. Doxygen comments have existed in ScummVM header files since forever, as they are a standard way of documenting C and C++ code, so that it can be used and understood by other developers (which is of huge importance especially in an open-source project).


However, the existing doxygen documentation was to a large extent incomplete and random. There was practically no structure to it, which I pointed out in my initial pull request to the repo that was part of my GSoD application. You can compare the screenshot of the old structure in that PR to the currently hosted version.


The specific tasks completed during GSoD involved:


  1. Setting up doxygen build configuration. In the course of discussions with developers, we decided to bring the doxygen build framework from a separate repository to the main one and change the configuration to a large extent.


  1. Refreshing the style of the doxygen output. This was a low priority task. I prepared 4 different CSS alternatives, uploaded them to my github.io server and presented to developers. The most favored option was chosen by popular vote.


  1. Adding doxygen groups to all header files from the main folders of the scummvm repository to create a structured documentation set. This was done at the same time as the actual review of the files (next point).


  1. Thoroughly reviewing all doxygen comments in the most important header files from folders ‘common’, ‘image’, ‘engines’, ‘graphics’, and ‘audio’.


In the course of GSoD, 12 of my pull requests were merged into the repository, all of them properly reviewed by code owners. The diffs in these pull requests demonstrate what kind of additions/changes I was making in these files. A typical ‘review’ of doxygen documentation in a header file involved:

  • Making sure the output would appear correctly in the structure.

  • Making sure every single C++ element from the file had doxygen documentation.

  • Checking and fixing doxygen syntax, or adding doxygen elements to improve the final output.

  • Checking and fixing doxygen errors, such as broken references or incorrectly documented arguments.

  • Typical language and style check.


After opening a PR, I would always generate an HTML documentation set for the specific set of files covered in the PR and upload it to my github.io, to provide reviewers with a preview of how my changes affect the resulting output.


I was also documenting my progress by creating weekly blog posts which the organization kindly hosted on their Planet platform so that anyone interested could learn exactly what I was working on.

Prioritization


After 2-3 weeks of the project, both me and mentors realized we could approach in a more efficient manner. I started off reviewing the files alphabetically, whereas the correct way would be to approach them from the highest to lowest priority. The other reason for this was that the reviews were taking longer than expected and it became obvious that reviewing all header files in the large ScummVM codebase was unrealistic.


We decided to create a Google Spreadsheet that contained all the header files which were in scope of GSoD, and a mentor assigned a priority to all of them. We decided to focus on high priority files first and then, if possible, move to lower priorities.

All high priority files have been reviewed and a number of the lower priority files underwent a language and style review.

Current state of the project


At this point, the most essential and most frequently used header files in the project have complete and informative documentation. This means that open-source developers trying to work with ScummVM code can now rely on documentation much more than before. Whenever they are in doubt how to use a certain method or structure in their code implementation, the new doxygen documentation should help them a lot.


The documentation set is well-structured, which means it is now possible to find information by just navigating the menu on the left, rather than trying to search and hoping for the best.


All this is particularly critical when it comes to attracting new contributors to the project as it lowers the initial level of difficulty when working with the ScummVM codebase.

Follow-up work


The final deliverable is not perfect and still requires work. Firstly, only high-priority header files ended up in the scope of GSoD, plus a small number of lower priority ones. This means that there are still some blank spaces in the documentation that need to be filled. There are some files that lack doxygen almost completely, and a close cooperation of a technical writer and developers will be required to fix these cases.


Secondly, I will be creating a doxygen style guide so that developers are aware of the rules for using doxygen. Most importantly, that everything should be documented. This is essential in an open-source project where developers sometimes come and go. If they leave their piece of code undocumented, it might become a problem. I already have a full set of rules in mind but need to discuss with the main stakeholders of the project and then include it in documentation.


Another potential improvement is adding more content to the main landing page of this documentation. It is currently almost blank while it could serve as a source of useful information, like a description of the high-level structure of the codebase that would help new developers navigate the source code.

Challenges


The biggest challenge I faced was that I was essentially working on the same set of files that all other developers. My deliverable is not created from a separate set of documentation source files that nobody else was interested in apart from me. It is generated from ‘living’ code. So whenever I was changing something in a file on my fork, it was always highly probable that some developer was making changes there too, and we would end up merging at roughly the same time. This is the case especially for very active projects, like ScummVM.

As a consequence, while the review of my PR was ongoing, some merge conflicts would be appearing and some of them could be quite nasty. Because of one such conflict, I had to completely revert my review of one big file that took me several hours to complete.

Summary


Overall, I see this project as a big step towards having really valuable developer documentation in ScummVM. There are definitely some further improvements that can be made (and I will be working on them), but compared to the state of this documentation from three months ago, the progress has been considerable.



GSoD week 10

 Hi!

GSoD timeline is slowly coming to an end and I am slowly approaching the end of the list of high priority files defined for this project. As always, you can track my progress here:

https://docs.google.com/spreadsheets/d/1eslXDVzNvqnoQH1teHpNl_-ZdQnRgYve98lwCeTodyg/edit#gid=0

As for the status of the PRs, we got #2627 merged, which is great because it was massive amount of content. We even managed to avoid conflicts. Another big one still open and awaiting review is #2633.

As you can see in the spreadsheet, there a couple files left for me to review. Most of the stuff from the 'graphics' folder is done locally but I am still running some last checks on it. The PR (#2645) is already open with the first two files.I got some comments there from @criezy and addressed them. The rest of the graphics files will be added tomorrow.

Then, the only remaining task will be the four last headers - one from 'image' and three from 'audio'. Busy weekend ahead of me :)

The Red Comrades game series goes supported

In 2019, Andrei Prykhodko (whiterandrek), a GSoC student, was working on support for the Red Comrades game series (Russian: Петька и Василий Иванович).

Red Comrades is a 2D adventure game played from a third-person perspective. The game's protagonists are from Dmitri Furmanov's 1923 novel Chapaev: historical Russian military figure Vasiliy Chapayev, his aide Petka, and the machine gunner, Anka.

After a long time in development, the ScummVM team is happy to announce support for the first two games in the series: Red Comrades 1: Save the Galaxy (Russian: Петька и Василий Иванович спасают галактику) and Red Comrades 2: For the Great Justice (Russian: Петька и Василий Иванович 2: Судный день).

So break out your copies of the games and give them a try! Unfortunately, only the original Russian Windows versions are supported. The English versions for iOS and Android use a completely different engine.

If you don’t own a copy, you can get the demo from our demos section. You will need a daily development build. As always, please submit the bug reports to our issue tracker.

Big adventures ahead - Little Big Adventures

Once upon a time in the past - around the year 1994 - a software company called Adeline Software International released a game titled “Little Big Adventure” or “Relentless: Twinsen's Adventure”. This game, a classic pseudo-3D action adventure game with an epic story set on a fantastic planet, has now entered the testing stage in ScummVM. Please, note that a few features of the original game are not implemented yet. However, we also added a few features which are new for the game.

Features not available yet and known issues:

  • The holomap is not yet available
  • Changing the rendering details is not supported yet
  • The credits sequence has a few rendering bugs

New features:

  • An option to disable wall collision damage
  • Improved UI navigation

This was all made possible by the people behind the TwinEngine project and the LBA community.

Please test the game with the latest development build and report any issues you find on our bug tracker.

GSoD week 9

 Hi!

Week 9 has been quite productive and I'm happy with the results, even though I haven't met my target of completing doxygen review on all high priority files.

The remaining files from the common folder are done and in PR#2627. They have been reviewed by @criezy (as always, huge thanks for the swift review) and I'm hoping we will get that merged soon.

I have also finished my work on headers from the engines folder and they can now be found in PR#2633. Four of these files have been categorized as high priority and to the remaining ones I have added a doxygen group definition so that they appear in the right place in the structure.

The engines headers gave me a bit of a headache and I had to spend much more time on them than initially planned. First challenge was to grasp all the concepts of engine, meta engine, Advanced Detector and how they relate to one another. I also had to make sure the references to various methods and structures that are declared in other files are resolved properly. Without any exaggeration, I can tell you that when reviewing a 700-line long file, I probably rebuild it about 50 times as I crawl through it, to make sure everything looks correctly. Fun!

I am now working on five files from 'graphics', they will be in PR very soon. Then one file from 'image' and three files from 'audio' and the high priority ones will be finally done.

This post is coming late (as usual), the next one will be here soon.


GSoD week 8

 Hi!

I'm posting a bit later than planned but I wanted to make sure one of the crucial PRs goes in before.

I've made some good progress this week and I am hugely thankful to @criezy for quickly jumping onto my reviews and leaving tons of great comments, some of which allowed me to understand the code better.

The biggest achievement of the week is that PR#2612 went in with some serious doxygen improvements in two big files: stream.h and system.h. Well over 2,5k lines of code overall. One significant change I made in system.h was the introduction of doxygen groups into the file (discussed in discord), so that now the documentation has a nice structure that is also reflected in the navigation pane on the left. I'm hoping this will improve the usability by a lot. Previously everything was presented on one page and because this is the largest header file in the scummvm codebase (or at least I believe so), this made the doc look a bit messy.

PR#2535 went in as well after a series of reviews.

Right now, I am preparing to open a PR for the last five high-priority files in the common folder (almost done locally). Then, I am moving on to engines, graphics, image, and audio. In each of these folders, we have identified a few header files as high priority.

For once I am happy with my progress this week and I'm truly hoping that in the next post I will be able to report that I am moving on to medium-priority files. Some of these have actually already been reviewed before we came out with the whole prioritization idea.

GSoD week 7

Hi!

I just realized that I completely forgot to write a blog post for last week so this one will be short and another one is coming in a few days.

I'm struggling quite hard right now to find time for the project and constantly feel that I am behind on the timelime. Unfortunately, my availability is hugely affected by the epidemic (the situation in Poland is really bad right now). Being locked up at home with two bored, attention-craving kids doesn't leave me much time for my regular job, let alone for GSoD.

The good news is that I got some great reviews on PRs #2535 and #2499. I have addressed all comments on the first one and it is now pending approval, while the second one is already merged.

One of these PRs contained tons of changes in the str.h file but in the end I had to revert all of them. This was due to the fact that in the meantime this file was changed extensively on master and resolving the resulting merge conflict would be too time consuming and too error prone for it to make sense. It is unfortunate to see a couple hours of work go to waste like that but I got used to the thought that it might and will happen.

I am finalizing a lot of changes locally and will be opening PRs during the weekend. I already made arrangements to ensure that I have sufficient time for GSoD in the next few days so hopefully you will see some more activity from me on Github and Discord.

GSoD week 6

 Hi!

Just a quick update this week.

I'm still waiting for someone to have a look at PRs #2535 and #2499. Hopefully they can get it soon.

I have quite a bit of work done locally but I'm not in a hurry to open another PR. My plan is to open a PR around the weekend with 10 more high-priority files with added/edited doxygen comments. The files in question will be the ones from stream.h up to metaengine.h from this list.

I am still investing a significant chunk of my time into developing the knowledge of aspects of C++ used in ScummVM headers. Figuring out how streams work was the latest challenge :)