GSoD summary report


My GSoD task was to improve the Doxygen-generated documentation of ScummVM in terms of its structure, completeness, language, style, coherence, and overall usability.

Completed tasks


This is my final GSoD deliverable:

https://doxygen.scummvm.org/modules.html


To be clear, I have not created all of this from scratch. Doxygen comments have existed in ScummVM header files since forever, as they are a standard way of documenting C and C++ code, so that it can be used and understood by other developers (which is of huge importance especially in an open-source project).


However, the existing doxygen documentation was to a large extent incomplete and random. There was practically no structure to it, which I pointed out in my initial pull request to the repo that was part of my GSoD application. You can compare the screenshot of the old structure in that PR to the currently hosted version.


The specific tasks completed during GSoD involved:


  1. Setting up doxygen build configuration. In the course of discussions with developers, we decided to bring the doxygen build framework from a separate repository to the main one and change the configuration to a large extent.


  1. Refreshing the style of the doxygen output. This was a low priority task. I prepared 4 different CSS alternatives, uploaded them to my github.io server and presented to developers. The most favored option was chosen by popular vote.


  1. Adding doxygen groups to all header files from the main folders of the scummvm repository to create a structured documentation set. This was done at the same time as the actual review of the files (next point).


  1. Thoroughly reviewing all doxygen comments in the most important header files from folders ‘common’, ‘image’, ‘engines’, ‘graphics’, and ‘audio’.


In the course of GSoD, 12 of my pull requests were merged into the repository, all of them properly reviewed by code owners. The diffs in these pull requests demonstrate what kind of additions/changes I was making in these files. A typical ‘review’ of doxygen documentation in a header file involved:

  • Making sure the output would appear correctly in the structure.

  • Making sure every single C++ element from the file had doxygen documentation.

  • Checking and fixing doxygen syntax, or adding doxygen elements to improve the final output.

  • Checking and fixing doxygen errors, such as broken references or incorrectly documented arguments.

  • Typical language and style check.


After opening a PR, I would always generate an HTML documentation set for the specific set of files covered in the PR and upload it to my github.io, to provide reviewers with a preview of how my changes affect the resulting output.


I was also documenting my progress by creating weekly blog posts which the organization kindly hosted on their Planet platform so that anyone interested could learn exactly what I was working on.

Prioritization


After 2-3 weeks of the project, both me and mentors realized we could approach in a more efficient manner. I started off reviewing the files alphabetically, whereas the correct way would be to approach them from the highest to lowest priority. The other reason for this was that the reviews were taking longer than expected and it became obvious that reviewing all header files in the large ScummVM codebase was unrealistic.


We decided to create a Google Spreadsheet that contained all the header files which were in scope of GSoD, and a mentor assigned a priority to all of them. We decided to focus on high priority files first and then, if possible, move to lower priorities.

All high priority files have been reviewed and a number of the lower priority files underwent a language and style review.

Current state of the project


At this point, the most essential and most frequently used header files in the project have complete and informative documentation. This means that open-source developers trying to work with ScummVM code can now rely on documentation much more than before. Whenever they are in doubt how to use a certain method or structure in their code implementation, the new doxygen documentation should help them a lot.


The documentation set is well-structured, which means it is now possible to find information by just navigating the menu on the left, rather than trying to search and hoping for the best.


All this is particularly critical when it comes to attracting new contributors to the project as it lowers the initial level of difficulty when working with the ScummVM codebase.

Follow-up work


The final deliverable is not perfect and still requires work. Firstly, only high-priority header files ended up in the scope of GSoD, plus a small number of lower priority ones. This means that there are still some blank spaces in the documentation that need to be filled. There are some files that lack doxygen almost completely, and a close cooperation of a technical writer and developers will be required to fix these cases.


Secondly, I will be creating a doxygen style guide so that developers are aware of the rules for using doxygen. Most importantly, that everything should be documented. This is essential in an open-source project where developers sometimes come and go. If they leave their piece of code undocumented, it might become a problem. I already have a full set of rules in mind but need to discuss with the main stakeholders of the project and then include it in documentation.


Another potential improvement is adding more content to the main landing page of this documentation. It is currently almost blank while it could serve as a source of useful information, like a description of the high-level structure of the codebase that would help new developers navigate the source code.

Challenges


The biggest challenge I faced was that I was essentially working on the same set of files that all other developers. My deliverable is not created from a separate set of documentation source files that nobody else was interested in apart from me. It is generated from ‘living’ code. So whenever I was changing something in a file on my fork, it was always highly probable that some developer was making changes there too, and we would end up merging at roughly the same time. This is the case especially for very active projects, like ScummVM.

As a consequence, while the review of my PR was ongoing, some merge conflicts would be appearing and some of them could be quite nasty. Because of one such conflict, I had to completely revert my review of one big file that took me several hours to complete.

Summary


Overall, I see this project as a big step towards having really valuable developer documentation in ScummVM. There are definitely some further improvements that can be made (and I will be working on them), but compared to the state of this documentation from three months ago, the progress has been considerable.



GSoD week 10

 Hi!

GSoD timeline is slowly coming to an end and I am slowly approaching the end of the list of high priority files defined for this project. As always, you can track my progress here:

https://docs.google.com/spreadsheets/d/1eslXDVzNvqnoQH1teHpNl_-ZdQnRgYve98lwCeTodyg/edit#gid=0

As for the status of the PRs, we got #2627 merged, which is great because it was massive amount of content. We even managed to avoid conflicts. Another big one still open and awaiting review is #2633.

As you can see in the spreadsheet, there a couple files left for me to review. Most of the stuff from the 'graphics' folder is done locally but I am still running some last checks on it. The PR (#2645) is already open with the first two files.I got some comments there from @criezy and addressed them. The rest of the graphics files will be added tomorrow.

Then, the only remaining task will be the four last headers - one from 'image' and three from 'audio'. Busy weekend ahead of me :)

The Red Comrades game series goes supported

In 2019, Andrei Prykhodko (whiterandrek), a GSoC student, was working on support for the Red Comrades game series (Russian: Петька и Василий Иванович).

Red Comrades is a 2D adventure game played from a third-person perspective. The game's protagonists are from Dmitri Furmanov's 1923 novel Chapaev: historical Russian military figure Vasiliy Chapayev, his aide Petka, and the machine gunner, Anka.

After a long time in development, the ScummVM team is happy to announce support for the first two games in the series: Red Comrades 1: Save the Galaxy (Russian: Петька и Василий Иванович спасают галактику) and Red Comrades 2: For the Great Justice (Russian: Петька и Василий Иванович 2: Судный день).

So break out your copies of the games and give them a try! Unfortunately, only the original Russian Windows versions are supported. The English versions for iOS and Android use a completely different engine.

If you don’t own a copy, you can get the demo from our demos section. You will need a daily development build. As always, please submit the bug reports to our issue tracker.

Big adventures ahead - Little Big Adventures

Once upon a time in the past - around the year 1994 - a software company called Adeline Software International released a game titled “Little Big Adventure” or “Relentless: Twinsen's Adventure”. This game, a classic pseudo-3D action adventure game with an epic story set on a fantastic planet, has now entered the testing stage in ScummVM. Please, note that a few features of the original game are not implemented yet. However, we also added a few features which are new for the game.

Features not available yet and known issues:

  • The holomap is not yet available
  • Changing the rendering details is not supported yet
  • The credits sequence has a few rendering bugs

New features:

  • An option to disable wall collision damage
  • Improved UI navigation

This was all made possible by the people behind the TwinEngine project and the LBA community.

Please test the game with the latest development build and report any issues you find on our bug tracker.

GSoD week 9

 Hi!

Week 9 has been quite productive and I'm happy with the results, even though I haven't met my target of completing doxygen review on all high priority files.

The remaining files from the common folder are done and in PR#2627. They have been reviewed by @criezy (as always, huge thanks for the swift review) and I'm hoping we will get that merged soon.

I have also finished my work on headers from the engines folder and they can now be found in PR#2633. Four of these files have been categorized as high priority and to the remaining ones I have added a doxygen group definition so that they appear in the right place in the structure.

The engines headers gave me a bit of a headache and I had to spend much more time on them than initially planned. First challenge was to grasp all the concepts of engine, meta engine, Advanced Detector and how they relate to one another. I also had to make sure the references to various methods and structures that are declared in other files are resolved properly. Without any exaggeration, I can tell you that when reviewing a 700-line long file, I probably rebuild it about 50 times as I crawl through it, to make sure everything looks correctly. Fun!

I am now working on five files from 'graphics', they will be in PR very soon. Then one file from 'image' and three files from 'audio' and the high priority ones will be finally done.

This post is coming late (as usual), the next one will be here soon.


GSoD week 8

 Hi!

I'm posting a bit later than planned but I wanted to make sure one of the crucial PRs goes in before.

I've made some good progress this week and I am hugely thankful to @criezy for quickly jumping onto my reviews and leaving tons of great comments, some of which allowed me to understand the code better.

The biggest achievement of the week is that PR#2612 went in with some serious doxygen improvements in two big files: stream.h and system.h. Well over 2,5k lines of code overall. One significant change I made in system.h was the introduction of doxygen groups into the file (discussed in discord), so that now the documentation has a nice structure that is also reflected in the navigation pane on the left. I'm hoping this will improve the usability by a lot. Previously everything was presented on one page and because this is the largest header file in the scummvm codebase (or at least I believe so), this made the doc look a bit messy.

PR#2535 went in as well after a series of reviews.

Right now, I am preparing to open a PR for the last five high-priority files in the common folder (almost done locally). Then, I am moving on to engines, graphics, image, and audio. In each of these folders, we have identified a few header files as high priority.

For once I am happy with my progress this week and I'm truly hoping that in the next post I will be able to report that I am moving on to medium-priority files. Some of these have actually already been reviewed before we came out with the whole prioritization idea.

GSoD week 7

Hi!

I just realized that I completely forgot to write a blog post for last week so this one will be short and another one is coming in a few days.

I'm struggling quite hard right now to find time for the project and constantly feel that I am behind on the timelime. Unfortunately, my availability is hugely affected by the epidemic (the situation in Poland is really bad right now). Being locked up at home with two bored, attention-craving kids doesn't leave me much time for my regular job, let alone for GSoD.

The good news is that I got some great reviews on PRs #2535 and #2499. I have addressed all comments on the first one and it is now pending approval, while the second one is already merged.

One of these PRs contained tons of changes in the str.h file but in the end I had to revert all of them. This was due to the fact that in the meantime this file was changed extensively on master and resolving the resulting merge conflict would be too time consuming and too error prone for it to make sense. It is unfortunate to see a couple hours of work go to waste like that but I got used to the thought that it might and will happen.

I am finalizing a lot of changes locally and will be opening PRs during the weekend. I already made arrangements to ensure that I have sufficient time for GSoD in the next few days so hopefully you will see some more activity from me on Github and Discord.

GSoD week 6

 Hi!

Just a quick update this week.

I'm still waiting for someone to have a look at PRs #2535 and #2499. Hopefully they can get it soon.

I have quite a bit of work done locally but I'm not in a hurry to open another PR. My plan is to open a PR around the weekend with 10 more high-priority files with added/edited doxygen comments. The files in question will be the ones from stream.h up to metaengine.h from this list.

I am still investing a significant chunk of my time into developing the knowledge of aspects of C++ used in ScummVM headers. Figuring out how streams work was the latest challenge :)


Android Love

Our new and shiny Android port for ScummVM v2.2.1 is now live on the Google Play Store. After quite a long period of dedicated work from our team developers, and a month of public beta testing by members of our community who helpfully reported quite a few issues for us to address, we are finally ready to give you the stable release for our ScummVM Android app.

This app has been significantly re-written and tested on modern Android devices, running up to Android 10+. It includes new features which bring it up to speed with the desktop ScummVM application, such as FluidSynth support, Cloud Saves and more localization choices for the UI. Also included is the Local File Server (LAN) feature, whereby your device can act as a temporary file server allowing you to download files (eg. save files and even the config file) or upload new ones (eg. game data) using a web browser from a PC or another client.

With this release we have resolved a few long standing issues such as:

  • upgrading from previous versions without losing access to your previous configuration and save files
  • folder navigation (based on storage access permission settings)
  • Working special keys on the virtual keyboard (such as F-keys) as well as handling of repeated keys

Other significant improvements have been made to the interface of the application, such as the ability to choose whether the GUI will feature filtered graphics (bilinear scaling) or not (nearest neighbor scaling), several fixes for the virtual keyboard behavior to make it more responsive and closer emulate the behavior of a physical keyboard, and support for virtual mouse control using the analogue joystick from your controller.

By upgrading to the new 2.2.1 version you will be able to enjoy support for all the game engines that have been introduced on ScummVM since 2.0.0, such as Blade Runner, Ultima IV, Quest for Glory IV, as well as all the bug fixes and improvements made to the existing ones.

Of course we continue to monitor our community’s feedback and welcome any suggestions for improvement. As always, please report any bugs you encounter on the ScummVM bug tracker site.

We thank all our beta testers for their most valuable feedback and hope that you enjoy this version of ScummVM Android.

GSoD week 5

 Hi!

The last week has been very busy and I've spent a lot of time working on finalizing PRs #2535 and #2499. The diff is quite massive so good luck to the reviewers :)

I am slowly crawling through the high-priority header files, as defined in the spreadsheet. Some of them can be really time-consuming, I think I spent over two days on str.h. A huge chunk of this time is spent on reading, googling, and expanding my knowledge of C++: how to understand operator functions, how iterators work, streams, arrays, lists, strings...I'm surely learning a lot but it annoys that it takes so much time that is then not really reflected in the work I'm submitting.

The problem I am already starting to see is that reviewing my PRs will take quite a bit of time. Every doxygen comment that I am adding there needs to verified, simply because my understanding of the function/type/attribute might have been wrong. And as my PRs are getting verified, the header files that they are massively editing are being changed on master by other developers, which produces some nasty merge conflicts. I guess fighting merge conflicts might become one of the main challenges in the remaining part of the project.

I'm going back to my files, you can track my progress in the Google spreadsheet if you are interested. I usually put a green 'Reviewed' field next to the files that I have already gone through and included in a PR.