GSoD week 1

Hi!

As the first week of my GSoD project is coming to an end, I'd like to give you a recap of what I've managed to accomplish so far.

I've worked a bit with my original pull request which was part of my GSoD application. I removed some files from it to restrict the scope and right now it is purely focused on adding doxygen groups to files in the common folder. This will greatly improve the structure of the doxygen output.We are still making some decisions there regarding terminology but I'm hoping this can go in pretty soon. You can view the output of that PR here.

Another PR opened right now is for the doxygen build framework. The idea is to put the doxygen build files in the main project repository. This will make it super easy for anyone who commits code into the repository to run a doxygen build on their updated header files, in order to check if the doc output is fine. You won't even need to push the files to do this, the build just runs locally from CLI using a simple doxygen command. Ideally, this kind of check could even become part of the ScummVM CI in distant future, but right now there are too many doxygen errors to even think about it. From my point of view, this PR is not very urgent - I have my local setup which I am using when rebuilding the doc a over and over as I work on the header files. What we commit into the repository needs to properly reviewed and discussed.

As to the main part of my work - according to the timeline, the first two weeks are about setting up the doxygen build and giving it a proper look. This is ongoing in the PR mentioned above. As the discussions there happen, I have started my work on the header files from the common folder, checking the language, doxygen usage and structure, adding missing information etc. I've got the first PR opened for a couple of header files and some more changed locally which will be put in a PR very soon.

The idea right now is to crawl over all of the header files alphatebitally and opening PRs for sets of 5-10 changed files. I don't think it makes sense to wait until I have done my work on all files from the common folder and only then open a pull request. That is 80 files and I don't want a discussion on one of them to block merging of all the other ones. Also, the longer the files remain in a pull request, the higher the risk of some nasty merge conflicts.

In week 2, I will continue my work in the common folder, hopefully reaching system.h within this week. This file is more important than the other ones and will require extra thourough review, and possibly extra support from the community to make sure everything in there is described in the proper way.

Season of Docs: Week 1

Well it’s nice to officially be underway! I’m happy to report that I’m ahead of my proposed timeline at the moment; some things that I expected to be time consuming have turned out to be pretty straightforward, and my proposal ended up with a lot more content in the prototype portal than I initially intended. All of which is to say that I’m very hopeful that this portal will be even more comprehensive than initially envisaged!  

A decision was made to focus (for now) on documentation for the more common ports - Windows, Linux, Mac, iOS and Android - so that has been my focus this week. I have first drafts in place for  “Getting Started” documents for most of those ports, which cover how to install ScummVM as well as how to get the game files where they need to be. 


My somewhat revised timeline now looks like this for the next few weeks:


Week 1 (this week): 

Getting Started:

Write draft install/quick start docs for Mac, Windows, Linux, iOS and Android.


Week 2: 

Using ScummVM:

•  The launcher - New screenshots and general edits.

•  Game/global settings - Edit engine-specific settings for clarity, general edits.

Advanced Options:

•  Configuration file - Rewrite. Possibly include the config keywords in the game/global settings pages and keep the contents of this page limited to “how to find” and “how to use” the config file.   


Week 3: 

Using ScummVM:

•  Keyboard shortcuts - Figure out the best way to display these!

•  Controls - Draft for control mapping on iOS/Android.

Advanced Options:

•  Command line interface - Edit to improve readability/usability.

•  Graphic filters and render modes - First draft.


I’ll probably keep making minor adjustments to this timeline in my weekly blog updates. 


Documentation can be viewed at https://scummvm-documentation.readthedocs.io/en/latest/


Well, I better get back to it. As always, hit me up on Discord with any ideas, thoughts or suggestions!


  


Game icons needed

Discord rich presence

The next version of ScummVM will support Discord rich presence, so you can show off to all your friends that you're playing ScummVM and the exact game you're playing. But to make the most of this feature, we need game icons.

If you ever wanted to contribute to ScummVM but couldn't find anywhere to apply your art/design skills, now’s your chance! Check out the new Game Icons repository for more details on how you can help.

Google Summer of Code Summary

Over the past few months I’ve made 451 commits to the main ScummVM repository, largely in the Macromedia Director and MacGUI subsystems:

451 commits to ScummVM

Most of this work was focused on the reimplementation of Lingo, Director’s programming language, so that Director games can be played in ScummVM. You can find every commit here, but this is the overview:

  • I did extensive work on Lingo objects;
    • I implemented factories, Lingo’s first iteration of objects.
    • I wrote a simple framework for XObjects (external objects). I started by implementing the FileIO XObject, and several XObjects have now been implemented by other contributors.
    • I implemented parent/child scripts, the successor to factories.
    • I wrote a template that allows any class to expose Lingo properties and methods, now used on scripts, windows, and cast members.
  • I did extensive work to make our Lingo implementation more robust, including:
    • Refactoring the grammar, reducing its hundreds of conflicts to just a handful
    • Properly differentiating between commands and functions
    • Implementing Lingo errors
    • Lots of bug fixing
  • I improved reading Director movies from file:
    • I identified and wrote code to handle quite a few previously understudied parts of the file format.
    • I implemented a reader for Macintosh sound resources.
    • I added support for loading Shockwave movies.
  • I implemented many minor Lingo features.
  • I rewrote the Lingo event system so that events are correctly ordered and delegated across multiple concurrent movies.
  • I refactored the monolithic Score class into a Movie, a Score, and a Cast, making the engine more faithful to Director and improving maintainability.
  • I implemented Movies In A Window (MIAWs), adding support for multiple concurrent movies.
  • I recreated the looks of the Mac System 7 desktop to go with MIAWs.

With the work I’ve done this summer, in combination with the other team members’ efforts, the Director engine is much more robust and has a greatly enhanced feature set. It can run a decent subset of Director 2-4 movies, but there is still plenty of work to be done to ensure feature completeness and to support later versions. I plan to stick around to help with this.

In addition, while my hope of full support of a real Director game has not yet been realized, I’ve helped to lay the foundations for that. With the foundational elements of Director and Lingo in place, if a bit more love is given to some individual games, full support should be easily attainable.

Please help us with testing upcoming ScummVM 2.2.0 release

Hello world, it is time for us to release the next version of ScummVM. Thus, we really need your help with testing the stable builds before the upcoming release, so we ensure better quality.

Please, pay your special attention to the following new titles:

  • Blazing Dragons
  • Eye of the Beholder - Sega CD version
  • Griffon Legend
  • Interactive Fiction games
  • Operation Stealth
  • Police Quest: SWAT
  • Prince and the Coward, English translation
  • Ultima IV - Quest of the Avatar
  • Ultima VI - The False Prophet
  • Ultima VIII - Pagan

We would also like you to play through another Sierra game: Police Quest: SWAT. Train to be a member of the SWAT team, and embark into dangerous missions to save civilian lives. This has been recently promoted for testing, so its call for testing has been included in the announcement for the upcoming release.

These games need to be tested with stable builds, not development ones. And report any bugs or oddities on our issue tracker.

We also need your help to update the ScummVM translations for this release. You can see the status of the translations, and work on them on our translations website. To log-in to the web site, you can either use your GitHub or Facebook account if you have one or create an account specific for this web site. Regular contributors can save changes to the translations but new contributors will only be able to make suggestions and vote on the suggestions. However we are looking for new regular contributors for some languages, so if you do not currently have full access and would like to get it, please contact us. Also, a few of our past contributors may have lost their full access a few months ago when we updated our permission settings. If you are one of them please contact us to see your full access restored.

Good luck with your playtesting, and let’s release version 2.2.0 at the end of September!

Wrapping it up

GSoC is almost over, less than a week now. Time to wrap things up and have a look at what has been done and what is still missing.

The overall goal was to bring the 3d functionality of the original Wintermute Engine to ResidualVM after the 2d parts had already been ported to ScummVM during previous GSoC events (based on Wintermute Lite instead), using OpenGL. Major steps include:

Rendering of 2d sprites using OpenGL. Also adding some of the 2d transformations supported by WME. Some functions are still not implemented although it is not clear if a 3d game is actually using them.

https://github.com/residualvm/residualvm/pull/1643

https://github.com/residualvm/residualvm/pull/1651

https://github.com/residualvm/residualvm/pull/1674

https://github.com/residualvm/residualvm/pull/1678

https://github.com/residualvm/residualvm/pull/1679

Loading of scene geometry and camera settings from a .3ds file and being able to display it for debugging purposes. A lot of the code here was taken from the original engine and adjusted where necessary. The .3ds loader was written from scratch as well as all OpenGL code.

https://github.com/residualvm/residualvm/pull/1647

Loading of models from a .X file, rendering and animating such models. In the first PR, again a lot of code was integrated from WME with minor adjustments (changing the math code or everything related to coordinate systems, for example). Most of my work consisted of the .X loader and everything mesh related like rendering and animation updates of the mesh. The first PR also contains some work related to path-finding. Again, a lot of the code was imported here but parts of it had to be changed.

https://github.com/residualvm/residualvm/pull/1649

https://github.com/residualvm/residualvm/pull/1662

https://github.com/residualvm/residualvm/pull/1663

https://github.com/residualvm/residualvm/pull/1665

https://github.com/residualvm/residualvm/pull/1666

https://github.com/residualvm/residualvm/pull/1668

https://github.com/residualvm/residualvm/pull/1670

https://github.com/residualvm/residualvm/pull/1672

https://github.com/residualvm/residualvm/pull/1676

https://github.com/residualvm/residualvm/pull/1680

https://github.com/residualvm/residualvm/pull/1681

https://github.com/residualvm/residualvm/pull/1682

https://github.com/residualvm/residualvm/pull/1683

https://github.com/residualvm/residualvm/pull/1684

https://github.com/residualvm/residualvm/pull/1686

https://github.com/residualvm/residualvm/pull/1687

https://github.com/residualvm/residualvm/pull/1688

https://github.com/residualvm/residualvm/pull/1689

Object selection and attachment of items to 3d actors.

https://github.com/residualvm/residualvm/pull/1653

Lighting and shadows. Lights are loaded from a .3ds file and set accordingly to script input. Stencil shadow volumes have been implemented and seem to be very stable. Shadow mapping, as supported by the original engine is still missing. Simple shadow textures which are displayed at the position of an actor are implemented but potentially buggy.

https://github.com/residualvm/residualvm/pull/1657

https://github.com/residualvm/residualvm/pull/1669

https://github.com/residualvm/residualvm/pull/1691

https://github.com/residualvm/residualvm/pull/1692

https://github.com/residualvm/residualvm/pull/1671

A shader based renderer has been implemented and the a 3d renderer interface was added to make the rest of the code independent of the specific renderer. Plenty of shader related stuff can be found in the above mentioned PR’s as missing function were implemented or bugs fixed only after the renderer was added.

https://github.com/residualvm/residualvm/pull/1659

Persistence for the ported 3d classes was implemented in a way that there would be compability between savefiles of ScummVM and ResidualVM.

https://github.com/residualvm/residualvm/pull/1655

Some somewhat unrelated PR’s, mainly smaller fixes and cleanup of the code.

https://github.com/residualvm/residualvm/pull/1660

https://github.com/residualvm/residualvm/pull/1661

https://github.com/residualvm/residualvm/pull/1664

https://github.com/residualvm/residualvm/pull/1667

https://github.com/residualvm/residualvm/pull/1677

https://github.com/residualvm/residualvm/pull/1690

https://github.com/residualvm/residualvm/pull/1693

https://github.com/residualvm/residualvm/pull/1697

https://github.com/residualvm/residualvm/pull/1698

https://github.com/residualvm/residualvm/pull/1699

The following PR did not go directly in ResidualVM, but first into ScummVM and was then synchronized as the fixes touched the common code between the two Wintermute ports.

https://github.com/scummvm/scummvm/pull/2345

This PR was just about integrating the existing ScummVM port into ResidualVM, so none of this code was actually written by me. It is included here for completeness sake.

https://github.com/residualvm/residualvm/pull/1644

Displayable results

The Wintermute games “Alpha Polaris” and “Mental Repair” have been successfully completed with the current port. The graphical output matches the original, at least to my current knowledge.

Known issues

Shadow mapping is used by a lot of games instead of shadow volumes and was not implemented, as mentioned above.

Sorting of rendered objects does not work correctly in some games, meaning that objects appear in the wrong order on the screen. At least in one case this seems to be related to viewport settings and 2d <-> 3d coordinate transformation. Also there seem to be issues with general 3d object transformations, leading to models being placed incorrectly on the screen or in the wrong position.

One game uses an unknown blend mode.

Viewport settings are still incomplete and don’t handle mismatches between requested dimensions by games and actual window sizes. The original has some code here which should be ported as well.

Future plans

I plan to tackle the above issues after taking a break from the code base for a few weeks and depending on how much spare time I will have available. Complete support for all Wintermute 3D games will probably be a more long term task, although I kind of like the idea of maintaining such a project.

Finishing up

Last week was the last official week in the GSoC work period. I plan to submit my final evaluation after I post this. There you will be able to see some of the highlights from my time with ScummVM this summer.

In the past week, though, I didn’t start work on any new features. I spent a while understanding the data path location mechanism that the Mac version of Journeyman Project 2: Buried in Time used, and I ultimately discovered some buried XFCNs. (As @djsrv helpfully explained, XFCNs are external functions from (HyperCard)[https://en.wikipedia.org/wiki/HyperCard] that can often be called as regular Lingo.) One of these just looped through all the volume names on the system, looking for the proper CD name. This Journeyman Project is split across three CDs, so some more additions will be needed.

I also fixed an obscure issue decoding the hex format strings for our Macintosh text implementation. Now, the main screen of Majestic looks almost perfect – with proper cursors, which I also patched to load properly – and consistent text formatting.

[]

Additionally, I investigated and fixed several bugs with my good old friends the widgets. Now Chop Suey also looks very accurate, and there is no more flashing cursor or cut-off text boxes. Text borders are now (almost) working properly again, which makes the Lingo Dictionary movies more pleasant to look at.

[]

For being the rendering guy this summer I haven’t posted nearly enough pictures, so I put those two here for you to enjoy. I’ve enjoyed working with ScummVM, and I plan to start contributing again once I get comfortable in my university schedule.

The (official) end of an amazing journey!

 Hello! GSoC has officially ended, and I'm in the final stage - submitting my work for the entire summer to Google!

When I first received the acceptance letter in May, I was very much baffled. It was exciting and a bit scary at the same time, because I have never ever worked on a project of this scale. Well, first time for everything I guess :)

So, what have I been doing this past week? Last week I left off saying my U32 PR was showing some activity, and this week, @criezy helped out a lot by reviewing my U32 PR, and providing much feedback. He also helped out pretty much entirely towards the MacOS/iOS platforms, because it was quite not an area (Objective-C) I was comfortable in. It was really a huge help. Apart from that, I mostly spent the entire week looking at the PR to see what could be changed, fixing my mistakes, and making the PR ready for merge. It has improved quite a lot! 

Criezy pushed an update for iOS, which led me to believe that there might be many more missing areas that weren't ported over to use U32Strings, so I had to check and implement for the same. Criezy and I left a comment regarding this at the same time!

First up was Android, which needed change in clipboards and OSD messages. OSD messages are simple messages displayed on the GUI. ScummVM supports many (many) platforms, as you might know already. Recently, a MorphOS Platform support PR was merged, which was quite nice! Anyways, some platforms themselves could be building stuff from GUI, and I haven't changed that or actually checked for those. So, I searched through the backends/platforms subdirectory to see where translations were used, as well as searching for Individual widget names. Doing so pointed out a lot of places where the arguments were expecting a U32String but were getting something else. After fixing those, and thoroughly reviewing, fixing more things, the U32 PR is in much nicer shape than before :)


Over to my Plugins task, Eugene pointed out that my solution towards the module separation of makefiles was not quite correct.  Well, I mean, it was correct - but it was not proper in the long term. I explained it last week, but the gist of it was that each engine would get a subfolder named detection and would have a module called "engines/engineName/detection". This would hold all detection objects, which would head on to the executable. And engines.mk would always enable an engine's detection module, whereas the engine itself would be dependent on parameters passed. Eugene's feedback was that it was duplication of work, and the objects to be added were to remain in the same module file.

I didn't think of how I could structure it more nicely, because everything was already neat and tidy. So, I also asked for some ideas on how would I solve the new suggested way, and he gave an idea that I could just run through modules list two times, and keep the information I need while resetting the other. Because this one approach would be better in the long run, I was trying to implement this next. I also had good progress with the create_project files, but that would probably need to redone due to the origin (makefiles) changing behavior.

I was working on these today and successfully changed it to as Eugene suggested. I am awaiting feedback though, and minor improvements will follow. After this, updating "create_project" files will follow. Once those are done, the Plugins task will also be in a very nice shape hopefully!

Submissions for our end report for GSoC will last till 31st August. I plan to submit mine after completing the create_project stuff and submit it maximum by 27th August.

On a more personal note, it's been pretty hectic recently. There's been a lot of to-do on my plate - not just GSoC stuff but also personal things, but I'm managing to keep up with everything, and am getting lots of things done! It'll probably be a while before I can relax & chill for a bit. My university released a schedule of my exams, which will start from 1 September, right when GSoC submission ends, which is really nice. These are my finals, which means I will be getting ready & preparing for new opportunities! I can only hope that these turn out to be as amazing as GSoC & ScummVM :P

Regarding GSoC, Most of my official tasks are either completed or near completion! I won't have as much time as I did during this summer, but I do want to keep contributing to ScummVM. It was my first time contributing to OSS. I hope the users find some use of my work done during this summer :D

It was an awesome summer. Thank you, everyone, for joining along my journey! 

Thanks for reading!

Are you ready for the Legend?

ScummVM Team is proud to announce support for a small freeware RPG game named The Griffon Legend. It was originally written in the FreeBASIC programming language by Daniel "Syn9" Kennedy and later ported by Dmitry Smagin to the GCW0 console, which we base this port on.

The original story says:

500 years after the Griffon-Dragon war, the Dragons have returned to reclaim their empire. Cities are falling, everyone is dieing, and the lone Griffon Knight Fayne is caught up in the middle of it all.

Since the game is freeware, you may grab a copy directly from our game downloads page or from the original site. You will need the latest development build to play it. Of course, if you encounter any bugs, please submit them to our bug tracker.

Wrapping up the Summer

Summer’s drawing to a close, and the school year is about to begin. Before that happens, I’m trying to squeeze in a few more tasks: exposing more Director features as Lingo objects, fully implementing chunk expressions, and cleaning up the implementations of Director 4 Lingo instructions.

One of the ways I’ve tried to make the D4 instructions cleaner is by reducing code duplication. First, I cut down the size of cb_objectcall by reusing functionality in the generic call. Then, I noticed that several variable-related instructions take a variable type argument, but none of the instructions handled every variable type correctly. After identifying the IDs for every type I could, I combined all this duplicated variable lookup code into a single helper function.

I also implemented Lingo errors, allowing for easier detection of illegal cases, which often indicate a mistake on our part.

Then, I began implementing better support of chunk expressions. Chunk expressions are references to substrings, such as word 2 to 4 of "the quick brown fox". You can use them to grab text:

set x to word 2 to 4 of "the quick brown fox"`

or you can use them manipulate the text:

put " foo" after word 2 of x

Previously, we only supported the former, and I’m in the process of implementing support for the latter.

Since summer’s almost over, I’d like to thank the ScummVM developers, especially sev, rvanlaar, and moralrecordings, as well as Google for its Summer of Code program. I’ve learned a lot and had a good deal of fun over the past three months.

I won’t be working on ScummVM full-time anymore, but this definitely won’t be the last you hear from me. We still have plenty of work to do on Director, and I plan to keep contributing in my spare time.

See you soon!