So, this week we have here, working playable version of engine. Some GUI bugs are still to be addressed, like scrollbars ( for Inventory and Console windows ) and titles, but we can keep them aside for later and focus on Game play part.
I intend to walk through game and and see what is working and what is not.
Already started noticing some bugs, like dragging torches into left torch holding hole makes them disappear and then I guess “click to continue” part needs some (much more) attention.
Am not sure about lifetime of torches and how to know when to lit new torch, but I feel like torches are burning out fast, so will have to work on that too.

Over the last week, I essentially rebuilt the Director engine’s Lingo compiler from scratch. In doing so, I’ve restructured it to eliminate many longstanding issues and greatly improved its long-term maintainability.

The Lingo compiler, in short, parses Lingo source code and turns it into bytecode, which is then interpreted by the Lingo runtime. Previously, the code generator and the parser were inextricably linked, both defined within a Bison grammar file. This made things really complicated for two major reasons:

1) Bison is designed to parse context-free languages – languages whose syntax can be described by a context-free grammar. Lingo is context-free in terms of syntax, so it can be parsed by a context-free grammar. However, Lingo in terms of semantics, Lingo is context-sensitive. In other words, the same thing can have different meanings based on the context it’s used in. One of the most obvious examples of this is variables. When there’s a global loop statement somewhere in the script, the variable loop refers to a global variable. When there’s a property loop statement somewhere in the script, the variable loop refers to a property variable. In most other cases, it refers to a local variable, but it can get much weirder. When used in a go command, the variable loop doesn’t refer to a variable at all, but to the symbol #loop. To deal with context-sensitive meaning within the context-free parser, we were basically keeping track of context with actions, and this context could then affect the lexer. This made the parser unnecessarily complicated.

2) Due to the complexities of doing code generation within the grammar, we had plenty of midrule actions, often alternating between symbols and actions several times in one rule. This, exacerbated by the context hacks, made the code hard to follow and quite brittle. Any tiny change brought the possibility of major breakage. Instead of properly defining a grammar that included all of Lingo’s often inconsistent syntax, we sometimes resorted to preprocessing Lingo source code into something the existing grammar could handle.

To fix this, I rebuilt the grammar from the ground up, leaving out anything from the old grammar that dealt with the language’s context-dependent semantics. What was left was a still rather ugly but straightforward, context-free grammar. And now, instead of outputting bytecode, the parser outputs an abstract syntax tree, which is then passed as input to a separate code generator.

An abstract syntax tree, if you’re not familiar with it, is basically just source code in the form of a tree data structure, making it easier to manipulate than raw text. For example, the Lingo source code:

on printStuff arg1
  put "foo"
  put 123
  put arg1
end

can be represented as a tree something like:

HandlerNode {
  name: "printStuff",
  args: "arg1"
  stmts: [
    CmdNode {
      name: "put",
      args: [
        StringNode {
          val: "foo"
        }
      ]
    },
    CmdNode {
      name: "put",
      args: [
        IntNode {
          val: 123
        }
      ]
    },
    CmdNode {
      name: "put",
      args: [
        VarNode {
          name: "arg1"
        }
      ]
    }
  ]
}

Now, to compile this function to bytecode, we just need to traverse this tree, generating instructions for each node:

c_stringpush "foo"
c_callcmd "put", 1
c_intpush 123
c_callcmd "put", 1
c_localpush "arg1"
c_callcmd "put", 1

In addition, having an AST makes it extremely easy to gather semantic information like variable types and adjust code generation as necessary.

With this refactoring, I was able to clean up the compiler quite a lot. Since parsing is separated from code generation, the grammar is now a lot simpler, and I’ve been able to eliminate almost all of the preprocessing and context hacks we used. Now, it’ll be trivial to make the compiler changes I need to finish up of chunk expressions. Perhaps more importantly, it’ll now be much, much easier to add support for later Lingo versions to the compiler, enabling more exciting stuff to come.

Interface first

hi hi, back here. another week working one ScummVM.

due to the final exam, i didn’t work whole heartly on ScummVM in the past week. But exam-period will come to end in a few days. So no worry is needed.

I just read the npjg’s blog yesterday. And found a lot of useful info through his blog which helps me understand a lot of concepts in director. By reading his blog, i’ve also get to know the history of how director is developed, and how those modules are created for.

I will keep reading those blogs. And sev recommend a book which contains the knowledge for understand how lingo worked. Also sev have gave me a link for a book which is about QuickDraw in macintosh, and that book is also mentioned by the blog of npjg. Looks like i really have a lot of thing to read and refer to. Although reading English books is a little hard for me(due to the very slow reading speed), i would still try my best to do it. Standing on the should of gaints, with those books and predecessors’ experiences, i’m very confident about the GSoC project.

Since those blogs are useful to me, then i will try to add some detailed info when i’m writing blogs for the future students.

A very brief summary for my past week work is fixing user interface in apartment. Fixing those widgets will help me getting closer to director.

I’ve added checkBox type for macbutton, allowing macbutton drawing different style marks for checkbox. We have already have the crosshair style mark, so it would be very easy to implement other style. I’ve found a common point when i was implementing checkBoxType. With the experience of fixing selEnd and selStart, i found those arrtibutes are movie-globally(scope is a movie). e.g. when you at a single movie, and when you are setting those attributes, all of the widgets will get affected. Thus i create a new function called updateGlobalAttr which will tell those widgets the real time global attr. And that could be optimized by telling widgets only when our global attributes are changed. We may get some same global attributes in the fuluture, thus we can simply extend this function to get the correct behaviour.

While i was checking the sprites size in every frame for finding a better solution to deal with sprites size. I found that even a stable widget which should the same size at every frame, those size of sprites will still change when changing the frame. And another interesting found is the first frame we saw that sprites has the correct size. Thus i just use the first sprites size to represent for all same cast members. with that, i was able to give a ok solution for temporary. But i know that won’t be the end, we still need to find some generic solutions for all those sprites, but not checking it everywhere.

And while i was fixing the zoombox, i creat screen for director which allows drawing the zoombox on it. With this, we can add more effects on screen just like how we are drawing the zoombox. But i think the way we are drawing zoombox still need to be refactored.

I’ve modified the mouse button event handling of director. Introducing two variables to it, one is for the hilite sprite, and the other is for the event sprite. Previously, if you hold the mouse button and move the cursor out of the bounds, that widget will invert incorrectly. I’ve checked how the mac director behaves, and bring the same behaviour to our director.

When dealing with the cursor, i found there is a interesting bug. if we are in the beam cursor surface, then macwindowmanager will force us to convert to the normal cursor, because it thought that beam cursor was caused by editable widget. Thus, i amend the wm to make it dealing with beam cursor more specifically. And we shall restore the cursor type when we are poping the cursor. Keeping the code logic follows it’s definition will always give us a good result.

I’ve amend the mactext for dealing with different encodeType. Previously, we just simply use UTF. And sev tolds me that there are plainByteMode for mactext. Thus i’ve modified the ctor of mactext, if we are passing the String instead of U32String, then we better specify the encodeType for the correct behaviour. For director, we are using kMacCentralEurope to encode.

I’ve created a test for testing our fonts handling. By reading the books of QuickDraw, i’ve amend the vertical offsets calculating for mactext by using ascent instead of text height to calc the offset. And that gives us a very correct behaviour when dealing with text with multi fonts and multi size.

while i was fixing the user interface, i’ve also fixed and implemented some lingo commands. And by fixing how we are dealing with the mask sprites. We are finally able to make the indicator work now. But with just one defect which is we are not able to update the text sprites immediatly when we are sliding the bar. I found how we are dealing with the setText for text castmember is we modified it’s text and set _modified to true. Then that castmember will create a new widget with correct text in next frame. I’ve added a widget pointer from castmember to widget. By creating a link from cast to widget, we are able to update widgets directly and immediately. Then we are able to bring a brand new indicators to director.

It is worth to mention that i didn’t handle that link to widget very correctly, and that crashed the buildbot. Today with sev and rvanlaar’s help, i managed to know how to check the buildbots. And i’ve fixed that problem by releasing the widget when we are changing the sprites. Because new widget and new castmember is coming, and we have to release the old one to free the slots.

Quite busy week. Still very happy when i’m coding and learning. Looking forward to the future work.

Hello. ayyg from Week 5 here about to talk about events from two (three?) weeks ago.

Disclaimer: The following will be a summary of the events. I will not detail everything, but rather convey the gist of the whole process.

During the first two weeks we mostly focused on getting the FTA2’s source code to compile under ScummVM and read their file contents through ScummVM’s OSystem. With that done, we have now opened up the doors for various developments to follow:

• Displaying images
• Receiving input
• Running scripts
• Playing audio

Among various other things. We will end up implementing all of them, as they are all necessary components of the final game, but our first milestone shall be this:

Displaying Graphics

Displaying images within ScummVM is simple. We mostly rely on OSystem::copyRectToScreen. This method takes in as parameters the image source data (as a pointer of bytes), the image pitch, it’s size and position.

So the question is: how do we get an image source data? You can go about this in two ways:

1. Fix each step of the event loop and one day you will (most likely) see something popping up.
2. Search through the source code for functions relating to graphics and observing its parameters.

Since we want to keep a tight eye on our progress, we want to make sure each system is working individually before proceeding on to tackle the event loop. Therefore, we chose the second option.

While I was searching for a possible image source data candidate, my mentor gave me a very much appreciated hint to look for window decorations. An example being autoMapDecorations, defined in the following way:

WindowDecoration autoMapDecorations[numAutoMapPanels] = {
	{ WindowDecoration(autoMapPanelRects[0], autoMapTopPanelResID) },
	{ WindowDecoration(autoMapPanelRects[1], autoMapMidPanelResID) },
	{ WindowDecoration(autoMapPanelRects[2], autoMapBotPanelResID) }
};

Looking at the code related to this variable, I found the following in automap.cpp:

pAutoMap->setDecorations(autoMapDecorations,
                         elementsof(autoMapDecorations),
                         decRes, 'M', 'A', 'P');

We may now look at what setDecorations does:

void DecoratedWindow::setDecorations(
    WindowDecoration *dec,
    int16           count,
    hResContext     *con) {
	int16           i;

	decorations = dec;
	numDecorations = count;

	//  For each "decorative panel" within the frame of the window
	for (i = 0; i < numDecorations; i++, dec++) {

		// request an image pointer from the imageCache

		dec->image = ImageCache.requestImage(con,
		                                     MKTAG('B', 'R', 'D', dec->imageNumber));
	}
}

Anything catch your eye here?

dec->image = ImageCache.requestImage(con, MKTAG('B', 'R', 'D', dec->imageNumber));

Aha, looks like we found our candidate for a image pointer.

Checking what requestImage does, we can see that it (basically) reduces to this call:

image = LoadResource(con, resID, "CImageNode Allocation");

So our first lead is to try loading this data with LoadResource.

For testing purposes, I introduced the testOpenImage function within main.cpp (and called it through the saga2 initialization method):

void testOpenImage() {
	hResContext     *decRes;

	decRes = resFile->newContext(MKTAG('A', 'M', 'A', 'P'), "Automap Resources");

	WindowDecoration *dec = &autoMapDecorations[1];
	dec->image = LoadResource(decRes, MKTAG('M', 'A', 'P', 1), "MAP1");
	Point16 pos(0, 0);
	drawCompressedImage(mainPort, pos, dec->image);
}

We then blitted it to the screen through drawCompressedImage in the following way:

map.data = (uint8 *)malloc(map.bytes());
if (map.data == NULL) return;
unpackImage(&map, map.size.x, map.size.y, hdr->data);

Graphics::Surface sur;
sur.create(map.size.x, map.size.y, Graphics::PixelFormat::createFormatCLUT8());
sur.setPixels(map.data);
sur.debugPrint();
g_system->copyRectToScreen(sur.getPixels(), sur.pitch, 0, 0, sur.w, sur.h);

However, you may notice the unpackImage in the code above. Sometimes images come packed (compressed), and we have to unpack them in order to display them correctly. The unpackImage routine was originally implemented in assembly, but it was also thankfully written in C.¹ Making use of that, we were able to finally see some progress:

You may be asking yourself about the big logo in the center. No, that’s not the image we’re trying to draw right now. It’s part of the loading screen that was implemented by my mentor. The thing on the top-left corner though? Now that’s progress.

And that’s the start of the journey, in my opinion. This time’s post was short, but to make up for that I’ll try to start posting more than once per week! (Who knows how that’s going to go…)

Next time I’ll continue talking about graphics. Particularly, drawing tiles.

1: In truth, before noticing that unpackImage had a implementation in C, we tried using SAGA’s version of that, not to much success.

Task 1: Fix the scrolling

The initial implementation of scrolling was really buggy, visible entries would become desynced at points, and wrong entries would be displayed. Scrolling far enough down would break the scrolling, and resizing was janky as well.

Other than scroll issues, also seen in above picture is that the items are always at constant distance from each other. So I went ahead and made the entries justify themselves in a row. UI related variables such as how close or far the columns can get, how big each entry is, what is the background color of thumbnail etc, will be exposed for user customisation through the Theme engine in due time. For now, these values remain hardcoded.

Also, the previous images/videos would not show this, but really the scrolling only worked when you scrolled on the “empty space” in the grid, as otherwise the scroll event was “caught” by the various component widgets of the items. This was also fixed.

Apart from logic changes in how scrolling works, and how the grid knows what games are visible, there were a few more things.

Interlude : The Invisible Change

I changed what the composition of a Grid Item is, so instead of containing child Widgets, the Item instead just draws the components on its own, which means that there is a potential to change the look of components like Thumbnails and Title without affecting other GraphicsWidgets or StaticTextWidgets. Of course, there is no change in how the items will look, I have just cut the bloat of creating 4 additional objects per grid item.

Task 2: Improving the look of items

Now that we have a bit more flexibility over how we draw our grid item, perhaps we can add a bit of flair to it.

The above image shows few additions :-

• A highlight on hovering over a grid item.
• Language display replaced by flags.
• A “tray” dialog that appears below the entry to perform the most common actions. Icons can later be placed instead of words.

In other news…

I got a new AC for my room .

Although I would prefer updating on every Tuesday, I figured that today (Sunday) is a good day because I need to think about next steps again, so why not spend time documenting instead.

Recent progress. Including but NOT limited to…..

– Learned how to modify previous commit with git
This includes editing the message, committed files, and files to commit. Awesome lesson by criezy. I use git rebase -i everyday now.
Other lessons on Github do apply.

– Where is the variable that defines if the game is on CD or floppy version
Kesha told me it was on dreamweb.h stored as a boolean. Because of this message, I not only can do work for dreamweb, but also, I knew that I had to find the variable in sci.cpp

– Functions that are used to draw text are usually cluster together
So I made some progress on Laurabow2 today. Yea!

– .h and .cpp files work together, so make sure to update them both
It should be obvious but …..


Lots to learn.

Finally some progress in bringing MacVenture engine back to life.

Progress :

1. Self Window is now displaying
2. offset problem inside some windows ( Inventory, Main, Commands) solved.
3. Command Window buttons are now working and not crashing

Problems yet to be solved, to make engine work as before :

1. Window positioning is slightly messed up (especially commands window)
2. I think offset inside Exit Window is kinda messed up
3. And a small task of resizing output console window

For the first week of GSoC, I don’t really remember what I did everyday, but here are a list of things that have been done though anyway:

Dreamweb intro functioning
The intro has proper TTS in most cases, we are fixing the way we do it, and adding support to Russian and Czech.
For other parts, the actions are already given TTS, so are most of the dialogues. The missing parts are objects, room names, and small parts of the dialogues

Laurabow2 intro progress
Most of the text are found in the code. I also have the game so we might consider working on it too….?

PR of Griffon Merged Merged
Started working on the April 1, merged around the first day of GSoC
Thank you criezy for merging and mentoring. Thank you also sev and Kesha for all the guidance, and also other community members for helping out. Yes, I will always remember this.

And this:

“Ever since I was a child…..”

This week I started wrapping up work on one of the last major Lingo features which was still unimplemented – chunk expressions.

Chunk expressions refer to chunks of text, such as char 3 of "test", word 1 to 10 of variable, or item 3 of "foo,bar,baz". These can be used to used to manipulate text in various ways, like highlighting chunks of text like in the screenshot above, deleting chunks of text, or inserting text before/into/after chunks of text.

After refactoring how references to text chunks work and implementing the hilite and delete commands, I ran into my old foe the Lingo parser again. Right now, the Lingo parser can’t differentiate between when we need something’s value (e.g. put char 1 of variable) and when we need a reference to something (e.g. hilite char 1 of variable). The current hack is to always push a reference and evaluate that reference later if needed. The parser’s gotten rather brittle, so it’s been difficult to change it to remove this hack… or to make any changes, really.

To solve this, I’ve started a major refactoring of the Lingo compiler which should make it a lot more flexible. This will allow references to be properly handled, and it should also allow us to eliminate most of the grammar’s plentiful hacks. I’ll be working on this most of this week, so more on this later!

Charm of OpenSource

hi hi, some simple talks here. Not really related to GSoC.

Few days ago, sev told me that some one fixed my previous work of debugflags.

I’m really surprising. I checked that PR, and got astonished that there are others helping me with those codes.

It’s really rare chance for me to see those codes which has been optimized based on my code.

Needless to say, it’s really a good chance to learn though those codes, and also with the reason about why to optimize.

Actually i can’t express my moods when i was seeing those thing. I’ve gained a lot though those days. With only 3 month, the abilities of coding, cooperating, expressing has improved a lot. And also there are goods friends who helped me a lot. Sev guided me from the very begining. He gived me a lot of advices about the coding, and most importantly, He taughed me how to be a good programmer. sluicebox and djsrv are helping with reviewing my codes and make improvments. There are also rvanlaar, rootfather and others. I want to be one of devs, and help to contribute to ScummVM.

I would like to say, this place is somewhere we can stick to with a very long period. We can know very skillful people here. We can gain happiness when we coding and creating something interesting. And we can learn knowledges and exercies here.