Scripting system

[culacant]

Redguard stores all (?) its map scripts in the RGM files.
Current state 22-7:

• script data can be read
• script logic can be ran with most "do-things" stubbed out
• task/function/multitask system that can run tasks on the object containing the scripts: implemented!
  • tasks are asynchronous, block further processing
  • functions are synchronous, should return immediately
  • multitasks are tasks ran in parallel to the main task, are not blocking
• family/group system implemented
• run task on named objects
• implement dictionary for objects; me, player and camera (a.o) are special-cases and should be accessible from any script.
  TODO:
• doublecheck script interpreter, might be missing/faulty/stubbed parts. Can be done on-the-go
  • known issue: string parameters dont work, they output as integers
• SOUPDEF tasks:
  • ongoing, progress tracked on github projects
  • for each task:
    • decode what the parameters are
    • implement behavior
  • known missing systems:
    - sound system
    - camera system
    - player controller
    - adjustments to unity lights system
    - shaders to switch textures

BACKLOG:

• we have functions that look like they could be used to write a character controller (key inputs, Walk* functions) but they are unused in the RGMs, are they hiding somewhere else?
• do tasks with coroutines so they dont block main thread

QRD on map scripts:

1. RGM files
   The RGM files are binary files that contain a bunch of level data for each level in redguard.
   Think static objects, scripted objects, animation stuff, even pathfinding node maps.
   More interestingly, the actual game logic for in-map objects is also encoded in these files. Think a lever being pulled doing something or a light turning on when you stand on a platform, that kinda stuff.

2. The scripts themselves
   The map scripts are in some binary format that has been (mostly) decoded already, turns out its written in a pretty simple language, you get if-statements, gotos, doing arithmetic on map-global and script-local variables and most importantly, calling functions that make the in-game objects do something. This last point is the biggest missing puzzle piece in making RG unity playable.

3. SOUPDEFS
   When the map script calls a function, it just refers to a number eg "function 05 with parameters 37,37". This would be hell to figure out, but luckily someone at bethesda decided to ship the SOUP386.DEF file with the game.
   This file contains plaintext names for all functions, map-global variables and script-local attributes. So instead of calling function 1 with parameter map variable 2, we know we want to call sRotate with as parameter "after_catacombs".

4. Where we're at with RGU
   In RGU, we can currently read the RGM files (a.o.) and parse them into C# classes to be used in unity.
   We can put the map objects in-engine in the correct positions, and link scripts to them by name.
   The scripts can be read and the control logic seems to work (mostly, i run into bugs there now and then), and there is a mechanism in place to call functions on the script objects.
   The only remaining work is implementing the actual functions and any systems they connect to, like the dialog systems (eg menuAddItem), player controller stuff (eg PlayerLineUp), camera system (eg showObj).

There are 367 functions in the SOUPDEF, i got 12 mostly working with some bugs, and about 50 that have their parameters deciphered (mostly the ones that take 0 parameters :D).

5. Quick orientation in the code
   I have been working on the observatory map in unity, and slowly implementing the game objects in there. This is in the Assets/Scenes/GameTest scene.

Looking at commit 4fdcfca, the implementation of the Light and LightOff functions, we can see the following:

• LoadWorld.cs:
  This is my test world loader, all scriptedObjects are in a big dictionary in ModelLoader.scriptedObjects, they have a unique ID read in from the RGM file. Scripts are default disabled, I turn them on for objects I'm currently working on, otherwise they spam the log with errors.
  You can also see the scriptstore global flag 195 being set, this maps to OB_TelH, the position the telescope in the observatory.
• RGScriptedObject.cs:
  This is the big ScriptedObject class, attached to each scripted map object.
  This contains an array of 367 function pointers, these will be called from the script. By default they map to the ones defined in soupdeffcn_nimpl, which just throws an error and returns 0.
  After implementing a function, you put it in the array and it will get called from the scripts, as you can see happen with the Light and LightOff functions.
  In the Light function, you can see I need to add a new unity object for the actual light, because it turns out the light can have a different position from the object itself, fun stuff.
• soupdeffcn_nimpl.cs:
  This file contains all functions in the soupdef stubbed out. They throw an error printing the name of the function and the values of the parameters, useful for debugging.
  I also put comments in there if I figure out what the parameters do and what I think the function should do.
• rgmtst and scriptstst.cs
  These are just small scripts I use to investigate stuff without having to be in unity, they are very useful sometimes but they need to be compiled separately, and I have no idea if this works for anyone else.

6. Additional details
   There are 3 types of functions in the soupdef, tasks, functions and multitasks
   Tasks take some time to complete and will block the scripts until they are complete, eg a moving platform will not do anything else until the move is complete.
   Functions return immediately, eg the playerstand function returns either 1 or 0 if the player is standing on something, no waiting involved
   Multitasks are tasks that can run in parallel, eg one of the observatory objects rotates while playing a sound.

A scripted object will run the script until a task is called, then it will block further execution until this task completes. Multitasks will run without blocking the script. Multitasks are usually blocked by a call to WaitOnTasks() later in the script to sync back to the script.
There are examples of tasks and functions already implemented, multitasks are just tasks with extra steps, and are also already implemented.

A script can call functions (or tasks or multitasks) on another object in the map. The same logic with functions/tasks/multitasks applies there, just on a different object. The objects that can be called from another object's script have names attached to them in the RGM (with exceptions to player, camera and self, which are a bit special), this is all taken care of in the ObjectStore and should just work ;).

[Draxinusom]

I believe I can assist with the family master/slave part.

These are set by the master and slave attributes (values 25 and 26) linked to the script objects in the RAAT section. They can get kinda messy as these can (and often do) work in chains and/or have multiple slaves at the same level that branch out into their own chains.

Fortunately, they don't get too deep (5 deep max), though there are some larger ones (up to 14 slaves), and they are setup correctly so you don't run into recursion issues.

Anyhow, the way these work is the master attribute is assigning that ID number to the record it's on and the slave attribute is the master value (assigned on another record) of that record's master. Both can be assigned on the same record (to different values).

So, as an example, record 123 in a map has the master attribute set to 50 and the slave attribute set to 0. That makes it the highest level master in whatever chain its in and what you should use as a starting point when building out the chain.

Next we'll say record 200 has a master value of 51 and slave value of 50. That makes it a slave of record 123 and the master of any records that have 51 set as its slave value.

From there, another record could also have 50 as its slave value, making record 123 have two direct slaves or could have 51 as its master, creating a chain of 3.

Here's a quick partial export from the RGM files that may illustrate it better:

MapID	RecordID	ScriptName	MasterID	SlaveID	Path
3	44	PI_CHND2	11	0	/044
3	45	PI_CHNDB	0	11	/044/045
3	48	PI_CHND1	10	0	/048
3	49	PI_CHNDA	0	10	/048/049
3	53	PI_PILA	20	0	/053
3	54	PI_PLTA	24	20	/053/054
3	55	PI_LVRA	0	24	/053/054/055
3	59	PI_POT2	2	0	/059
3	60	PI_POT1	0	2	/059/060
4	98	CV_CAGE5	10	0	/098
4	97	CV_CAGEC	0	10	/098/097
4	100	CV_CAGE3	11	10	/098/100
4	96	CV_CAGE4	0	11	/098/100/096
4	99	CV_TELE3	0	11	/098/100/099
5	46	OB_ORY01	101	0	/046
5	64	OB_ORY17	117	101	/046/064
5	62	OB_ORY14	114	117	/046/064/062
5	63	OB_ORY15	0	114	/046/064/062/063
5	65	OB_ORY11	111	101	/046/065
5	47	OB_ORY02	102	0	/047
5	48	OB_ORY04	0	102	/047/048
5	59	OB_ORY12	112	102	/047/059
5	57	OB_ORY13	113	112	/047/059/057
5	61	OB_ORY10	0	113	/047/059/057/061
5	60	OB_ORY03	103	102	/047/060
5	49	OB_ORY05	105	103	/047/060/049
5	69	OB_ORY16	0	105	/047/060/049/069
5	50	OB_ORY06	0	103	/047/060/050
5	51	OB_ORY08	0	103	/047/060/051
5	67	OB_ORY07	0	103	/047/060/067
5	68	OB_ORY09	0	103	/047/060/068

Separately from the master/slaves are groups (though top level masters are often part of a group). These are simpler, using only the attribute "group" (29) and setting the same value on all group members.

I'm not entirely clear the difference in how they function, though I suspect it's something like slaves only (always?) move when the master does, whereas groups are semi independent and are specifically synced as part of the object's script.

Anyways, hopefully this made some sense.

[culacant]

You're a day late and a dollar short on that one :D
Here's the orrery working, 100% scripted by the OG RGM scripts:
https://github.com/user-attachments/assets/89b87967-1860-40d0-9802-c052a354dcb1
You're spot on with the groups, there is a SyncWithGroup function that sets a syncpoint id, then waits until every group member hits that syncpoint.

[Draxinusom]

Bah, story of my life :P

Anyhow, that looks really good, nice work!

Not sure how helpful it'll be, but I've tossed some of my notes/extracts/etc up here. Based on the progress you're making I'm guessing it probably won't be all that useful but the RGM dump may be handy.

Drax