Ceiling Mode

[nanospork]

Context

The official NOLO software package created by LYRobotix allows users to attach a Nolo base station to the ceiling facing downward, rather than having it sit upright on a desk, chair, etc. From the ceiling, a Nolo setup is subject to reduced occlusion - that is, it is less likely for something to block the line-of-sight between the base station and your controller or headset, interrupting tracking. This generally provides for larger tracking areas and better 360-degree VR experiences.

Objectives

• Implement a "Ceiling Mode" in the NOFC so that users can attach their NOLO base station to the ceiling for reduced occlusion.

Tasking

• Attempt to generate a server config file that makes the appropriate coordinate system changes allowing for ceiling mode.
  • This should be all it takes - swapping and offsetting some axes, and possibly performing some rotations.
  • There is a chance that ceiling mode without changes to the firmware will impact the way the Nolo controllers report their yaw value. If this is the case, we will have contact LYRobotix for assistance.

How you can help!

Developing the ceiling mode configuration file will require the assistance of someone from the community who has a ceiling mode setup for their hardware, and a lot of trial and error. We will need to open an active line of communication (IM/Skype/etc.) so that I can suggest changes that you can test immediately. Please post in this thread if you are interested so that we can arrange this discussion.

[johnlajoie]

I'm willing to put some time into this. I will need a little bit to get set up - I'll need to get a ceiling bracket printed and take down a ceiling fan to get it set up first.

[johnlajoie]

So, I gave the ceiling configuration a quick try with a temporary setup this morning. This is really just a first step and it generated more questions than answers.

First off - here's my setup. I'm using NOFC 0.2 and the NOLO hardware is updated to the latest firmware. I started out with the NOLO base station on the desk in the "normal" orientation and verified everything worked as expected, and then moved the base station to the bottom of my ceiling fan, pointing down.

The HDK2 IR camera is located in the same place, on the wall facing towards the active area:

This brings me to my first question - I think that the HDK IR tracking is just used for head orientation, while the NOLO is used or location? This seems consistent with how things behave (see below) but I don't really know....

With the NOLO ceiling mounted I repeated the SteamVR room setup and entered SteamVR with a fairly reasonable view. From there I experimented with the movement of one of the controllers to see how it moved. In what I list below I'm using what I think is the standard OSVR coordinate system - y is vertical, x is right, z is pointing towards you.

Move NOLO controller in +x (in "real" space) ---> controller moves in -z in SteamVR
Move NOLO controller in +y (in "real" space) ---> controller moves in -x in SteamVR
Move NOLO controller in +z (in "real" space) ---> controller moves in +y in SteamVR

I moved the HMD as well, and the motion of the HMD seemed to be consistent with the motion of the controllers.

OK - so it does look like a change of coordinates systems will do the trick. I modified the OSVR server configuration file as follows:

        "/me/head": {
	    "child": {
		"changeBasis": {
		    "x" : "-z",
		    "y" : "x",
		    "z" : "y"
		}
	    },
	    "child": "/je_nourish_fusion/NoloFusedHead/tracker/0"
        },
        "/me/hands/left": {
	    "child": {
		"changeBasis": {
		    "x" : "-z",
		    "y" : "x",
		    "z" : "y"
		}
	    },
	    "child": "/org_osvr_filter_oneeuro/FilteredControllerLeft/tracker/0"
        },
        "/me/hands/right": {
	    "child": {
		"changeBasis": {
		    "x" : "-z",
		    "y" : "x",
		    "z" : "y"
		}
	    },
	    "child": "/org_osvr_filter_oneeuro/FilteredControllerRight/tracker/0"
        }

At this point I've pushed my ignorance about as far as I can - I think this is supposed to apply a coordinate transform to the tracking data for the device, but I may not have this correct. I fired things up with the new server config, but there was no change in the movement of the controllers.

I've got a NOLO ceiling bracket printing so I hope to set this up into a longer-term configuration in the near future and continue testing. Suggestions, pointers, etc. are all appreciated.

[nanospork]

The HDK camera is not used with NOFC v0.2 at all. Position information is obtained from the Nolo headset marker and rotation information is fused from both the Nolo headset marker and the HDK gyros.

Your config is on the right track. Can you put the full file on Pastebin or similar and share it? I think you have the correct transforms but perhaps the wrong aliases, or additional aliases may be necessary.

[johnlajoie]

Good to know - one more thing I can eliminate from my desk.

Here's a pastebin link to the full server config:

https://pastebin.com/L69pzhiR

[johnlajoie]

I've made a little more progress. To start with, I've cleaned up my setup and removed the HDK2 IR camera and the light kit from the ceiling fan so that the NOLO base station has an unobstructed view of the play area. Using osvr_print_server I found that indeed the changeBasis was not being applied and tracked that down to a syntax error in the server config file (the second child statement show be enclosed by the first). For simplicity I worked with just the headset ("/me/head") to see if I could get it working properly.

I am able to get translation along the axes working properly - for example, the configuration:

        "/me/head": {
	    "child": {
		"changeBasis": {
		    "x" : "x",
		    "y" : "-z",
		    "z" : "y"
		},
		"child": "/je_nourish_fusion/NoloFusedHead/tracker/0"
	    }
        },

has translation working properly within the FOV in the headset. That is to say, if I step forward or backwards, left or right, up or down the view changes appropriately. However, the FOV is rotated clockwise by about 90 degrees.

This seems to indicate that a coordinate transformation is also necessary for at least some of the yaw information as well?

[Conzar]

Awesome work. I was playing with

"rotate":{
                "axis": "x",
                "degrees": 180 
            },  

but not having much luck. What do you mean that the FOV is rotated clockwise by 90'degrees?

[johnlajoie]

The field of view in the HMD is rotated, so that "up" is to the right. The field of view (FOV) does change as I rotate the headset. In experimenting with the changeBasis for "/me/head" I found that the FOV would have some very odd rotations and would not move as expected when I rotated my head (which was all very nauseating). It seems like I need to fix a transform for the yaw first, and then get the translation axes set for the head and controllers, so I suspect what I listed above will have to change.

I see that in the configuration file the yaw already has a rotation:

	"/NOFC/head/yaw": "/hdkYawSpace",
	
	"/hdkYawSpace": {
		"rotate": {
			"axis": "y",
			"degrees": -180
		},
		"child": "/com_osvr_Multiserver/OSVRHackerDevKit0/semantic/hmd"
        },

I tried added the same changeBasis to hdkYawSpace but this didn't work, I ended up with the FOV at an angle and it looked like all the coordinates were mixed?

I have the feeling that if I understood the coordinate systems used in OSVR for location and yaw that this really isn't all that complicated.

[johnlajoie]

A bit more progress today....

Basically, I have a group of settings that work correctly for tracking both the headset and the controllers, without the FOV issues I describe above. Here's a Pastbin to the full config file:

https://pastebin.com/PVUfdtVv

And here's the relevant transformation for /me/head (the controllers are similar):

        "/me/head": {
	    "child": {
		"changeBasis": {
		    "x" : "x",
		    "y" : "-z",
		    "z" : "y"
		},
		"child": "/je_nourish_fusion/NoloFusedHead/tracker/0"
	    }
        },

This is not what I would have expected given the OSVR coordinate system, if what I was transforming was the OSVR coordinate system for the tracking devices. I think that might also explain why a simple rotation around the x axis didn't work either, but it's not clear to me exactly what coordinate system I am transforming with this.

In any case, by trial and error, this works - for the tracked location of the headset and controllers. There are still some oddities with positioning/orientation that need to be sorted out:

1. The headset pitch is still correct, but yaw and roll seem swapped or mixed (really hard to tell which without making yourself want to throw up). A yaw movement doesn't change the view as you would expect, but instead it rotates about the z-axis (into the screen). It's hard to do a true roll without a little yaw so that's less clear.

2. The controller orientations are mixed up. For example, pointing the controller to the left angles it to the right. Up and down (pitch) seems OK, consistent with how the headset behaves.

I wouldn't think that the headset orientation would be affected at all, same for the controllers? It's possible that I arrived at something with changeBasis that seems to work but isn't really doing what I think it is, which is very possible given that I don't understand to transformation that was necessary to make the tracking work in the first place.

[lonetech]

The changeBasis describes a 90 degree rotation about X of the entire coordinate space. That should be correct in principle. The problem is likely to be in how the Nolo tracks orientation; every tracked device has its own idea, and they normally just assume they're pointed forward when zeroed. Given this behaviour, rotating the orientation along with the position is the wrong thing to do. Best case, we only want the transform you describe on position, nothing else. At least the Nolo reports orientations in quaternion format; pitch/yaw/roll is much harder to deal with, with their order dependency and gimbal locks.

[johnlajoie]

For the headset I tried things like:

        "/NOFC/head/roll": "/hdkRollSpace",
	"/NOFC/head/pitch": "/hdkPitchSpace", 
	"/hdkRollSpace": {
		"postrotate": {
			"axis": "x",
			"degrees": 90
		},
		"child": "/com_osvr_Multiserver/OSVRHackerDevKit0/semantic/hmd"
        },
	"/hdkPitchSpace": {
		"postrotate": {
			"axis": "x",
			"degrees": 90
		},
		"child": "/com_osvr_Multiserver/OSVRHackerDevKit0/semantic/hmd"
        },

and this did decouple the roll and yaw for the headset, but I always ended up staring at the floor in SteamVR Home. All the translations (tracked positions) were still correct (as expected), the orientation was just off. It works great if you look up at the ceiling.

Is there a way, through the server configuration file, to apply transforms to the tracking and orientation information separately? Or would this have to be done at the driver level?

Or, could transformations be done separately by defining aliases for (for example) "/me/hands/left/position" and "/me/hands/left/orientation/roll", etc.?

[johnlajoie]

Thanks @lonetech, your suggestion that all we want to transform was the position data led me to the solution. Here's a Pastebin with a working configuration file:

https://pastebin.com/iZAGMdaX

The trick was to define an alias for the position data that included the transform, and use that in the driver section, instead of a global transformation on "/me/head" or "/me/hands/left". So, for example, for the head tracking I added an alias:

        "/NOFC/CeilingHeadTracker": {
	    "child": {
		"changeBasis": {
		    "x" : "x",
		    "y" : "-z",
		    "z" : "y"
		},
		"child": "/com_osvr_Nolo/LYRobotix Nolo/tracker/1"
	    }
	},

and then in the driver declaration above for "NoloFusedHead" I made the change:

		//"position": "/com_osvr_Nolo/LYRobotix Nolo/tracker/1",
		"position": "/NOFC/CeilingHeadTracker",

Similar changes were made for the controllers. This works beautifully - the orientation is untouched (and correct) and everything works exactly as you would expect.

I must say, with the NOLO base station mounted on the ceiling the tracking is very clean and smooth and the ability to do a full 360 is really nice. At the height I have the base station mounted (about 2ft above my head) I only have a few feet of movement forward/backward/left/right but it's really quite nice. I may look into removing the ceiling fan fully and mounting it a bit higher.

All things considered, this breathes new life into my NOLO + HDK2 setup. So far I have only played around in SteamVR home, but over the next few days I plan to try a few games - I'll let you know how it goes.

[nanospork]

Well done @johnlajoie! Glad to see the success of another community member’s involvement!

Next step is to make sure the config file is squeaky clean (make sure names and format seem consistent with the existing config) then submit a pull request.

Also, I would encourage you to post about this on Reddit, if you so desire. Get those who have been itching for ceiling mode a temporary solution until v0.3 is ready.