GoingToCamp Parks Canada does not identify available sites correctly

[jonoff]

Describe the bug

Going To Camp Parks Canada does not identify available sites correctly

Original Camply Command (with --debug)

camply --debug campsites   --provider goingtocamp   --rec-area 14   --campground -2147483648  --start-date 2023-09-01   --end-date 2023-09-02 --equipment-id -32768

Expected behavior
Sites are found and returned

Console Output (with --debug)

2023-08-10 00:26:29,171 [  CAMPLY]: camply, the campsite finder⛺️⛺
2023-08-10 00:26:29,172 [   DEBUG]: Setting up camply debugging
2023-08-10 00:26:29,172 [   DEBUG]: Camply Version: 0.29.0
2023-08-10 00:26:29,172 [   DEBUG]: Python Version: 3.9.17
2023-08-10 00:26:29,172 [   DEBUG]: Platform: linux
2023-08-10 00:26:29,174 [    INFO]: Using Camply Provider: "GoingToCamp"
2023-08-10 00:26:29,181 [    INFO]: 1 booking nights selected for search, ranging from 2023-09-01 to 2023-09-01
2023-08-10 00:26:29,184 [    INFO]: Retrieving Facility Information for Recreation Area ID: `14`.
2023-08-10 00:26:29,191 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:29,373 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/resourceLocation HTTP/1.1" 200 None
2023-08-10 00:26:29,442 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:29,761 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/maps HTTP/1.1" 200 None
2023-08-10 00:26:30,208 [    INFO]: 1 Matching Campgrounds Found
2023-08-10 00:26:30,209 [    INFO]: ⛰  Parks Canada (#14) - 🏕  Banff - Tunnel Mountain Village 2 (#-2147483648)
2023-08-10 00:26:30,228 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:30,418 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/availability/map?mapId=-2147483611&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768 HTTP/1.1" 200 None
2023-08-10 00:26:30,427 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:30,609 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/availability/map?mapId=-2147483610&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768 HTTP/1.1" 200 None
2023-08-10 00:26:30,630 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:30,750 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/availability/map?mapId=-2147483609&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768 HTTP/1.1" 200 None
2023-08-10 00:26:30,759 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:30,928 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/availability/map?mapId=-2147483608&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768 HTTP/1.1" 200 None
2023-08-10 00:26:30,938 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:31,049 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/availability/map?mapId=-2147483607&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768 HTTP/1.1" 200 None
2023-08-10 00:26:31,058 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:31,234 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/attribute/filterable HTTP/1.1" 200 None
2023-08-10 00:26:31,262 [   DEBUG]: Starting new HTTPS connection (1): reservation.pc.gc.ca:443
2023-08-10 00:26:31,382 [   DEBUG]: https://reservation.pc.gc.ca:443 "GET /api/resource/details?resourceId=-2147474297 HTTP/1.1" 200 None
2023-08-10 00:26:31,386 [    INFO]: ❌ ❌ ❌ ❌     0 Reservable Campsites Matching Search Preferences
2023-08-10 00:26:31,393 [  CAMPLY]: Exiting camply 👋

Additional context
Looking at the API return values: https://reservation.pc.gc.ca/api/availability/map?mapId=-2147483610&resourceLocationId=-2147483648&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&isReserving=True&getDailyAvailability=False&partySize=1&numEquipment=1&equipmentCategoryId=-32768&subEquipmentCategoryId=-32768

{"mapId":-2147483610,"mapAvailabilities":[5],"resourceAvailabilities":{"-2147474478":[{"availability":4,"remainingQuota":null}],"-2147474477":[{"availability":1,"remainingQuota":null}],"-2147474473":[{"availability":1,"remainingQuota":null}],"-2147474468":[{"availability":1,"remainingQuota":null}],"-2147474459":[{"availability":1,"remainingQuota":null}],"-2147474442":[{"availability":1,"remainingQuota":null}],"-2147474438":[{"availability":1,"remainingQuota":null}],"-2147474437":[{"availability":1,"remainingQuota":null}],"-2147474436":[{"availability":1,"remainingQuota":null}],"-2147474435":[{"availability":1,"remainingQuota":null}],"-2147474432":[{"availability":1,"remainingQuota":null}],"-2147474430":[{"availability":5,"remainingQuota":null}],"-2147474425":[{"availability":1,"remainingQuota":null}],"-2147474421":[{"availability":4,"remainingQuota":null}],"-2147474418":[{"availability":1,"remainingQuota":null}],"-2147474417":[{"availability":4,"remainingQuota":null}],"-2147474416":[{"availability":1,"remainingQuota":null}],"-2147474414":[{"availability":1,"remainingQuota":null}],"-2147474408":[{"availability":1,"remainingQuota":null}],"-2147474407":[{"availability":1,"remainingQuota":null}],"-2147474404":[{"availability":1,"remainingQuota":null}],"-2147474403":[{"availability":1,"remainingQuota":null}],"-2147474401":[{"availability":1,"remainingQuota":null}],"-2147474396":[{"availability":1,"remainingQuota":null}],"-2147474391":[{"availability":1,"remainingQuota":null}],"-2147474389":[{"availability":1,"remainingQuota":null}],"-2147474387":[{"availability":1,"remainingQuota":null}],"-2147474384":[{"availability":1,"remainingQuota":null}],"-2147474383":[{"availability":1,"remainingQuota":null}],"-2147474379":[{"availability":1,"remainingQuota":null}],"-2147474378":[{"availability":1,"remainingQuota":null}],"-2147474374":[{"availability":1,"remainingQuota":null}],"-2147474372":[{"availability":1,"remainingQuota":null}],"-2147474363":[{"availability":1,"remainingQuota":null}],"-2147474354":[{"availability":1,"remainingQuota":null}],"-2147474353":[{"availability":1,"remainingQuota":null}],"-2147474348":[{"availability":1,"remainingQuota":null}],"-2147474345":[{"availability":1,"remainingQuota":null}],"-2147474341":[{"availability":1,"remainingQuota":null}],"-2147474333":[{"availability":1,"remainingQuota":null}],"-2147474325":[{"availability":1,"remainingQuota":null}],"-2147474317":[{"availability":1,"remainingQuota":null}],"-2147474314":[{"availability":1,"remainingQuota":null}],"-2147474312":[{"availability":1,"remainingQuota":null}],"-2147474307":[{"availability":1,"remainingQuota":null}],"-2147474306":[{"availability":1,"remainingQuota":null}],"-2147474303":[{"availability":1,"remainingQuota":null}],"-2147474302":[{"availability":1,"remainingQuota":null}],"-2147474293":[{"availability":1,"remainingQuota":null}],"-2147474291":[{"availability":1,"remainingQuota":null}],"-2147474288":[{"availability":1,"remainingQuota":null}],"-2147474283":[{"availability":1,"remainingQuota":null}],"-2147474281":[{"availability":5,"remainingQuota":null}],"-2147474275":[{"availability":1,"remainingQuota":null}],"-2147474274":[{"availability":1,"remainingQuota":null}]},"mapLinkAvailabilities":{}}

The two sites marked "availability":5 are indeed available and lit up green in the web UI: https://reservation.pc.gc.ca/create-booking/results?mapId=-2147483610&searchTabGroupId=0&bookingCategoryId=0&startDate=2023-09-01&endDate=2023-09-02&nights=1&isReserving=true&equipmentId=-32768&subEquipmentId=-32761&partySize=1&filterData=%7B%22-32582%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%2C%22-32736%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%2C%22-32735%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%2C%22-32756%22%3A%22%5B%5B1%5D%2C0%2C1%2C0%5D%22%2C%22-32574%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%2C%22-32573%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%2C%22-32758%22%3A%22%5B%5B-1%5D%2C0%2C1%2C0%5D%22%7D&searchTime=2023-08-10T00%3A15%3A08.278&equipmentCapacity=1&resourceLocationId=-2147483648

[jonoff]

I'm guessing the dataset is different for this rec area, should there be an enum to describe availability around this line:

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Line 537 in 383617a

if availability_details[0]["availability"] == 0:

[jonoff]

A potentially unrelated issue, this rec area's equipment (https://reservation.pc.gc.ca/api/equipment) isn't displayed correctly, due to the equipment category at index 1 not 0:

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Lines 472 to 474 in 383617a

	# Only allow equipment from non-group equipment category (the 0th
	# element in results)
	for sub_category in results[0]["subEquipmentCategories"]:

[juftin]

Thanks for the great information @jonoff, this one looks to be a legit bug. Full disclosure, I didn't originally write the GoingToCamp provider so this will take me a bit of time to dig into and clean up. I'll follow back up once I've had some time to dig in

[juftin]

Okay. Following up here. There are two different issues that cause this campsite not to show. As of writing this comment there is only one campsite available @ Banff - Tunnel Mountain Village 2 - and that's spot A18, (AKA Site ID # -2147474345).

When we look at that's site availability, it comes back with 5

{
  "-2147474345": [
      {
        "availability": 5,
        "remainingQuota": null
      }
    ]
}

This line of code says it should be 0 to be considered available:

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Line 537 in 383617a

if availability_details[0]["availability"] == 0:

I also see a campsite with a 4 in there that's not available - so maybe 0 and 5 are the enum values that are acceptable? 🤷 I wonder how I could confirm this.

The next issue that this campsite returned - that ultimately prevented it from showing up as available was its minCapacity. See the campsite API response here:

{
  "maxStayIsAggregate": null,
  "minCapacity": null,
  "maxCapacity": 6,
  "mapId": -2147483610
}

And here is where we exclude that:

camply/camply/search/search_going_to_camp.py

Lines 174 to 180 in 383617a

	# Some rec areas have zero-capacity sites, which should not
	# be viable for camping. Skip all zero-capacity sites.
	if (
	not site_details["minCapacity"]
	or not site_details["maxCapacity"]
	):
	continue

---

So after allowing the enum values of 0 and 5 - and also allowing for sites where minCapacity is null - the issue is resolved and camply finds our one available campsite. I can open up a PR but I'm a bit uncertain if there will be any downstream consequences.

@acaloiaro might have have some more background on whether this was an actual zero capacity site. The sight seems to think it's legit:

And lol, that A18 campsite just got booked as I was finishing up writing this.

Here's the related code for above: https://github.com/juftin/camply/compare/fix/going_to_camp_missing_sites

[acaloiaro]

@juftin a big part of my process for figuring out the meaning of certain enums and values in the API was by seeing how the GTC's web interface presents those values.

If we continue down that path, we should assume that availability = 5 and minCapacity = null are considered available.

I was simply never presented with those values in my testing with specific GTC instnaces. I would consider adding a new constant for true availabilities, e.g. AVAILABLE_AVAILABILITIES = [0, 5] so we can check for availability_details[0]["availability"] in AVAILABLE_AVAILABILITIES (or something similar/naming things is hard).

Similar for minCapacity.

---

Speculation: I suspect that GTC has an admin interface that lets administrators create all manner of enums, and possibly even the semantics for those enums. E.g. in this case 5 probably has a nice availability name in that admin interface; perhaps something like A_CUSTOMER_WANTS_TO_RESERVE_BUT_HAS_NOT_YET_PAID. Such a status should be "available", but might be a way to keep things "on hold" when customers call to make reservations. Pure speculation, of course. But I do know through observation that there are many enums that are instance-specific. So either there's an admin interface that allows hosts to control some of the enums (and their semantics), or GTC sets these up on a per-instance basis during customer onboarding.

[juftin]

Ah, confirmed. These enums line up with the "Availability Legend"

I was able to find this snippet deep within the sites Javascript for https://reservation.pc.gc.ca/

  return (i = t || (t = {}))[i.Available = 0] = "Available",
  i[i.Unavailable = 1] = "Unavailable",
  i[i.NotOperating = 2] = "NotOperating",
  i[i.NonReservable = 3] = "NonReservable",
  i[i.Closed = 4] = "Closed",
  i[i.Invalid = 5] = "Invalid",
  i[i.InvalidBookingCategory = 6] = "InvalidBookingCategory",
  i[i.PartiallyAvailable = 7] = "PartiallyAvailable",
  i[i.Held = 8] = "Held",

---

---

Here's the same info for https://washington.goingtocamp.com/

return (i = t || (t = {}))[i.Available = 0] = "Available",
i[i.Unavailable = 1] = "Unavailable",
i[i.NotOperating = 2] = "NotOperating",
i[i.NonReservable = 3] = "NonReservable",
i[i.Closed = 4] = "Closed",
i[i.Invalid = 5] = "Invalid",
i[i.InvalidBookingCategory = 6] = "InvalidBookingCategory",
i[i.PartiallyAvailable = 7] = "PartiallyAvailable",
i[i.Held = 8] = "Held",

This all seems-ish to line up with those enums we're seeing across providers. It seems like we're currently doing the right thing - what do you think @acaloiaro / @jonoff? Should we be using enum values other than 0?

[acaloiaro]

It seems like we shouldn't use anything but 0 according to this code.

From jon's original issue:

The two sites marked "availability":5 are indeed available and lit up green in the web UI

This leaves me with one question. It showed up, but were they actually bookable? It's possible that was a frontend bug, and if the user tried booking, perhaps the backend would have failed, since 5 is supposed to be Invalid.

---

I found this comment in the search code and now I remember writing it

                    # Some rec areas have zero-capacity sites, which should not
                    # be viable for camping. Skip all zero-capacity sites.
                    if (
                        not site_details["minCapacity"]
                        or not site_details["maxCapacity"]
                    ):

I recall writing that because min/max capacity was coming back as 0 for some "sites", and those should almost certainly be filtered out. At the time, I recall thinking that min/maxCapacity = False must mean the same thing -- but that may not have been accurate. Hence, this code filters both min/maxCapacity = 0 and False.

I recall that some instances in CA list things like picnic pavilions as "sites" with a capacity of 0, so I think we should continue skipping those, but perhaps False deserves further investigation.

[jonoff]

This leaves me with one question. It showed up, but were they actually bookable? It's possible that was a frontend bug, and if the user tried booking, perhaps the backend would have failed, since 5 is supposed to be Invalid.

While I didn't book the above example sites, I was able to complete booking for a different site.
Here's a current example I randomly found:

Green site showing available via tooltip on site 723

availability for site 723:

"-2147475829":[{"availability":5,"remainingQuota":null}]

And since no sites are showing up as "0", I'm not sure we reached a conclusion for this part of the issue.

For the other part, this site has the same capacity details (https://reservation.pc.gc.ca/api/resource/details?resourceId=-2147475829):
minCapacity":null,"maxCapacity":6,

Another guess is that minCapacity is optional, and some admins only set max? If so, changing the OR into an AND could work for this line:
not site_details["minCapacity"] or not site_details["maxCapacity"]

I recall that some instances in CA list things like picnic pavilions as "sites" with a capacity of 0, so I think we should continue skipping those, but perhaps False deserves further investigation.

We could verify those types of sites set both max and min capacity to 0 vs. only one of the values (to distinguish 0/false from null).

[juftin]

So I think what's actually happening is an equipment mismatch. I realized that the 5 enum represents available sites that don't match the user's equipment selection

So take a look at this booking URL: https://reservation.pc.gc.ca/create-booking/results?mapId=-2147483624&bookingCategoryId=0&equipmentCategoryId=-32768&startDate=2023-09-07&endDate=2023-09-08&getDailyAvailability=false&isReserving=true&partySize=1

We can see that there are "Restricted" sites, but they're bookable. Here's the corresponding API Result:
https://reservation.pc.gc.ca/api/availability/map?mapId=-2147483624&bookingCategoryId=0&equipmentCategoryId=-32768&startDate=2023-09-07&endDate=2023-09-08&getDailyAvailability=false&isReserving=true&partySize=1

It has a number of 5 enum values because we're asking it to filter on equipmentCategoryId=-32768

When we remove that equipmentCategoryId param entirely from the endpoint, all of our 5s turn to 0s:
https://reservation.pc.gc.ca/api/availability/map?mapId=-2147483624&bookingCategoryId=0&startDate=2023-09-07&endDate=2023-09-08&getDailyAvailability=false&isReserving=true&partySize=1

Here's where we apply all those search params:

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Lines 506 to 518 in 383617a

	search_filter = {
	"mapId": campground.map_id,
	"resourceLocationId": campground.facility_id,
	"bookingCategoryId": 0,
	"startDate": start_date.isoformat(),
	"endDate": end_date.isoformat(),
	"isReserving": True,
	"getDailyAvailability": False,
	"partySize": 1,
	"numEquipment": 1,
	"equipmentCategoryId": NON_GROUP_EQUIPMENT,
	"filterData": [],
	}

I believe that we should only provide a equipmentCategoryId when the subEquipmentCategoryId is also specified. @jonoff in your example above the booking link subEquipmentCategoryId doesn't match the API subEquipmentCategoryId - if we remove them entirely from the API response our 5s turn to 0s

[jonoff]

Good find, "5" does look like it refers to site available but not matching current criteria. I wasn't making sure equipment and sub-equipment IDs were accurate in my API calls and when I originally tried to search I couldn't identify which ID(s) to pass due to #291 (comment), this line may not be the same for each rec-area

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Line 25 in 383617a

NON_GROUP_EQUIPMENT = -32768

and it seems subEquipmentCategoryId is not properly supported

camply/camply/providers/going_to_camp/going_to_camp_provider.py

Line 478 in 383617a

equipment_id = sub_category["subEquipmentCategoryId"]

[acaloiaro]

The equipment category/sub-category code is likely to be some of the shoddiest in the GTC implementation. I had a feeling it wouldn't be correct for 100% of GTC rec areas, but also didn't want to agonize over making it correct for 100% of rec areas at the time of implementation.

Equipment is another thing that can be defined on a rec-area by rec-area basis (or perhaps it's instance-wide i.e. per subdomain). An example of this is that one rec area might list "Travel Trailers up to 25'" as a category, but a different rec area might have "Travel Travelers up to 27'".

I made little effort to dig into the category/sub-category functionality since rec-areas can define their own categories. As a result, I can't say with any confidence what sub-equipment is.

But as @juftin found out, it's clear now that "availability" can be linked to equipment choices.

I'm happy to lend a better hand at some point in mid September. Currently I have a busy holiday schedule in Switzerland and all these mountain bike trails won't ride themselves :)