Fix KeyError: 'Slot' when running breadth command

[1234-ad]

Description

This PR fixes the KeyError: 'Slot' error that occurs when running the breadth command, addressing issue #34.

Problem Statement

When running:

python3 gyfe.py breadth --notp --year 3 --session 2023-2024 --semester AUTUMN

The script crashes with a KeyError: 'Slot' error. This happens because:

1. The code tries to access cells[5] without checking if the array has enough elements
2. None values in the unavailable_slots list cause issues in downstream processing
3. The find_all_unavailable_slots function doesn't handle None/empty values properly
4. Missing checks for table existence before processing

Root Causes Identified

1. Unsafe Array Access (Line 322-326)

# Before (unsafe)
course["Course Code"] = cells[0].text
try:
    course["Slot"] = cells[5].text
except Exception:
    course["Slot"] = None

2. None Values in unavailable_slots (Line 333)

# Before (includes None)
unavailable_slots = (
    df_all[df_all["Course Code"].isin(core_course_codes)]["Slot"].unique().tolist()
)

3. No None Handling in find_all_unavailable_slots

The function didn't filter out None/empty values before processing.

4. Missing Table Existence Check

No check if parentTable exists before calling find_all.

Changes Made

1. Added Proper Bounds Checking

# After (safe)
if len(cells) > 5:
    course = {}
    course["Course Code"] = cells[0].text.strip() if cells[0].text else ""
    course["Slot"] = cells[5].text.strip() if cells[5].text else None
    courses.append(course)

2. Filter None Values from unavailable_slots

# After (filters None)
unavailable_slots = (
    df_all[df_all["Course Code"].isin(core_course_codes)]["Slot"]
    .dropna()  # Remove None values
    .unique()
    .tolist()
)

3. Enhanced find_all_unavailable_slots

# Filter out None values and empty strings at the start
unavailable_slots = [slot for slot in unavailable_slots if slot]

# Skip empty strings during iteration
for slot in unavailable_slots:
    if not slot:  # Skip empty strings
        continue
    
    # Safe dictionary access
    if slot in overlaps:
        all_unavailable_slots.extend(overlaps[slot])

4. Added Table Existence Check

parentTable = soup.find("table", {"id": "disptab"})

if parentTable is None:
    print("Warning: Could not find course table. Proceeding without slot filtering.")
    unavailable_slots = []
else:
    # Process table...

5. Safe Filtering with Empty Check

# Only filter if we have unavailable slots
if all_unavailable_slots:
    df = df[~df["Slot"].str.contains("|".join(all_unavailable_slots), na=False)]

Benefits

✅ Crash Prevention: No more KeyError when cells array is shorter than expected
✅ None Handling: Properly filters out None values throughout the pipeline
✅ Graceful Degradation: Continues execution even if course table is missing
✅ Data Integrity: Strips whitespace and validates data before processing
✅ Better Error Messages: Warns users when tables are not found
✅ Robust Filtering: Handles edge cases in slot filtering logic

Testing Checklist

• Added bounds checking for all array accesses
• Filter None values using .dropna()
• Added None/empty string checks in helper functions
• Added table existence validation
• Added safe dictionary key access
• Maintained backward compatibility
• No breaking changes to existing functionality

Technical Details

Error Flow (Before):

1. Parse table rows → Some rows have < 6 cells
2. Try to access cells[5] → IndexError caught, set to None
3. Add course with Slot: None to list
4. Create DataFrame with None values
5. Extract unavailable_slots → List contains None
6. Call find_all_unavailable_slots(unavailable_slots) → Tries to iterate None
7. CRASH: KeyError or AttributeError

Fixed Flow (After):

1. Parse table rows → Check len(cells) > 5 first
2. Only process rows with enough cells
3. Strip text values to avoid empty strings
4. Create DataFrame with valid data only
5. Extract unavailable_slots → Use .dropna() to filter None
6. Call find_all_unavailable_slots(unavailable_slots) → Filter empty values first
7. SUCCESS: Clean processing with no errors

Example Output

Before (crashes):

KeyError: 'Slot'
Traceback (most recent call last):
  ...

After (works):

INFO:root: [SESSION STATUS]: New
Available electives saved to available_breadths.txt

Or if table is missing:

Warning: Could not find course table. Proceeding without slot filtering.
Available electives saved to available_breadths.txt

Related Issue

Closes #34

Additional Notes

• The fix is comprehensive and handles multiple edge cases
• All changes are defensive programming practices
• No external dependencies added
• Maintains existing functionality while adding robustness
• The warning message helps users understand when data might be incomplete