Field discovery only fetches 4 hardcoded issue types, missing Spike, Sub-task, and custom types

[itdove]

Problem Description

The JIRA field discovery mechanism in JiraFieldMapper.discover_fields() only fetches field metadata for 4 hardcoded issue types: ["Bug", "Story", "Task", "Epic"]. This results in incomplete available_for lists in field mappings, causing validation failures and requiring manual workarounds.

Current Behavior

File: devflow/jira/field_mapper.py:76

createmeta = self._fetch_createmeta(project_key, ["Bug", "Story", "Task", "Epic"])

Issue Types Missed:

• Spike (investigation/research tasks - common in Scrum)
• Sub-task (child issues)
• Improvement / Enhancement
• Any custom organization-specific issue types

Impact

1. Incomplete available_for Lists

If a field is available for both "Story" and "Spike", the discovered metadata will only show:

{
  "acceptance_criteria": {
    "available_for": ["Story", "Epic"],  // ❌ Missing "Spike"
    "required_for": ["Story"]
  }
}

2. Validation Failures

When users try to create a Spike with fields that ARE available for Spikes in JIRA:

$ daf jira create spike --summary "Test" --field acceptance_criteria="..."
✗ Validation failed:
  • Field 'acceptance_criteria' is not available for issue type 'Spike'.
    Available for: Story, Epic

3. Manual Workarounds Required

Users must manually override via enterprise.json:

{
  "backend_overrides": {
    "field_mappings": {
      "acceptance_criteria": {
        "available_for": ["Bug", "Story", "Task", "Epic", "Spike"]
      }
    }
  }
}

Root Cause

The _fetch_createmeta() method at line 218-309 actually DOES fetch all issue types from JIRA:

# Step 1: Get ALL issue types for the project
response = self.client._api_request(
    "GET",
    f"/rest/api/2/issue/createmeta/{project_key}/issuetypes"
)
all_issue_types = response.json().get("issueTypes", [])

# Step 2: Filter to ONLY requested types ⚠️ This is the problem
for issue_type in all_issue_types:
    if issue_type_name not in issue_types:  # ← Only process hardcoded 4 types
        continue

The API returns all issue types, but the code filters out everything except the 4 hardcoded types.

Expected Behavior

Field discovery should fetch metadata for ALL issue types available in the JIRA project, not just 4 hardcoded ones.

Proposed Solution

Option 1: Fetch All Issue Types (Recommended)

Modify discover_fields() to pass None instead of hardcoded list:

File: devflow/jira/field_mapper.py:76

# Before:
createmeta = self._fetch_createmeta(project_key, ["Bug", "Story", "Task", "Epic"])

# After:
createmeta = self._fetch_createmeta(project_key, issue_types=None)  # None = all types

Update _fetch_createmeta() to handle None:

File: devflow/jira/field_mapper.py:218-309

def _fetch_createmeta(self, project_key: str, issue_types: Optional[List[str]] = None) -> Dict:
    # Step 1: Get all issue types
    response = self.client._api_request(
        "GET",
        f"/rest/api/2/issue/createmeta/{project_key}/issuetypes"
    )
    all_issue_types = response.json().get("issueTypes", [])
    
    # Step 2: If no filter specified, use ALL types
    if issue_types is None:
        issue_types = [it["name"] for it in all_issue_types]
    
    # Step 3: Filter and fetch fields for requested types
    # ... rest of existing code ...

Option 2: Make It Configurable

Add a config option for organizations with many custom issue types:

File: organization.json

{
  "jira_discovery_issue_types": null,  // null = all types (default)
  // Or specify custom list:
  // "jira_discovery_issue_types": ["Bug", "Story", "Task", "Epic", "Spike", "Sub-task"]
}

Performance Considerations

Most JIRA projects have <10 issue types. Fetching metadata for all types instead of 4 adds minimal overhead:

• Current: 4 API calls (1 per type)
• Proposed: ~6-10 API calls (all types)

The field metadata is cached (24 hours by default), so discovery only runs once per day.

Files to Modify

1. devflow/jira/field_mapper.py:76 - Change hardcoded list to dynamic/configurable
2. devflow/jira/field_mapper.py:218 - Update _fetch_createmeta() signature to accept None
3. tests/test_field_mapper.py - Add tests for "fetch all types" behavior

Testing

Add test cases for:

• Discovery with issue_types=None fetches all types
• Discovery with Spike issue type includes Spike-specific fields
• Discovery with custom issue types works correctly
• Validation passes for fields available on non-standard issue types

References

• Red Hat ORGANIZATION.md shows Spike template (confirms Spike is used)
• AGENTS.md line 1663 shows validation error examples
• Field mapper code: devflow/jira/field_mapper.py:41-610
• Validation code: devflow/jira/validation.py:224-234

Priority

Medium-High - Affects users with Spike/Sub-task/custom issue types, requires manual workarounds

[itdove]

Pull request created: #362

[itdove]

✅ Session Complete

Project: devaiflow
Goal: #361: Field discovery only fetches 4 hardcoded issue types, missing Spike, Sub-task, and custom types
Time Spent:
2h 59m total

• dvernier: 2h 58m
• unknown: 0h 0m
  Branch: 361

Perfect! 🎉

The cleanup journey is complete:

✅ Started with: 5 field overrides (all incorrect or unnecessary)
✅ Ended with: backend_overrides: null (completely clean!)
✅ Removed: 54 lines of redundant configuration
✅ Result: Everything handled dynamically via DevAIFlow's built-in APIs

Key Discovery: You found that daf jira update already uses the editmeta API to dynamically discover editable fields - no manual overrides needed at all!

All changes pushed to:

• ✅ devaiflow (GitHub branch 361) - Core fix for field discovery
• ✅ devflow-for-red-hatters (GitLab main) - Clean enterprise.json template

The configuration is now minimal, accurate, and self-maintaining! 🚀

Generated by DevAIFlow