Skip to content

Conversation

@MohammedTaherMcW
Copy link

@MohammedTaherMcW MohammedTaherMcW commented Aug 24, 2025

Ticket

Link to JIRA Issue

Problem description

Added Performance and Accuracy Test for Gemma variants [1B, 4B, 27b]

What's changed

  • Added performance metrics support in simple_text_demo
  • Added performance metrics support in simple_vision_demo
  • Added Accuracy metrics support in test_accuracy
  • Added refpt file for Gemma variants [1B, 4B, 27b]
  • Added baseline perf metrics of top-1% and top-5% for Gemma variants [1B, 4B, 27b]
  • Added CI Test Support for Gemma variants [1B, 4B, 27b]

Checklist

@MohammedTaherMcW MohammedTaherMcW force-pushed the mcw/gemma_3_27b/pr_3_vLLM branch 2 times, most recently from d08eccc to 71eb2ec Compare September 2, 2025 17:12
@nikileshx nikileshx force-pushed the mcw/gemma_3_27b/pr_4_perf branch from 53ec75a to bdc6b6f Compare September 12, 2025 09:55
@MohammedTaherMcW MohammedTaherMcW changed the title Add Accuracy and Performance metrics for Gemma-3-27b-it Add Accuracy and Performance metrics for Gemma variants [1B, 4B, 27b] Sep 12, 2025
@MohammedTaherMcW MohammedTaherMcW force-pushed the mcw/gemma_3_27b/pr_3_vLLM branch from 71eb2ec to 28175ec Compare September 23, 2025 10:17
@nikileshx nikileshx force-pushed the mcw/gemma_3_27b/pr_4_perf branch from bdc6b6f to aa5c10c Compare September 23, 2025 11:00
@jennychristopher jennychristopher force-pushed the mcw/gemma_3_27b/pr_3_vLLM branch from 28175ec to c4efaa0 Compare September 24, 2025 12:31
@nikileshx nikileshx force-pushed the mcw/gemma_3_27b/pr_4_perf branch from 0cf2700 to c3de21c Compare September 24, 2025 15:26
@MohammedTaherMcW MohammedTaherMcW force-pushed the mcw/gemma_3_27b/pr_3_vLLM branch from c4efaa0 to 734ffa9 Compare October 14, 2025 10:25
@nikileshx nikileshx force-pushed the mcw/gemma_3_27b/pr_4_perf branch from 1b73ea3 to a1ff51b Compare October 15, 2025 05:54
@nikileshx nikileshx force-pushed the mcw/gemma_3_27b/pr_4_perf branch from a1ff51b to 5bf58c0 Compare October 16, 2025 04:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants