This repository contains the analysis and findings of an A/B test conducted on the shopper hiring funnel. The goal of the test was to determine if initiating the required applicant background check earlier in the hiring process would improve conversion rates.
The dataset for this analysis can be downloaded here).
In this simulated dataset, we conducted an A/B test to evaluate the impact of initiating the background check earlier in the shopper hiring process. The hypothesis was that by completing the necessary prerequisites faster, shoppers would maintain momentum and be more likely to start quickly.
Key information about the test:
- Background check cost: $30
- Prerequisites: Activating a prepaid debit card and passing the background check
- Success criteria: Completion of the shopper's first batch of orders
Based on the analysis of the data, the following conclusions can be drawn:
-
Conversion Rate Improvement: The treatment group, where the background check was initiated earlier, exhibited a higher conversion rate compared to the control group. This suggests that initiating the background check earlier positively impacted shopper engagement.
-
Statistical Significance: Statistical tests were performed to determine the significance of the results. The p-value was below the significance level, indicating that the observed differences in conversion rates are unlikely due to chance.
-
Cost-effectiveness: To evaluate the cost-effectiveness of this change, we need to consider the incremental benefit of increased conversion rates against the cost of conducting the background check earlier. Further analysis is needed to determine the exact cost-effectiveness of the change.
Based on the findings, the following recommendations can be made:
-
Implement the Change: Initiate the background check earlier in the shopper hiring process as it has shown a positive impact on conversion rates. Monitor the long-term effects to assess sustainability.
-
Conduct Cost-Benefit Analysis: Perform a detailed cost-benefit analysis to understand the financial implications of conducting the background check earlier. Consider factors such as increased conversion rates, costs of the background check, and potential revenue gains.
-
Further Experimentation: Explore other areas for improvement in the shopper hiring funnel. Consider testing additional changes, such as optimizing the activation process for prepaid debit cards or evaluating the impact of completing the online orientation.
Please refer to the attached slides) for a summary of the findings and recommendations. The appendix contains additional content, including code snippets and workbooks used for the analysis.
Thank you for your interest in this A/B test analysis. Should you have any further questions or require additional information, please feel free to reach out to us.
