feat: implement and test remove_duplicates from list

[NueloSE]

Overview

This PR implements a solution to the "Remove Duplicates from Sorted Array" problem using Cairo. The implementation provides a function that processes a sorted array and removes duplicate elements while maintaining the original order of unique elements.

Implementation Details

Approach

The implementation follows these key steps:

1. Initial Check: Handle the empty array edge case upfront.
2. Create Collection for Unique Elements: Since direct in-place manipulation isn't possible in Cairo, we create a temporary array to collect unique elements.
3. Single Pass Algorithm: We use a one-pass approach by iterating through the array and checking if the current element is different from the previously added unique element.
4. Result Preparation: After identifying all unique elements, we clear the original array and repopulate it with the unique elements.
5. Return Value: We return the count of unique elements.

This approach ensures O(n) time complexity, where n is the size of the input array.

Cairo-specific Considerations

Several aspects of Cairo required special attention:

1. Immutability Challenge: Unlike traditional languages, Cairo treats arrays as immutable data structures. This necessitated a different approach to the concept of "in-place" modification.
2. Array Clearing: In Cairo, we can't simply resize an array or set elements to null. Instead, we need to clear the array by popping all elements and then rebuilding it:

// Clear the original array by popping all elements
loop {
    match nums.pop_front() {
        Option::Some(_) => {},
        Option::None => { break; }
    };
}

3. Ownership and References: Working with references (ref) in Cairo requires careful consideration, especially when manipulating arrays.
4. Gas Optimization: The implementation prioritizes gas efficiency by minimizing operations, particularly by using a single pass through the array.

Differences Between In-place Modification in Traditional Languages vs. Cairo

The concept of "in-place" modification differs significantly between traditional languages and Cairo:
Traditional Languages (e.g., C++, Java, Python)
In traditional languages, in-place modifications typically:

• Directly overwrite values in memory
• Preserve the original array address/reference
• Can efficiently use a "two-pointer" technique where one pointer tracks the position for the next unique element
• Have O(1) space complexity (excluding the input array)

In Cairo:

• Arrays are fundamentally immutable
• "In-place" modification is simulated by recreating the array
• We can't directly overwrite array elements at specific indices
• Space complexity is O(k) where k is the number of unique elements (for the temporary array)

The approach in Cairo is functionally equivalent to in-place modification from the user's perspective but requires additional memory and operations to achieve.

Test Cases

The implementation includes comprehensive test cases covering:

1. Empty Array: Testing the edge case of an empty input array

2. No Duplicates: Testing arrays with all unique elements

3. Example Cases: Testing the examples provided in the requirements

4. All Duplicates: Testing an array where all elements are duplicates

5. Optimized Implementation: Parallel tests for the gas-optimized implementation

Each test verifies both the return value (number of unique elements) and the content of the modified array to ensure correctness.

Gas Efficiency Analysis

During testing, observed a significant difference in gas consumption between the standard and optimized implementations:

test_remove_duplicates_example2: ~131530 L2 gas
test_remove_duplicates_optimized_example2: ~109390 L2 gas

This represents approximately a 20% reduction in gas usage for the optimized implementation when processing the same input. The gas savings primarily come from:

• Reduced memory operations in the optimized approach
• More streamlined handling of array manipulation
• Elimination of redundant operations in the array processing flow

These gas savings could be significant for applications processing large arrays or making frequent calls to this function. While both implementations provide the same functionality and correctness guarantees, the optimized version should be preferred in gas-sensitive contexts.

Screenshots

Summary

This implementation successfully solves the "Remove Duplicates from Sorted Array" problem in Cairo while accounting for the language's functional and immutable paradigm. Although Cairo's constraints required a different approach than traditional languages, the solution maintains the expected behavior and performance characteristics.
The key achievement is simulating in-place modification in a language that doesn't natively support such operations, while maintaining clarity and gas efficiency.

Closes #214

[coxmars]

LGTM 🚀