Skip to content

Conversation

@lukasnee
Copy link

@lukasnee lukasnee commented Mar 3, 2025

I found these bottlenecks while profiling my python application based on the min host using pyspeedscope. My application uploads firmware to a microcontroller and I was able to improve throughput by about 10% with these modifications.

Regarding commit perf(host): speed up bytes_to_hexstr up to 100x, here's quick benchmark test:

# test.py

import time

def bytes_to_hexstr_OLD(bytesequence: bytes) -> str:
    return "".join(f"{b:02X}" for b in bytesequence)

def bytes_to_hexstr_NEW(bytesequence: bytes) -> str:
    return bytesequence.hex().upper()

def timeit(f):
    start_time = time.time()
    f()
    end_time = time.time()
    return end_time - start_time

def test_case(bytesequence_size, run_count):
    input = bytes([i % 256 for i in range(bytesequence_size)])
    time1 = sum(timeit(lambda: bytes_to_hexstr_OLD(input))
                for _ in range(run_count)) / run_count
    time2 = sum(timeit(lambda: bytes_to_hexstr_NEW(input))
                for _ in range(run_count)) / run_count
    print(
        f"NEW vs. OLD {time1 / time2:.2f}x faster (bytesequence_size={bytesequence_size}, run_count={run_count})")

test_case(256, 1)
test_case(256, 10)
test_case(256, 100)
test_case(256, 1000)
test_case(256, 10000)
test_case(256*256, 1)
test_case(256*256, 10)
test_case(256*256, 100)
test_case(256*256, 1000)
python3 --version
Python 3.10.12
python3 test.py
NEW vs. OLD 4.44x faster (bytesequence_size=256, run_count=1)
NEW vs. OLD 63.05x faster (bytesequence_size=256, run_count=10)
NEW vs. OLD 60.77x faster (bytesequence_size=256, run_count=100)
NEW vs. OLD 77.76x faster (bytesequence_size=256, run_count=1000)
NEW vs. OLD 68.41x faster (bytesequence_size=256, run_count=10000)
NEW vs. OLD 127.16x faster (bytesequence_size=65536, run_count=1)
NEW vs. OLD 105.42x faster (bytesequence_size=65536, run_count=10)
NEW vs. OLD 122.96x faster (bytesequence_size=65536, run_count=100)
NEW vs. OLD 119.90x faster (bytesequence_size=65536, run_count=1000)

@lukasnee lukasnee changed the title Python host optimizations Python host optimizations (bytes_to_hexstr, log formatting, _now_ms) Mar 3, 2025
@lukasnee lukasnee force-pushed the python-host-optimizations branch from b834875 to 6f09169 Compare March 3, 2025 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant