Skip to content

sys/luid: improve luid_base() algorithm#15239

Merged
benpicco merged 1 commit intoRIOT-OS:masterfrom
benpicco:sys/luid-tweak
Oct 19, 2020
Merged

sys/luid: improve luid_base() algorithm#15239
benpicco merged 1 commit intoRIOT-OS:masterfrom
benpicco:sys/luid-tweak

Conversation

@benpicco
Copy link
Contributor

@benpicco benpicco commented Oct 18, 2020

Contribution description

CPU IDs may contain very little entropy. Often only a single byte may change between two CPUs.

To mitigate this use a somewhat 'proper' hash algorithm (I chose DJB2 hash simply because it's small) and 'extend' it to the required output size.

For 50 STM32 CPU IDs from the grenoble site, this produces a much more random LUID output.
Before especially the last bytes (used as short address) would often be the same for different nodes.

Testing procedure

For 50 iotlab-m3 nodes, this generates much more evenly distributed (64 bit) LUIDs:

# CPU ID -> luid-64
32 ff d4 05 33 48 32 36 16 62 02 43 -> 5d 5f c0 8b 28 db 6e 13 
32 ff d6 05 33 48 32 36 07 67 03 43 -> 7b fc 7f 35 b1 8a 1f 2e 
32 ff d6 05 33 48 32 36 07 71 03 43 -> 7b f4 2d 4d d3 ec fd 40 
32 ff d6 05 33 48 32 36 09 81 03 43 -> 67 52 e9 6b a3 3c ad 70 
32 ff d6 05 33 48 32 36 11 81 03 43 -> a7 da e9 93 b3 cc 9d 60 
32 ff d6 05 33 48 32 36 14 76 03 43 -> 5d a7 34 86 b0 a9 5e 4d 
32 ff d6 05 33 48 32 36 25 78 03 43 -> bf b2 ca f3 7a 23 94 b7 
32 ff d7 05 33 48 32 36 10 62 02 43 -> a7 9a 28 91 74 8e 20 a4 
32 ff d7 05 33 48 32 36 12 77 03 43 -> ab c8 5d 92 0b 35 cb fb 
32 ff d7 05 33 48 32 36 21 71 03 43 -> a5 8b d9 e5 59 03 75 69 
32 ff d7 05 33 48 32 36 22 71 03 43 -> ab 58 6f f6 69 13 65 99 
32 ff d7 05 33 48 32 36 28 78 03 43 -> a7 6e 52 04 72 4c a2 62 
32 ff d8 05 33 48 32 36 05 72 03 43 -> 7b 74 f4 6f 94 33 82 e3 
32 ff d8 05 33 48 32 36 09 69 03 43 -> 7b f0 35 6f f3 4e 25 ae 
32 ff d8 05 33 48 32 36 11 77 03 43 -> ab 08 c3 77 cd 58 7b 18 
32 ff d8 05 33 48 32 36 23 82 03 43 -> bf ce 00 d5 ac 5b 9a db 
32 ff d9 05 33 48 32 36 04 77 03 43 -> fb dc ed 8c 23 df 57 1d 
32 ff d9 05 33 48 32 36 13 82 03 43 -> dd e5 5c 81 4c 98 34 1e 
32 ff d9 05 33 48 32 36 18 81 03 43 -> db c0 5b a4 ed 59 15 1f 
32 ff d9 05 33 48 32 36 24 71 03 43 -> bb 54 73 50 dd 49 05 6f 
32 ff d9 05 33 48 32 36 28 81 03 43 -> bb d0 5b 74 8d b9 35 3f 
32 ff d9 05 33 48 32 36 32 69 03 43 -> 5f 6e 7b 9e 51 4d 81 63 
32 ff da 05 33 48 32 36 04 71 03 43 -> 65 5d eb 28 53 f8 0d 7c 
32 ff da 05 33 48 32 36 11 72 03 43 -> 5f de 30 03 d4 41 52 55 
32 ff da 05 33 48 32 36 16 72 03 43 -> a5 a1 f4 a4 54 c1 d2 d5 
32 ff da 05 33 48 32 36 17 72 03 43 -> 5b 80 3c 4d ec 49 5a 4d 
32 ff da 05 33 48 32 36 20 71 03 43 -> bd f1 93 5c 5b f0 05 04 
32 ff da 05 33 48 32 36 22 62 02 43 -> bd 55 e4 17 00 ed 7e 21 
32 ff da 05 33 48 32 36 24 68 03 43 -> a5 0f 86 f2 12 df 0c 53 
32 ff da 05 33 48 32 36 24 69 03 43 -> a5 0d a3 e8 4b c0 f5 14 
32 ff da 05 33 48 32 36 26 76 03 43 -> a5 d1 30 cc a0 0d 1e 81 
32 ff da 05 33 48 32 36 27 79 03 43 -> bb 70 ed a5 93 b8 4d bc 
32 ff da 05 33 48 32 36 28 79 03 43 -> ed 89 5b b4 03 08 bd cc 
32 ff da 05 33 48 32 36 29 60 02 43 -> ef ba ae 1e b2 7f ac 33 
32 ff da 05 33 48 32 36 35 76 03 43 -> e7 e2 fc 97 38 b5 a6 79 
32 ff da 05 33 48 32 36 39 60 02 43 -> ef 8a 8e 4e 72 bf 6c f3 
32 ff da 05 33 48 32 36 39 76 03 43 -> ef 8e 8c 2b 10 9d 8e 11 
32 ff db 05 33 48 32 36 09 72 03 43 -> ad 1d 64 ff 90 fe 8c 3c 
32 ff db 05 33 48 32 36 21 80 03 43 -> bd 6d 9e 53 ba 10 22 66 
32 ff db 05 33 48 32 36 24 75 03 43 -> a7 72 9b c4 59 97 05 a5 
32 ff db 05 33 48 32 36 25 75 03 43 -> a5 93 5d 69 c5 93 81 a1 
32 ff db 05 33 48 32 36 27 75 03 43 -> 6d 79 49 9f b1 bf ed bd 
32 ff db 05 33 48 32 36 30 78 03 43 -> 5f 22 3a 0c a2 28 1a 6e 
32 ff dc 05 33 48 32 36 13 76 03 43 -> a7 42 bc 75 f8 b3 ee 7b 
32 ff dc 05 33 48 32 36 14 62 02 43 -> a5 61 e4 e9 18 d3 0e 5b 
32 ff dc 05 33 48 32 36 17 69 03 43 -> af 76 09 ad ab 62 dd 6a 
32 ff dc 05 33 48 32 36 18 82 03 43 -> ad cd b4 8e 20 cb 26 43 
32 ff dc 05 33 48 32 36 26 71 03 43 -> ed 39 47 e6 e7 2e 21 a6 
32 ff dd 05 33 48 32 36 20 72 03 43 -> bb 54 0c 1c 34 9c b4 12 
32 ff df 05 33 48 32 36 42 76 03 43 -> 7b 90 b0 a2 20 d2 2c 58 
source code
#include <stdio.h>
#include <string.h>
#include <stdint.h>

#define ARRAY_SIZE(a) (sizeof((a)) / sizeof((a)[0]))

#define CPUID_LEN   12

static const uint8_t cpu_ids[][CPUID_LEN] = {
    { 0x32, 0xff, 0xd4, 0x05, 0x33, 0x48, 0x32, 0x36, 0x16, 0x62, 0x02, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x07, 0x67, 0x03, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x07, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x09, 0x81, 0x03, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x11, 0x81, 0x03, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x14, 0x76, 0x03, 0x43 },
    { 0x32, 0xff, 0xd6, 0x05, 0x33, 0x48, 0x32, 0x36, 0x25, 0x78, 0x03, 0x43 },
    { 0x32, 0xff, 0xd7, 0x05, 0x33, 0x48, 0x32, 0x36, 0x10, 0x62, 0x02, 0x43 },
    { 0x32, 0xff, 0xd7, 0x05, 0x33, 0x48, 0x32, 0x36, 0x12, 0x77, 0x03, 0x43 },
    { 0x32, 0xff, 0xd7, 0x05, 0x33, 0x48, 0x32, 0x36, 0x21, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xd7, 0x05, 0x33, 0x48, 0x32, 0x36, 0x22, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xd7, 0x05, 0x33, 0x48, 0x32, 0x36, 0x28, 0x78, 0x03, 0x43 },
    { 0x32, 0xff, 0xd8, 0x05, 0x33, 0x48, 0x32, 0x36, 0x05, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xd8, 0x05, 0x33, 0x48, 0x32, 0x36, 0x09, 0x69, 0x03, 0x43 },
    { 0x32, 0xff, 0xd8, 0x05, 0x33, 0x48, 0x32, 0x36, 0x11, 0x77, 0x03, 0x43 },
    { 0x32, 0xff, 0xd8, 0x05, 0x33, 0x48, 0x32, 0x36, 0x23, 0x82, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x04, 0x77, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x13, 0x82, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x18, 0x81, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x24, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x28, 0x81, 0x03, 0x43 },
    { 0x32, 0xff, 0xd9, 0x05, 0x33, 0x48, 0x32, 0x36, 0x32, 0x69, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x04, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x11, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x16, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x17, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x20, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x22, 0x62, 0x02, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x24, 0x68, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x24, 0x69, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x26, 0x76, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x27, 0x79, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x28, 0x79, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x29, 0x60, 0x02, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x35, 0x76, 0x03, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x39, 0x60, 0x02, 0x43 },
    { 0x32, 0xff, 0xda, 0x05, 0x33, 0x48, 0x32, 0x36, 0x39, 0x76, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x09, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x21, 0x80, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x24, 0x75, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x25, 0x75, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x27, 0x75, 0x03, 0x43 },
    { 0x32, 0xff, 0xdb, 0x05, 0x33, 0x48, 0x32, 0x36, 0x30, 0x78, 0x03, 0x43 },
    { 0x32, 0xff, 0xdc, 0x05, 0x33, 0x48, 0x32, 0x36, 0x13, 0x76, 0x03, 0x43 },
    { 0x32, 0xff, 0xdc, 0x05, 0x33, 0x48, 0x32, 0x36, 0x14, 0x62, 0x02, 0x43 },
    { 0x32, 0xff, 0xdc, 0x05, 0x33, 0x48, 0x32, 0x36, 0x17, 0x69, 0x03, 0x43 },
    { 0x32, 0xff, 0xdc, 0x05, 0x33, 0x48, 0x32, 0x36, 0x18, 0x82, 0x03, 0x43 },
    { 0x32, 0xff, 0xdc, 0x05, 0x33, 0x48, 0x32, 0x36, 0x26, 0x71, 0x03, 0x43 },
    { 0x32, 0xff, 0xdd, 0x05, 0x33, 0x48, 0x32, 0x36, 0x20, 0x72, 0x03, 0x43 },
    { 0x32, 0xff, 0xdf, 0x05, 0x33, 0x48, 0x32, 0x36, 0x42, 0x76, 0x03, 0x43 },
};

static void _luid_round(const uint8_t *in, size_t in_len, uint8_t *out, size_t out_len)
{
    union {
        uint32_t u32;
        uint8_t u8[4];
    } hash;

    hash.u32 = 5381;

    for (size_t i = 0; i < in_len; i++) {
        hash.u32 = hash.u32 * 33 + in[i];
        out[i % out_len] ^= hash.u8[0];
    }
}

void luid_base(const uint8_t *cid, size_t cpuid_len, void *buf, size_t len)
{
    memset(buf, 0x23, len);
    _luid_round(cid, cpuid_len, buf, len);
    _luid_round(buf, len, buf, len);
}

int main(void) {

    for (unsigned i = 0; i < ARRAY_SIZE(cpu_ids); ++i) {
        uint8_t eui[8];
        luid_base(cpu_ids[i], CPUID_LEN, eui, sizeof(eui));

        for (int j = 0; j < CPUID_LEN; ++j) {
            printf("%02x ", cpu_ids[i][j]);
        }

        printf("-> ");

        for (int j = 0; j < sizeof(eui); ++j) {
            printf("%02x ", eui[j]);
        }
        printf("\n");
    }

    return 0;
}
before
32 ff d4 05 33 48 32 36 16 62 02 43 -> e1 09 24 56 dc 10 11 11 
32 ff d6 05 33 48 32 36 07 67 03 43 -> df 52 72 21 10 6b f5 11 
32 ff d6 05 33 48 32 36 07 71 03 43 -> 64 df 21 52 10 6b f5 11 
32 ff d6 05 33 48 32 36 09 81 03 43 -> df 52 94 2f 10 6b f5 11 
32 ff d6 05 33 48 32 36 11 81 03 43 -> 94 52 df 37 10 6b f5 11 
32 ff d6 05 33 48 32 36 14 76 03 43 -> 63 32 df 52 10 6b f5 11 
32 ff d6 05 33 48 32 36 25 78 03 43 -> 03 6d 52 df 10 6b f5 11 
32 ff d7 05 33 48 32 36 10 62 02 43 -> 13 53 05 be 6b 26 f4 11 
32 ff d7 05 33 48 32 36 12 77 03 43 -> 12 07 ab 53 6b 26 f4 11 
32 ff d7 05 33 48 32 36 21 71 03 43 -> 34 ad 53 12 6b 26 f4 11 
32 ff d7 05 33 48 32 36 22 71 03 43 -> ad 53 12 37 6b 26 f4 11 
32 ff d7 05 33 48 32 36 28 78 03 43 -> 3d a4 53 12 6b 26 f4 11 
32 ff d8 05 33 48 32 36 05 72 03 43 -> 9f 14 25 89 11 6b 10 15 
32 ff d8 05 33 48 32 36 09 69 03 43 -> 25 9f 18 92 11 6b 10 15 
32 ff d8 05 33 48 32 36 11 77 03 43 -> 25 8c 00 9f 11 6b 10 15 
32 ff d8 05 33 48 32 36 23 82 03 43 -> 79 25 9f 32 11 6b 10 15 
32 ff d9 05 33 48 32 36 04 77 03 43 -> 12 51 9f fe 6b 10 15 11 
32 ff d9 05 33 48 32 36 13 82 03 43 -> 9f a4 12 e9 6b 10 15 11 
32 ff d9 05 33 48 32 36 18 81 03 43 -> 9f 12 a7 e2 6b 10 15 11 
32 ff d9 05 33 48 32 36 24 71 03 43 -> 9f 57 12 de 6b 10 15 11 
32 ff d9 05 33 48 32 36 28 81 03 43 -> 9f a7 d2 12 6b 10 15 11 
32 ff d9 05 33 48 32 36 32 69 03 43 -> c8 4f 9f 12 6b 10 15 11 
32 ff da 05 33 48 32 36 04 71 03 43 -> ad fa 28 22 11 15 10 11 
32 ff da 05 33 48 32 36 11 72 03 43 -> 28 37 fa ae 11 15 10 11 
32 ff da 05 33 48 32 36 16 72 03 43 -> 30 28 ae fa 11 15 10 11 
32 ff da 05 33 48 32 36 17 72 03 43 -> ae 31 28 fa 11 15 10 11 
32 ff da 05 33 48 32 36 20 71 03 43 -> fa 06 28 ad 11 15 10 11 
32 ff da 05 33 48 32 36 22 62 02 43 -> 04 fb 28 be 11 15 10 11 
32 ff da 05 33 48 32 36 24 68 03 43 -> fa 28 02 b4 11 15 10 11 
32 ff da 05 33 48 32 36 24 69 03 43 -> b5 fa 28 02 11 15 10 11 
32 ff da 05 33 48 32 36 26 76 03 43 -> fa 28 00 aa 11 15 10 11 
32 ff da 05 33 48 32 36 27 79 03 43 -> 01 28 fa a5 11 15 10 11 
32 ff da 05 33 48 32 36 28 79 03 43 -> fa 0e 28 a5 11 15 10 11 
32 ff da 05 33 48 32 36 29 60 02 43 -> bc fb 0f 28 11 15 10 11 
32 ff da 05 33 48 32 36 35 76 03 43 -> 28 fa 13 aa 11 15 10 11 
32 ff da 05 33 48 32 36 39 60 02 43 -> bc 1f fb 28 11 15 10 11 
32 ff da 05 33 48 32 36 39 76 03 43 -> 28 fa 1f aa 11 15 10 11 
32 ff db 05 33 48 32 36 09 72 03 43 -> 8a 18 68 65 15 10 dc 11 
32 ff db 05 33 48 32 36 21 80 03 43 -> 78 65 68 30 15 10 dc 11 
32 ff db 05 33 48 32 36 24 75 03 43 -> 8d 68 35 65 15 10 dc 11 
32 ff db 05 33 48 32 36 25 75 03 43 -> 68 8d 65 34 15 10 dc 11 
32 ff db 05 33 48 32 36 27 75 03 43 -> 65 8d 68 36 15 10 dc 11 
32 ff db 05 33 48 32 36 30 78 03 43 -> 21 68 65 80 15 10 dc 11 
32 ff dc 05 33 48 32 36 13 76 03 43 -> 25 aa 56 02 11 ff 10 6b 
32 ff dc 05 33 48 32 36 14 62 02 43 -> 56 24 05 be 11 ff 10 6b 
32 ff dc 05 33 48 32 36 17 69 03 43 -> 25 56 b5 06 11 ff 10 6b 
32 ff dc 05 33 48 32 36 18 82 03 43 -> 25 56 5e 09 11 ff 10 6b 
32 ff dc 05 33 48 32 36 26 71 03 43 -> 37 ad 56 25 11 ff 10 6b 
32 ff dd 05 33 48 32 36 20 72 03 43 -> 31 54 16 52 fe dc 10 6b 
32 ff df 05 33 48 32 36 42 76 03 43 -> 67 ff 65 57 dc 11 10 6b 

Issues/PRs references

fixes #15023 for real

@benpicco benpicco added Area: sys Area: System Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation labels Oct 18, 2020
@benpicco benpicco requested review from fjmolinas and maribu October 18, 2020 12:24
@benpicco benpicco added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Oct 18, 2020
@benpicco benpicco added Area: sys Area: System and removed Area: sys Area: System labels Oct 18, 2020
CPU IDs may contain very little entropy. Often only a single byte
may change between two CPUs.

To mitigate this use a somewhat 'proper' hash algorithm (I chose DJB2
hash simply because it's small) and 'extend' it to the required output
size.

For 50 STM32 CPU IDs from the grenoble site, this produces a much more
random LUID output.
@benpicco benpicco added Area: sys Area: System and removed Area: sys Area: System labels Oct 18, 2020
Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK. Using a well known and well tested hash algorithm is certainly better than something home grown.

@maribu
Copy link
Member

maribu commented Oct 19, 2020

Note that this will of course change the L2 addresses of nodes without EUI provider. This might upset some folks. IMO this is fine, as people requiring fixed L2 addresses really should add an EUI provider.

But it would be good if we stop changing LUID generation with every release.

@maribu
Copy link
Member

maribu commented Oct 19, 2020

I wonder if we should consider this an 2 ACK change because of the change of the L2 addresses.

Maybe @aabadie could throw in a second ACK, just in case? I think the improved entropy in the LUIDs generated will be especially useful for testbeds.

@benpicco
Copy link
Contributor Author

benpicco commented Oct 19, 2020

But it would be good if we stop changing LUID generation with every release.

😅 I was wondering if we should backport this to keep the LUID changes confined to one release. (last change to LUID output was #14986)

@maribu
Copy link
Member

maribu commented Oct 19, 2020

But it would be good if we stop changing LUID generation with every release.

sweat_smile I was wondering if we should backport this to keep the LUID changes confined to one release.

Strong 👍. @bergzand? If so, I think a single ACK would be sufficient, as a change in L2 address generation is imminent anyway in the upcoming release.

@benpicco benpicco merged commit fdc201a into RIOT-OS:master Oct 19, 2020
@benpicco benpicco deleted the sys/luid-tweak branch October 19, 2020 07:55
@benpicco benpicco mentioned this pull request Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: sys Area: System CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: enhancement The issue suggests enhanceable parts / The PR enhances parts of the codebase / documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gnrc: all iotlab-m3 nodes have identical short HW addresses

2 participants