Skip to content

Conversation

@zcfh
Copy link

@zcfh zcfh commented Jul 12, 2024

probem

Recently, commit 623c777 upgraded perf_data_converter. There are some changes in the PerfParser::MapBranchStack. In the new version, even if the mapping of from_ip fails, the mapping process for to_ip will still proceed. This may lead to a situation where from.offset_ equals the initial value 0 when from_ip mapping fails, and to.offset_ equals the actual offset when to_ip mapping succeeds. In addition, in my test, these unmatched IPs are very large kernel stack addresses such as 0xffffffffa9400ac7. This inconsistency may cause the subsequent check to erroneously warn about "Bogus LBR data". A large number of such warnings during use are more likely to cause confusion and need to be filtered in advance.
Here is my solution. If from does not match, continue in advance, probably not the best solution.

test case output

 mapped
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20240712 19:35:47.081317 388180 sample_reader.cc:343] Bogus LBR data (duplicated top entry)
W20240712 19:35:47.081511 388180 sample_reader.cc:343] Bogus LBR data (duplicated top entry)
W20240712 19:35:47.081524 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=2
W20240712 19:35:47.081542 388180 sample_reader.cc:343] Bogus LBR data (duplicated top entry)
W20240712 19:35:47.081550 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=2
W20240712 19:35:47.081559 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=3
W20240712 19:35:47.081569 388180 sample_reader.cc:343] Bogus LBR data (duplicated top entry)
W20240712 19:35:47.081578 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=2
W20240712 19:35:47.081586 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=3
W20240712 19:35:47.081594 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=4
W20240712 19:35:47.081605 388180 sample_reader.cc:343] Bogus LBR data (duplicated top entry)
W20240712 19:35:47.081614 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=2
W20240712 19:35:47.081622 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=3
W20240712 19:35:47.081630 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=4
W20240712 19:35:47.081638 388180 sample_reader.cc:352] Bogus LBR data (range is negative): 1895->0 index=5
I20240712 19:35:47.085575 388180 symbol_map.cc:477] Adding loadable exec segment: offset=0 vaddr=200000

test case

set -e
echo "**** init test file"
test_file=test_warning.cc
cat << EOF >${test_file}
#include <vector>
#include <stdio.h>
#include <iostream>
#include <fstream>
#include <string>
using namespace std;

class Base {
  public:
    long long v_;
    Base(long long v) : v_(v) {}
    virtual void test_func(long long) = 0;
    virtual ~Base() = default;
};


class Child1 : public Base {
  public:
    Child1(long long v) : Base(v) {}
    void test_func(long long v) override {
        v_ += v + 1;
    }
};


class Child2 : public Base {
  public:
    Child2(long long v) : Base(v) {}
    void test_func(long long v) override {
        v_ += v + 2;
    }
};

class Child3 : public Base {
  public:
    Child3(long long v) : Base(v) {}
    void test_func(long long v) override {
        v_ += v + 3;
    }
};



int class_type = 1;

void test_pgo(Base *p, const vector<long long> &arry) {
    if (arry.empty())
        return;
    long long end = arry.back(), idx = 0;
    for (long long i = 0; i < end; ++i) {
        if (i > arry[idx])
            ++idx;
        long long add_val = 0;
        long long mod = arry[idx] % 4;
        if (mod == 3) {
            add_val = i;
        } else  if (mod == 1) {
            add_val = -i;
        } else if (mod == 0) {
            add_val = i / 2;
        } else if (mod == 2) {
            add_val = i * 2;
        }
        p->test_func(add_val);
    }
    printf("%lld\n", p->v_);
}


vector<long long> read_data() {
    std::ifstream file("data.txt");
    vector<long long> ret;
    std::string line;
    while (std::getline(file, line)) {
        std::cout << line << std::endl; 
        ret.push_back(atoll(line.c_str()));
    }
    file.close();
    
    return ret;
}

void test() {
    Base *p = nullptr;
    switch (class_type) {
      case 1:
        p = new Child1(0);
        break;
      case 2:
        p = new Child2(0);
        break;
      case 3:
        p = new Child3(0);
        break;
      default:
        p = new Child1(0);
    }
    vector<long long> arry = read_data();
    for (int i = 0; i < 1; ++i)
      test_pgo(p, arry);
}

int main(int argc, char** argv) {
    printf("------begin\n");
    if (argc > 1) {
        class_type = atoi(argv[1]);
    }
    test();
}
EOF


cat << EOF >data.txt
3
3000
300000
500000000
2000000000
5000000000
EOF

clang="/home/distcc/clang-11.1.0/bin/clang++"
create_llvm_prof="build/create_llvm_prof"
cppflags="-O2  -fuse-ld=lld -fPIC  -ggdb -Wl,--build-id -funique-internal-linkage-names -fdebug-info-for-profiling"
ldflags="-Wl,--no-rosegment "

# build clang-11
echo "**** build clang-11 test"
set -x
${clang} ${test_file} -o test_clang_pgo_before ${cppflags} ${ldflags}
# The test program is relatively simple. Sometimes perf cannot collect samples and needs to be executed several times.

size_threshold=$((1 * 1024 * 512)) # 512K in bytes

max_attempts=10
attempt=1

while [ $attempt -le $max_attempts ]
do
    echo "Attempt #$attempt: Running perf..."
    perf record -F2000 -e cycles:up -j any,u -o clang_perf.data ./test_clang_pgo_before
    actual_size=$(stat --format=%s "clang_perf.data")
    if [ $actual_size -lt $size_threshold ]; then
        echo "Output file size is less than 512KB. Retrying..."
        ((attempt++))
    else
        echo "Output file size is sufficient."
        break
    fi
done

${create_llvm_prof} --profile clang_perf.data --binary test_clang_pgo_before  --out=clang.llvm.prof

@google-cla
Copy link

google-cla bot commented Jul 12, 2024

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@zcfh
Copy link
Author

zcfh commented Jul 15, 2024

Could you take a look at this question?
@shenhanc78

Recently, commit 623c777 upgraded perf_data_converter. There are some changes
in the `PerfParser::MapBranchStack`. In the new version, even if the mapping
of from_ip fails, the mapping process for to_ip will still proceed. This may
lead to a situation where from.offset_ equals the initial value 0 when from_ip
mapping fails, and to.offset_ equals the actual offset when to_ip mapping
succeeds. This inconsistency may cause the subsequent check to erroneously
warn about 'Bogus LBR data'. A large number of such warnings during use are
more likely to cause confusion and need to be filtered in advance.
@zcfh zcfh force-pushed the ignore_warning branch from 9817616 to 35a32aa Compare July 29, 2024 12:38
@zcfh zcfh mentioned this pull request Jul 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant