Regarding Rescaling types supported by Dacapo

[trajore]

Hi

I am trying run the SNRRescaling for dacapo.

Based on my understanding the tools/optimizer.cpp starts the overall optimization strategy and then internally calls the DacapoPlannerPass that handles the core logic, if I need to experiment with the rescaling and scale management types.

what should be the approach as replacing the Proactive with SNR does not work in the DaCapoPlanner? how do add new rescaling schemes if I need to?

Thanks and regards
Tanmay

[Seonyoung-Cheon]

Hello :)

We found there are some issues for supporting SNRRescaling because SMU analysis result is not consistent during executing BypassDetection and CoverageRecorder. We will fix DaCapo to support general scale management scheme as soon. We will also add options to change scale management scheme used in DaCapo.

Thank you!

[timzsu]

Hello,

I also encountered this issue. Can you elaborate a bit more on the inconsistency? When do you expect to fix it?

Besides, I found that SNR rescaling sets waterline + 4 for "noisy operations" (i.e. sop.isNoisy() = true) and waterline for other operations. But based on my understanding, the waterline should be based on the estimated noise (e.g. waterline + std::log2(sop.getNoise())) instead of an indicator of noisy or not. Is that expected or is that a circumvention given that the ciphertext level is hard to estimate?

Thank you!

[rovelace]

We are finding the reason for the incompatibility between SNR rescaling and DaCapo.

For the questions about SNR rescaling:

1. As you mentioned, it is too complex to incorporate the ciphertext level during the waterline analysis.
   This is because the SNR rescaling uses a rescaling level, not a ciphertext level during the analysis.
   They have a circular dependency, so SNR rescaling simply assumes the maximum noise (the highest level).

2. The waterline parameter means the minimum level for rescaling, so the minimum scale for each operation should be waterline + log2(operationNoise(l)) - log2(rescalingNoise), if we can know the ciphertext level.

Hello,

I also encountered this issue. Can you elaborate a bit more on the inconsistency? When do you expect to fix it?

Besides, I found that SNR rescaling sets waterline + 4 for "noisy operations" (i.e. sop.isNoisy() = true) and waterline for other operations. But based on my understanding, the waterline should be based on the estimated noise (e.g. waterline + std::log2(sop.getNoise())) instead of an indicator of noisy or not. Is that expected or is that a circumvention given that the ciphertext level is hard to estimate?

Thank you!

[timzsu]

According to the paper and profile_SEAL_CPU.json, rotation and ctxt-ctxt multiplication have a similar noise. But in EarthOps.td, isNoisy()=true only for RotateOp but not for MulOp. So return smu.inNoisyGroup(v) ? waterline + 4 : waterline in int64_t calcWaterline(hecate::ScaleManagementUnit &smu, Value v) will return waterline + 4 for rotation and waterline for all other operations. Is that right?

According to your description, I think this calcWaterline function should be something like (the following is my attempt to implement this without using SMU)

int64_t calcWaterline(Operation *op) {
  auto heScaleInterface = dyn_cast<hecate::earth::HEScaleOpInterface>(op); 
  auto heProfInterface = dyn_cast<hecate::earth::HEProfInterface>(op); 
  if (
    llvm::isa<hecate::earth::RotateOp>(heScaleInterface) ||
    (llvm::isa<hecate::earth::MulOp>(heScaleInterface) && !heScaleInterface.isSingle())
  ) {
    auto worstOperationNoise = heProfInterface.getNoiseOf(hecate::earth::EarthDialect::levelUpperBound); 
    auto RescalingNoise = hecate::earth::EarthDialect::getNoiseTable()["earth.rescale_single"][0]; // rescaling noise is independent of level
    return waterline + std::log2(worstOperationNoise) - std::log2(RescalingNoise);
  } else {
    return waterline;
  }
}

I am wondering where the "+4" comes from in the original code, and whether my workaround is correct.

Thank you!

[rovelace]

1. The waterline should be larger than 10 in general, so the scale after the multiplication never hits the waterline for noisy multiplication, so the current implementation is fine.
   In theory, we need to return !isSingle() in isNoisy() in MulOp. I decided to skip this because assuming this is much simpler to think about correctness than allowing different waterlines for each multiplication.

2. The profiled noise in the config file is actually a "variance" of the noise. You need to change it to waterline + (std::log2(worstOperationNoise) - std::log2(RescalingNoise))/2. I assumed that (std::log2(worstOperationNoise) - std::log2(RescalingNoise))/2 = 4.

[timzsu]

That makes sense. Thank you for the clarification!

[Seonyoung-Cheon]

Hello,

I fixed and uploaded (2024.11.26) several issues to support SNR Rescaling. To fully enable these updates, you will need to rename functions in five specific files. I recommend referring to comments on commit [Add Comments in Passes Depending on ScaleManagement Scheme] (commit number is 91c97f0) for guidance on these changes.

I tested the updates using the SiLU and AveragePooling modes on ResNet-20, AlexNet, MobileNet, VGG16, and SqueezeNet, and all compiled successfully. If you encounter any issues, please let me know, and I’ll be happy to assist.

Additionally, we plan to modify the compiler structure to allow for easier adaptation if the scale management scheme changes in the future.

Thank you!

[timzsu]

Hi,

Thanks for the fast update! I encountered an error after making the changes and running hbt dacapo 40 MLP SEAL CPU. Here is the stack trace:

hecate-opt: /home/timsu/Code/Research/secure_quantization/dacapo/lib/Dialect/Earth/Analysis/ScaleManagementUnit.cpp:244: auto hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation *)::(anonymous class)::operator()(hecate::earth::HEScaleOpInterface) const: Assertion `0 && "Currently not supported"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt --dacapo --ckks-config=/home/timsu/Code/Research/secure_quantization/dacapo/profiled_SEAL_CPU.json --waterline=40 --enable-debug-printer /home/timsu/Code/Research/secure_quantization/dacapo/examples/traced/MLP.mlir --mlir-print-debuginfo --mlir-pretty-debuginfo --mlir-print-local-scope --mlir-disable-threading --mlir-timing --mlir-print-ir-after-failure -o /home/timsu/Code/Research/secure_quantization/dacapo/examples/optimized/dacapo/MLP.40.mlir
 #0 0x000055de95c216d8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x7db6d8)
 #1 0x000055de95c1f32e llvm::sys::RunSignalHandlers() (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x7d932e)
 #2 0x000055de95c21d6d SignalHandler(int) Signals.cpp:0:0
 #3 0x00007fb8ae470770 (/usr/lib/libc.so.6+0x3c770)
 #4 0x00007fb8ae4c132c (/usr/lib/libc.so.6+0x8d32c)
 #5 0x00007fb8ae4706c8 raise (/usr/lib/libc.so.6+0x3c6c8)
 #6 0x00007fb8ae4584b8 abort (/usr/lib/libc.so.6+0x244b8)
 #7 0x00007fb8ae4583dc (/usr/lib/libc.so.6+0x243dc)
 #8 0x00007fb8ae468d46 (/usr/lib/libc.so.6+0x34d46)
 #9 0x000055de956dd161 hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1::operator()(hecate::earth::HEScaleOpInterface) const /home/timsu/Code/Research/secure_quantization/dacapo/lib/Dialect/Earth/Analysis/ScaleManagementUnit.cpp:246:27
#10 0x000055de956dd0dc std::enable_if<!llvm::is_one_of<hecate::earth::HEScaleOpInterface, mlir::Operation*, mlir::Region*, mlir::Block*>::value && std::is_same<void, void>::value, void>::type mlir::detail::walk<(mlir::WalkOrder)1, mlir::ForwardIterator, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1, hecate::earth::HEScaleOpInterface, void>(mlir::Operation*, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1&&)::'lambda'(mlir::Operation*)::operator()(mlir::Operation*) const /usr/local/include/mlir/IR/Visitors.h:340:3
#11 0x000055de956dd06d void llvm::function_ref<void (mlir::Operation*)>::callback_fn<std::enable_if<!llvm::is_one_of<hecate::earth::HEScaleOpInterface, mlir::Operation*, mlir::Region*, mlir::Block*>::value && std::is_same<void, void>::value, void>::type mlir::detail::walk<(mlir::WalkOrder)1, mlir::ForwardIterator, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1, hecate::earth::HEScaleOpInterface, void>(mlir::Operation*, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1&&)::'lambda'(mlir::Operation*)>(long, mlir::Operation*) /usr/local/include/llvm/ADT/STLFunctionalExtras.h:45:5
#12 0x000055de955a19f1 llvm::function_ref<void (mlir::Operation*)>::operator()(mlir::Operation*) const /usr/local/include/llvm/ADT/STLFunctionalExtras.h:68:5
#13 0x000055de955a19ba void mlir::detail::walk<mlir::ForwardIterator>(mlir::Operation*, llvm::function_ref<void (mlir::Operation*)>, mlir::WalkOrder) /usr/local/include/mlir/IR/Visitors.h:188:1
#14 0x000055de955a1976 void mlir::detail::walk<mlir::ForwardIterator>(mlir::Operation*, llvm::function_ref<void (mlir::Operation*)>, mlir::WalkOrder) /usr/local/include/mlir/IR/Visitors.h:180:27
#15 0x000055de956dd01a std::enable_if<!llvm::is_one_of<hecate::earth::HEScaleOpInterface, mlir::Operation*, mlir::Region*, mlir::Block*>::value && std::is_same<void, void>::value, void>::type mlir::detail::walk<(mlir::WalkOrder)1, mlir::ForwardIterator, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1, hecate::earth::HEScaleOpInterface, void>(mlir::Operation*, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1&&) /usr/local/include/mlir/IR/Visitors.h:341:3
#16 0x000055de956db8dd std::enable_if<llvm::function_traits<std::decay<hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1>::type>::num_args == 1, void>::type mlir::Operation::walk<(mlir::WalkOrder)1, mlir::ForwardIterator, hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1, void>(hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*)::$_1&&) /usr/local/include/mlir/IR/Operation.h:794:5
#17 0x000055de956daf79 hecate::ScaleManagementUnit::ScaleManagementUnit(mlir::Operation*) /home/timsu/Code/Research/secure_quantization/dacapo/lib/Dialect/Earth/Analysis/ScaleManagementUnit.cpp:250:7
#18 0x000055de955c63bc mlir::detail::AnalysisModel<hecate::ScaleManagementUnit>::AnalysisModel<mlir::func::FuncOp&>(mlir::func::FuncOp&) /usr/local/include/mlir/Pass/AnalysisManager.h:120:48
#19 0x000055de955c6355 std::__detail::_MakeUniq<mlir::detail::AnalysisModel<hecate::ScaleManagementUnit>>::__single_object std::make_unique<mlir::detail::AnalysisModel<hecate::ScaleManagementUnit>, mlir::func::FuncOp&>(mlir::func::FuncOp&) /home/timsu/miniforge3/envs/dacapo/bin/../lib/gcc/x86_64-conda-linux-gnu/13.3.0/../../../gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/bits/unique_ptr.h:1070:34
#20 0x000055de955c6284 auto mlir::detail::AnalysisMap::constructAnalysis<hecate::ScaleManagementUnit, mlir::func::FuncOp, (void*)0>(mlir::AnalysisManager&, mlir::func::FuncOp) /usr/local/include/mlir/Pass/AnalysisManager.h:233:12
#21 0x000055de955c60bf hecate::ScaleManagementUnit& mlir::detail::AnalysisMap::getAnalysisImpl<hecate::ScaleManagementUnit, mlir::func::FuncOp>(mlir::PassInstrumentor*, mlir::func::FuncOp, mlir::AnalysisManager&) /usr/local/include/mlir/Pass/AnalysisManager.h:211:32
#22 0x000055de955c5fd5 std::enable_if<std::is_constructible<hecate::ScaleManagementUnit, mlir::func::FuncOp>::value || std::is_constructible<hecate::ScaleManagementUnit, mlir::func::FuncOp, mlir::AnalysisManager&>::value, hecate::ScaleManagementUnit&>::type mlir::detail::AnalysisMap::getAnalysis<hecate::ScaleManagementUnit, mlir::func::FuncOp>(mlir::PassInstrumentor*, mlir::AnalysisManager&) /usr/local/include/mlir/Pass/AnalysisManager.h:167:5
#23 0x000055de955c5f84 hecate::ScaleManagementUnit& mlir::AnalysisManager::getAnalysis<hecate::ScaleManagementUnit, mlir::func::FuncOp>() /usr/local/include/mlir/Pass/AnalysisManager.h:326:5
#24 0x000055de955c5f42 hecate::ScaleManagementUnit& mlir::Pass::getAnalysis<hecate::ScaleManagementUnit, mlir::func::FuncOp>() /usr/local/include/mlir/Pass/Pass.h:225:5
#25 0x000055de955c5cb5 hecate::ScaleManagementUnit& mlir::OperationPass<mlir::func::FuncOp>::getAnalysis<hecate::ScaleManagementUnit>() /usr/local/include/mlir/Pass/Pass.h:373:5
#26 0x000055de955cf1bf (anonymous namespace)::SNRRescalingPass::runOnOperation() /home/timsu/Code/Research/secure_quantization/dacapo/lib/Dialect/Earth/Transforms/SNRRescaling.cpp:42:9
#27 0x000055de95a59449 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x613449)
#28 0x000055de95a59cd8 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x613cd8)
#29 0x000055de95a5bee4 mlir::detail::OpToOpPassAdaptor::runOnOperationImpl(bool) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x615ee4)
#30 0x000055de95a5964b mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x61364b)
#31 0x000055de95a5d05c mlir::PassManager::run(mlir::Operation*) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x61705c)
#32 0x000055de955cacfc (anonymous namespace)::CandidateSelectionPass::runOnOperation() /home/timsu/Code/Research/secure_quantization/dacapo/lib/Dialect/Earth/Transforms/CandidateSelection.cpp:52:14
#33 0x000055de95a59449 mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x613449)
#34 0x000055de95a59cd8 mlir::detail::OpToOpPassAdaptor::runPipeline(mlir::OpPassManager&, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int, mlir::PassInstrumentor*, mlir::PassInstrumentation::PipelineParentInfo const*) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x613cd8)
#35 0x000055de95a5bee4 mlir::detail::OpToOpPassAdaptor::runOnOperationImpl(bool) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x615ee4)
#36 0x000055de95a5964b mlir::detail::OpToOpPassAdaptor::run(mlir::Pass*, mlir::Operation*, mlir::AnalysisManager, bool, unsigned int) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x61364b)
#37 0x000055de95a5d05c mlir::PassManager::run(mlir::Operation*) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x61705c)
#38 0x000055de954b89b2 performActions(llvm::raw_ostream&, std::shared_ptr<llvm::SourceMgr> const&, mlir::MLIRContext*, mlir::MlirOptMainConfig const&) MlirOptMain.cpp:0:0
#39 0x000055de954b7ae4 mlir::LogicalResult llvm::function_ref<mlir::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>::callback_fn<mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&)::$_0>(long, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&) MlirOptMain.cpp:0:0
#40 0x000055de95b82720 mlir::splitAndProcessBuffer(std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::function_ref<mlir::LogicalResult (std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, llvm::raw_ostream&)>, llvm::raw_ostream&, bool, bool) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x73c720)
#41 0x000055de954b29a9 mlir::MlirOptMain(llvm::raw_ostream&, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer>>, mlir::DialectRegistry&, mlir::MlirOptMainConfig const&) (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x6c9a9)
#42 0x000055de954991ca main /home/timsu/Code/Research/secure_quantization/dacapo/tools/optimizer.cpp:194:14
#43 0x00007fb8ae459cd0 (/usr/lib/libc.so.6+0x25cd0)
#44 0x00007fb8ae459d8a __libc_start_main (/usr/lib/libc.so.6+0x25d8a)
#45 0x000055de954983a9 _start (/home/timsu/Code/Research/secure_quantization/dacapo/build/bin/hecate-opt+0x523a9)

From the trace, I think the reason is that bootstrapping which is inserted in BootstrapPlacement is not supported by SMU by an explicit assertion. Have you encountered this problem? How to resolve it?

Thank you!

[Seonyoung-Cheon]

I compiled the hbt dacapo 40 MLP SEAL CPU and it’s working well. I also changed SMU design in relation to the bootstrapping operation. Did you get the full update for recent commits?

Thank you!

[timzsu]

I recompiled the code base again and there is no error. I guess the issue might be related to an incomplete build. Sorry for disturbing you. Thanks!