确定性负载（Deterload）

确定性负载（Deterload）是一个为香山生态（包括 香山处理器、 香山NEMU 和香山GEM5 ）生成确定性工作负载的框架。

Deterload is a framework for generating Deterministic Workloads for the XiangShan ecosystem (including XiangShan Processor, XiangShan NEMU, and XiangShan GEM5 ).

背景（Background）

香山是一款开源的高性能RISC-V处理器，其核心理念是敏捷开发。 香山的工作负载指运行在香山处理器上的各类程序，是开发、调试、评估、研究时不可或缺的组件。

XiangShan is an open-source high-performance RISC-V processor, built around the core concept of agile development. XiangShan's workloads refer to various programs running on XiangShan processor, which are essential components for development, debugging, evaluation, and research.

为了能更加敏捷地生成各类工作负载，我们开发了Deterload项目。 Deterload在checkpoint_scripts框架上，引入了确定性。 此外，Deterload不仅支持生成切片镜像，还计划支持香山的各类工作负载，包括非切片镜像和裸机镜像。

To enable more agile generation of various workloads, we developed the Deterload project. Deterload is based on the checkpoint_scripts framework and adds the deterministic feature. Moreover, Deterload not only supports generating checkpoint images but also plans to support various workloads for XiangShan, including non-checkpoint images and bare-metal images.

关于“确定性”（About "Deterministic"）

🤔什么是“确定性”？ 😺无论何时何地，两次构建同一个工作负载，都应该得到完全相同的结果！

🤔为什么需要“确定性”？ 😺它能让开发更敏捷。无论何时何地，你都能轻松重现bug和性能异常！

🤔如何实现“确定性”？ 😺使用确定性包管理器Nix并且控制所有随机性！

🤔What is "Deterministic"? 😺It means that whenever and wherever building the workload twice should yield the same result!

🤔Why do we need "Deterministic"? 😺It enables more agile development. You can reproduce bugs and performance anomalies anytime, anywhere, without hassle!

🤔How to achieve "Deterministic"? 😺Using the deterministic package manager Nix and controlling all possible sources of randomness!

使用方法（Usage）

Deterload由Nix驱动。 如果你尚未安装Nix，请参考Nix官方安装指南。

Deterload is powered by Nix. If you haven't installed Nix, please refer to the Nix official installation.

# 进入nix shell（推荐使用direnv自动进入nix shell）：
# Enter the nix shell (direnv is recommended for auto entering the nix shell):
nix-shell

# 用10个线程为<benchmark>生成切片，切片存于result/：
# Generate checkpoints for <benchmark> using 10 threads, saved in result/:
nom-build -A <benchmark> -j10

# 显示帮助信息：
# Display help information:
h

关于如何运行生成的切片镜像，请参考💽运行镜像。

For instructions on running the generated checkpoint images, please refer to 💽Running Images.

🧾配置（Configurations）

TODO:

Priority:

  • CLI Configurations
  • Benchmark-Specific Configurations
  • Global Configurations

🔨构建框架（Builders）

TODO:

The project uses Nix to manage dependencies and build the necessary components:

  • QEMU: Modified version of QEMU with checkpoint and profiling capabilities
  • Simpoint: Simpoint is a tool for profiling and checkpointing in XiangShan
  • OpenSBI: RISC-V OpenSBI firmware
  • Linux: Custom Linux kernel image
  • Profiling tools: Scripts and plugins for analyzing checkpoint data

📈基准测试（Benchmarks）

TODO:

TODO:

Preparing SPEC CPU2006 Source Code

Before using this project, you need to prepare the SPEC CPU2006 program source code yourself. Please follow these steps:

  1. Obtain the SPEC CPU2006 source code (we cannot provide the source code due to licensing restrictions).
  2. It is recommended to store the SPEC CPU2006 source code directory separately, not in the same location as this repository.
  3. Rename the obtained source code folder to "spec2006", like ~/workspace/spec2006.
  4. Please do not modify the SPEC CPU2006 source code, as this may cause the build to fail.
  5. Note that the spec2006/default.nix directory in this repository is different from the SPEC CPU2006 source code directory. The former can be considered as a Nix build script.

Note: Generating checkpoints may take several or more than ten hours, depending on the complexity of the benchmark.

Please note that the build process may take a considerable amount of time:

  1. First, the script will fetch and compile the RISC-V GCC toolchain, Linux kernel, QEMU, and other necessary components. This step takes approximately 1 hour.

  2. Then, it will use QEMU for profiling, SimPoint sampling, and QEMU checkpoint generation. Generating spec2006 ref input checkpoint typically requires about 10 hours.

If you want to quickly test the system, you can start by setting the input size to "test":

  1. Edit the conf.nix file
  2. Change size = xxx to size = "test"

With the test input size, the entire process should complete in about 30 minutes.

Finally, it will generate a result folder, you will get all the checkpoints in the result folder

If you want to back up some checkpoints: run

nom-build -j 30
python3 backup_checkpoints.py

It will copy checkpoints from nix path to local pwd path, named backup_XXX (timestamp). Notice: backup_XXX is about 100GB!

💽运行镜像（Running Images）

TODO:

Running on Gem5

The checkpoints generated by this repository can be run on Gem5, Nemu, and XiangShan RTL in the XiangShan repository. Here we explain the considerations for running on Gem5.

Since the checkpoints in this repository are currently single-core and without V extension (will be updated once vector extension support is stable), please configure according to the README in https://github.com/OpenXiangShan/GEM5.

Note: This repository's checkpoints by default place the checkpoint restorer code at the beginning of the checkpoint address space (refer to opensbi/default.nix). Therefore, there's no need to specify the $GCB_RESTORER environment variable; you can set it to empty.

If you encounter difftest errors during Gem5 execution, you can follow these steps:

  1. Visit the Gem5 releases page
  2. Download a stable version of NEMU
  3. Set the corresponding $GCBV_REF_SO environment variable

This approach can help resolve difftest errors and ensure proper execution of checkpoints on Gem5.

If you encounter other Gem5 runtime errors, try debugging by adding gdb --args before the gem5 command. You can also open an issue in this repository or the Gem5 repository.

Please note that some checkpoints may fail to run in Gem5. This issue is currently being addressed. However, over 90% of the checkpoints should run correctly, and the resulting scores should be generally accurate. After running all checkpoints in Gem5, you can use the following script to calculate the SPEC CPU 2006 slice scores:

https://github.com/shinezyy/gem5_data_proc

You may need to make some minor modifications to this repository, as gem5_data_proc is designed for internal checkpoints, and there are slight differences in file naming between it and the Nix checkpoints. For example, "hmmer" vs "456.hmmer".

This script can help you process the data generated by Gem5 and calculate the final scores for the SPEC CPU 2006 benchmark. Even if a few checkpoints fail to run, this script should still provide you with a fairly accurate performance assessment.