Operator Optimizations on CPUs

In the past three chapters we mainly focus on the functionality of operators, namely, how to implement the operators to function correctly in TVM. However, getting right results out is not sufficient. Operators could perform poorly even if they execute correctly.

Starting from this chapter, we will talk about the performance optimization to the operators. Specifically, we will work on operator optimizations on CPUs in this chapter, and move on to GPUs in the next chapter.

• 1. CPU Architecture
  • 1.1. Arithmetic Units
  • 1.2. Parallel Execution
  • 1.3. Memory Subsystem
  • 1.4. Summary
• 2. Function Call Overhead
  • 2.1. Execution Time Measurement
  • 2.2. The Non-Negligible Overhead
  • 2.3. Overhead of NumPy, TVM and MXNet
  • 2.4. Summary
• 3. Vector Add
  • 3.1. Setup
  • 3.2. Default Schedule
  • 3.3. Parallelization
  • 3.4. Vectorization
  • 3.5. Summary
• 4. Broadcast Add
  • 4.1. Setup
  • 4.2. Good Schedule
  • 4.3. Bad Schedule
  • 4.4. Summary
  • 4.5. Exercise
• 5. Matrix Multiplication
  • 5.1. Setup
  • 5.2. Default Schedule
  • 5.3. Reordering Axes
  • 5.4. Parallelization
  • 5.5. Summary
  • 5.6. Exercises
• 6. Improve Cache Efficiency by Blocking
  • 6.1. Blocked Matrix Multiplication
  • 6.2. Write Cache
  • 6.3. Summary
  • 6.4. Exercises
• 7. Convolution
  • 7.1. Setup
  • 7.2. Blocked Convolution
  • 7.3. Summary
  • 7.4. Exercises
• 8. Packed Convolution
  • 8.1. Packing Data and Weight
  • 8.2. Computation
  • 8.3. Schedule
  • 8.4. Summary
  • 8.5. Exercises
• 9. Depthwise Convolution
  • 9.1. Packing Data and Weight
  • 9.2. Computation
  • 9.3. Schedule
  • 9.4. Benchmark
  • 9.5. Summary
  • 9.6. Exercises
• 10. Pooling
  • 10.1. Max Pooling
  • 10.2. Avg Pooling
  • 10.3. Summary
• 11. Batch Normalization
  • 11.1. Setup
  • 11.2. Schedule
  • 11.3. Benchmarking
  • 11.4. Summary