CoreMark Technical Fundamentals
What is CoreMark?
As its name implies, CoreMark is a simple, yet sophisticated, benchmark that is designed specifically to test the functionality of a processor core. CoreMark is not system dependent, therefore it functions the same regardless of the platform (e.g. big/little endian, high-end or low-end processor). Running CoreMark produces a single-number score allowing users to make quick comparisons between processors.
Why is CoreMark needed now?
Simply put, it’s time to eliminate the use of Dhrystone. Processors and associated systems are getting increasingly complex requiring increasingly complex benchmarks to analyze. The current and future EEMBC benchmarks are aimed at specific embedded market segments and are very successful at approximating real-world performance of embedded devices. However, there is also a need for a widely-available, generic benchmark specifically targeted at the processor core.
What are the basic features and characteristics of CoreMark?
CoreMark is comprised of small and easy to understand ANSI C code with a realistic mixture of read/write operations, integer operations, and control operations. CoreMark has a total binary size of no more then 16K using gcc on an x86 machine (this small size makes it more convenient to run using simulation tools).
Unlike EEMBC’s primary benchmark suites, CoreMark is not based on any real application, but the workload is actually comprised of several commonly used algorithms that include matrix manipulation (to allow for the use of MAC and common math operations), linked list manipulation (to exercise the common use of pointers), state machine operation (common use of data dependent branches), and Cyclic Redundancy Check (CRC is a very common function used in embedded).
What functionality of a processor core does CoreMark actually test?
Due to the focused nature of CoreMark, it will be useful for testing a processor’s pipeline operation, memory (or cache if any) access, and handling of integer operations.
NOTE: To thoroughly comprehend the capability of a processor’s core and an associated compiler, EEMBC recommends that you also utilize the EEMBC suites including AutoBench, DENBench, Networking, OABench, TeleBench, and (if applicable) MultiBench,.
How does CoreMark's small size help me test a platform’s memory subsystem?
The small size of CoreMark allows it to easily fit in a processor’s cache. One of the goals of CoreMark is to make it suitable for testing on a very wide range of processors. Some low-end microcontrollers don’t even have caches, let alone large amounts of system memory. Testing a platform’s memory subsystem requires more substantial benchmarks, such as those EEMBC provides with its many different benchmark suites that are aimed at testing the entire processor platform.
Is it possible for a compiler to optimize away the CoreMark workload?
While compilers may find more efficient ways of processing the workloads contained in CoreMark, the work itself cannot be optimized away. Furthermore, CoreMark does not use special libraries that can be artificially manipulated, and it was specifically designed not to make any library calls from within the timed portion of the benchmark.
What are the similarities and differences between CoreMark and Dhrystone?
Both CoreMark and Dhrystone are free, small, easily portable, available for practically any processor, and display a single number benchmark score. Despite these similarities, Dhrystone has several serious drawbacks which can be summarized into several categories:
Power and Energy Measurement
Similar to EEMBC’s application benchmark suites, does CoreMark support EnergyBench?
Power consumption is a very important parameter for analyzing a processor’s characteristics.
Yes, the EnergyBench enabled version of CoreMark is available to all EEMBC members and it may also be licensed separately by non-member companies. EEMBC® EnergyBench™ provides data on the amount of energy a processor consumes while running EEMBC's performance benchmarks. EEMBC has standardized on the use of National Instruments' LabVIEW graphical development environment and data acquisition hardware to implement EnergyBench.
Will CoreMark work for analyzing multicore performance?
You can run multiple instantiations of CoreMark (a rate-type benchmark) and the source code includes a flag that will allow you to compile in code that supports a ‘create and destroy’ context functionality (where a context can be either threads or processes with shared memory or with sockets). Regardless, it’s expected that adding more cores will yield an almost linear performance behavior. For analyzing multicore performance, we highly recommend that you use the EEMBC MultiBench software.
Comparing CoreMark to EEMBC Application Suites
How does CoreMark differ from previous EEMBC benchmarks?
Current (and future) EEMBC benchmarks will typically target specific application areas, whereas CoreMark is a generic and simple benchmark, producing a single number and completely focusing on a core’s behavior. Over the course of EEMBC’s life, the benchmarks have been steadily increasing in size and complexity and are better suited for determining how a processor will actually behave in a modern application. On the other hand, CoreMark’s primary goals are simplicity and providing a method for testing only a processor’s core features.
Is CoreMark similar to other EEMBC benchmarks?
All EEMBC benchmarks, including CoreMark, are defined by committee and are thoroughly tested on a wide variety of platforms prior to release. Also, similar to other EEMBC benchmarks, the portability of CoreMark is also a key requirement. Unlike the primary EEMBC benchmarks, CoreMark is strictly focused on analyzing the core’s performance.
Under what circumstances should I use EEMBC's full-featured benchmarks rather than CoreMark?
CoreMark does not replace or supersede any of EEMBC’s application suites. CoreMark provides a very basic starting point for measuring a processor’s core performance, and is also applicable for a wide range of processors ranging from 8-bit microcontrollers to high-end 32-bit devices and architectures. While the small size of CoreMark makes it a practical test for processor cores, EEMBC’s full-featured application benchmarks are much better suited for testing a processor’s capability in a real application. Furthermore, processors are becoming increasingly complex and one core-based benchmark is insufficient for a comprehensive analysis.
How long does it take to set up and run CoreMark
CoreMark is easily portable to a wide range of microcontrollers and microprocessors. Therefore, if you have support for the basic functionality (i.e. clock(), GetSystemTimeAsFileTime(), lock_gettime()), it could take as little as 5 minutes. Otherwise, you need to specify how timing is to be accomplished in the porting layer. Furthermore, CoreMark includes full documentation to support your porting efforts and comprehend the run guidelines.
Under the standard EEMBC rules, all scores must be certified prior to publication. Why is this requirement changed for distribution of CoreMark scores?
The simplicity of CoreMark (i.e. on par with Dhrystone) makes it more suitable for self-verification of results and helps eliminate many common benchmarking errors encountered with the use of Dhrystone. Furthermore, the wide accessibility of this benchmark makes it easy to accomplish peer reviews and help ensure benchmark score reliability. However, to guarantee the repeatability and quality of the CoreMark scores, EEMBC also provide certifications on CoreMark for EEMBC members.
How does EEMBC publish scores for CoreMark?
EEMBC has set up a specific website (www.coremark.org) for the distribution of CoreMark and the publication of scores. All registered CoreMark users are encouraged to enter their scores and platform configurations on this website.