Instruction vs data cache usage

Question

Say I've got a cache memory where instruction and data have different cache memories ("Harvard architecture"). Which cache, instruction or data, is used most often? I mean "most often" as in time, not amount of data since data memory might be used "more" in terms of amount of data while instruction cache might be used "more often" especially depending on the program.

Are there different answers a) in general and b) for a specific program?

• dagnelies: I guess this is widely program specific. Why not simply take a few assembler example programs and analyze/count their instructions/memory accesses?

Answer 1

Well, there are instructions that don't access the data cache, but it's impossible to access the data cache without using an instruction, so by definition the instruction cache is used more often.

If you're talking about which one has fewer cache misses, that's going to be highly program specific. A tight loop that accesses a gigabyte of memory will have zero instruction misses and a ton of data misses. A large program consisting entirely of register operations will have a ton of instruction misses and no data misses.

• Billy ONeal: +1 -- but yes and no. If an instruction is repeated many times inside the decode hardware then the instruction cache will not be touched, but there will potentially be several "touches" against the data cache. (This is more common in CISC machines than RISC machines, of course)

Answer 2

This is going to be entirely program specific. On the one hand, imagine a program that does nothing but a bunch of jumps around; which is exactly the size of the instruction cache. Zero data cache use, heaviest possible instruction cache use. Now imagine a program that does nothing but a bunch of memcpy operations everywhere. Minimal use of the instruction cache (assuming the memcpy gets pipelined); heaviest possible use of the data cache.

Of course, real software falls somewhere in the middle. But it still varies from program to program.

If recent CPU designs from the "Big Boys" are any indication, they typically use L1 caches of the same size for data and instruction caches; and have a large, shared, L2 cache (and occasionally an L3 cache).

Answer 3

Kind of depends on how you define "used most often". In today's modern multi-core architectures, cache coherence requires the data cache to watch the address bus and see if any other cores are modifying the same data and, if so, update their copies. So data caches may be updated (used) even if the core to which they're attached is idle.