Herb Sutter @ NWCPP: Machine Architecture: Things Your Programming Language Neve

What people said

Recommends Pete Kirkham · 2010-06 · Stack Overflow · on “Multi-threaded random_r is slower than single threaded version”

A very simple change to space the data out in memory: struct random_data* rand_states = (struct random_data*)calloc(NTHREADS * 64, sizeof(struct random_data)); char* rand_statebufs = (char*)calloc(NTHREADS*64, PRNG_BUFSZ); pthread_t* thread_ids; int t = 0; thread_ids = (pthread_t*)calloc(NTHREADS, sizeof(pthread_t)); /* create threads */ for (t = 0; t < NTHREADS; t++) { initstate_r(random(), &rand_statebufs[t*64], PRNG_BUFSZ, &rand_states[t*64]); pthread_create(&thread_ids[t], NULL, &thread_run, &rand_states[t*64]); } results in a much faster running time on my dual-core machine. This would confirm the suspicion it was meant to test - that you are mutating values on the same cache line in tw

Recommends Branko Dimitrijevic · 2011-09 · Stack Overflow · on “which is faster? "vector of structs" or "a number of vectors"?”

CPUs love prefetching. If you are going to linearly traverse your data in the following pattern... abcabcacb... ...then you are better off (performance-wise) with solution #1. If you are going to access them as: aaa...bbb..ccc... ...then go for solution #2. However , if you are not going to do a linear traversal or if you did not actually benchmark your code and concluded that you really need to squeeze every last drop of performance out of this piece of code, do your maintainability a favor and stick with Solution #1. --- EDIT --- In a multi-threaded environment, the physical layout of data may lead to false sharing [LINK] . Essentially, keeping too close the pieces of data that are concurr

Recommends amdn · 2012-08 · Stack Overflow · on “How to get better at optimization?”

Here are some links that might be helpful in general on the subject of memory optimizations What Every Programmer Should Know About Memory by Ulrich Drepper [LINK] Herb Sutter: The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software [LINK] Slides: Herb Sutter: Machine Architecture (Things Your Programming Language Never Told You) [LINK] Video: Herb Sutter @ NWCPP: Machine Architecture: Things Your Programming Language Never Told You [this video] The microarchitecture of Intel, AMD and VIA CPUs An optimization guide for assembly programmers and compiler makers, by Agner Fog [LINK]

Recommends Alois Kraus · 2013-01 · Stack Overflow · on “Determine Values AND/OR Address of Values in CPU Cache”

To optimize for one specific CPU cache size is usually in vain since this optimization will break when your assumptions about the CPU cache sizes are wrong when you execute on a different CPU. But there is a way out there. You should optimize for certain access patterns to allow the CPU to easily predict what memory locations should be read next (the most obvious one is a linear increasing read). To be able to fully utilize a CPU you should read about cache oblivious [LINK] algorithms where most of them follow a divide and conquer strategy where a problem is divided into sub parts to a certain extent until all memory accesses fit completly into the CPU cache. It is also noteworthy to mention

Recommends Jacob Parker · 2013-03 · Stack Overflow · on “Memory & file loading c++?”

Your operating system will cache files in unused memory. See wikipedia [LINK] . This will disappear after reboots, or if you use a lot of memory, or if you open many more files. The details vary based on your operating system. For a broader overview on modern computer architecture, this talk [this video] by Herb Sutter is good.

Recommends Marc Claesen · 2014-01 · Stack Overflow · on “Cache split levels vs one whole cache?”

Because fast cache is expensive (L1 is faster than L2 is faster than L3 ...). By splitting it up in layers you can expand the size of the cache at a relatively low cost. Even the " slow " cache will be orders of magnitude faster than retrieving data from RAM memory (or, even worse, HDD). Efficient heuristics exist to guess what data should be in which level of cache. Some more information is available in the answers here: What is "cache-friendly" code? [LINK] The following video about Herb Sutter talking about caching [this video] contains a whole lot of very useful information about caches (it becomes particularly relevant for this question around the 12 min mark).

Recommends Ali · 2014-02 · Stack Overflow · on “Is multiplication faster than array access?”

Konrad Rudolph commented on the issues [LINK] with the benchmarking. So I am ignoring the benchmark and focus on the question: Is multiplication faster than array access? Yes, it is very likely. It used to be the other way around 20 or 30 years ago. Roughly speaking, you can do an integer multiplication in 3 cycles (pessimistic, if you don't get vector instructions), and a memory access costs you 4 cycles if you get it straight from the L1 cache but it is straight downhill from there. For reference, see Latencies and throughput in Appendix C of the Intel 64 and IA-32 Architectures Optimization Reference Manual [LINK] Approximate cost to access various caches and main memory? [LINK] Herb Sutt

Recommends Rob11311 · 2014-05 · Stack Overflow · on “Improving data locality of reference when storing pointers to objects”

Some measurements in a chart, from example worked in the Pitfalls Presentation, showing effect of standard OO, locality with contiguous storage and then additional improvements reorganising data structures flattening them further and prefetching - [LINK] Pitfalls of OO Programming presentation, that is written for games programmers which I found extremely clear, explains the issue with traditional objects using many pointers, on modern systems "Pitfalls of OO" - [LINK] Furthermore it's not just about cache prefetch & locality on a server/desktop system, but also the possibility of page faults (may be to/from disk) which can make scattered fragmented object allocations very inefficient. Secon

Cites Andy Prowl · 2013-01 · Stack Overflow · on “Does the WKWYL optimization in make_shared<>() introduce a penalty for some multithreading applications?”

A few days ago I happened to watch this very interesting presentation [LINK] by Stephan T. Lavavej, which mentions the " We Know Where You Live " optimization (sorry for using the acronym in the question title, SO warned me the question might have been closed otherwise), and this beautiful one [this video] by Herb Sutter on machine architecture. Briefly, the " We Know Where You Live " optimization consists in placing the reference counters on the same memory block as the object which make_shared is creating, thus resulting in one single memory allocation rather than two and making shared_ptr more compact. ( Note to readers: This was an incorrect interpretation of WKWYL) After summing up what I le

Cites Marc Claesen · 2013-05 · Stack Overflow · on “What does it mean for code to be "cache-friendly"?”

Preliminaries On modern computers, only the lowest level memory structures (the registers ) can move data around in single clock cycles. However, registers are very expensive and most computer cores have less than a few dozen registers. At the other end of the memory spectrum ( DRAM ), the memory is very cheap (i.e. literally millions of times cheaper ) but takes hundreds of cycles after a request to receive the data. To bridge this gap between super fast and expensive and super slow and cheap are the cache memories , named L1, L2, L3 in decreasing speed and cost. The idea is that most of the executing code will be hitting a small set of variables often, and the rest (a much larger set of va