Please hold while I load your data…says no CPU ever! (Well, we’d like to think so at least but it also does hold true for the most part!)
My, how technology has changed in just such a short amount of time compared to its earlier developments. Some concepts and advancements took months or years, if not longer. The pace in which advancements continue nowadays is simply astonishing. It seems that every 18 months or so a new technological advancement is released. Knowing that we’ve gone from floppies boasting a mere 1.44 MB of storage capacity to mechanical hard disks offering ~20 GB, give or take, of storage capacity (which grew over time as well), to now where we have SSDs touting terabyte capacities for fairly cheap.
Further yet, and a main topic of this blog post, is the amount of main memory – allocable memory accessible by currently running computer software applications – available today. 256MB dynamic random access memory (DRAM) modules were once thought to be out of this world. Yet, again, here we are with typical gaming computers housing 16GBs or more of available DRAM!
Don’t Hold Me Up, Slowpoke
Our ever-beloved central processing unit (CPU) is at the heart of a computer, so to speak. Read Highlight of Computer Chipsets for more understanding of what I mean by this but in short, there is way more going on inside a computer than what meets the eyes of a user using a personal computer. The CPU can be thought of as the ultimate delegator, assigning tasks to one of its many available “grunts” to carry out its will.
Now, the CPU runs at quite the pace, it’s constantly in the “go, go, go” mentality. It doesn’t like to be throttled nor slowed down. Idle time? Who’s heard of such a thing – I can imagine a CPU thinking this while waiting for data to be fetched or processed.
With that said, let me expand on How CPUs Utilize Computer Memory by breaking down the hierarchical memory pyramid, explaining differences in types of memory, and how the different levels work together to maximize CPU utilization, offering ultra-fast performance. Here’s a visual aid from Wikipedia’s memory hierarchy reference:
Different Levels of Memory and Their Use Cases
There are four major references to the different levels of memory:
- internal memory, which are processor registers and cache;
- main memory, known as DRAM, which comes in the form of RAM modules of varying speeds and capacities;
- auxiliary memory, also referred to as secondary storage, which comes in the form of mechanical drives or other mediums of longer-term storage;
- and last is off-line storage devices, which will not be discussed in this post
The first two are considered volatile, where stored data is lost when power is cut off. On the flip side, the latter two store data persistently even after power cut off and are considered non-volatile.
Bearing in mind that the CPU is really only focused on accessing what it needs when it needs it, then carries on to the next task at hand, this requires fast speeds. RAM offers fairly rapid access times, relatively speaking, compared to secondary memory. The closer the memory bank is to the CPU, the faster the speeds generally are. One very important piece of information to point out is that there are two other levels of memory between RAM the CPU – the CPU registers and the CPU cache.
Be sure to check out this Wikipedia link for some rated speeds of the varying levels of memory.
Secondary Storage, Long-Term Data Storage
I’m going to start backwards with secondary storage, go over main memory, then finish up with internal memory. The closer to the CPU, the more complex the topic so let’s ease our way in by starting here.
Not as common nowadays on more modern computer systems, but there used to be an issue with available computer memory – specifically available main memory. Main memory back in the days weren’t as large in capacity as it is nowadays. When I got my first batch of testing hardware, the RAM modules had only 256MB of memory! If computers needed more memory than was available in RAM, certain sectors on your hard disk acted as swap space; alternatively, you could upgrade the capacity of your main memory.
The CPU automatically offsets less-used data stored in main memory to swap space. This frees up the faster RAM for other application uses and is virtually seamless to users. There is more to how a CPU offsets this data, along with how it allocates virtual memory (combination of RAM and swap space) address spaces in the first place, as well as other security considerations to bear in mind (ASLR); lookout for a future blog post about these topics since this one is fairly long as is.
Due to increased latency when working with swap space, its use is undesirable and should raise flags of concern if you’re still running into this issue today. There is no real reason to skimp on RAM, after all, RAM is what allows the CPU’s cycles to run more efficiently in executing requested program instructions. This latency may not be noticeable by average users, since internal computer speeds have increased alongside other advancements as well, but you can open your computer’s Task Manager (Windows OSes) and check just how much RAM is being used. If you do this during “sluggish” times and the available RAM is steadily saturated, then that’s a sign of needing to increase your RAM’s capacity. On the flip side, if your RAM usage is moderate but your CPU is spiked, then that application may be too much for your CPU to run smoothly.
Main Memory, DRAM versus SRAM
Before we get too into main memory, I wanted to note that there is a slight crossover when it comes to RAM. Main memory typically consists of DRAM, but there is also static RAM (SRAM) which is mainly used in processor cache and covered in the next section.
DRAM is the most common form of RAM purchased for main memory upgrades. It’s referred to as “dynamic” because of the periodic need to refresh its stored content. That is, the data stored here slowly leaks and requires being updated every few milliseconds. This makes it slower and not as desirable for use as cache memory. On the opposite side of the spectrum is SRAM. You may have guessed already but SRAM doesn’t require this refreshing aspect, which makes it faster but also more expensive. There are other use cases where SRAM is used besides processor cache where limited power consumption is imperative, such as in mobile devices.
To be clear, SRAM is used in the different layers of cache available nearer to the CPU and in power-aware use cases. It is faster but typically not upgradeable. DRAM is purchased separately and can be upgradeable to increase capacity. It is the most common form of main memory.
The more RAM you have, the more efficient concurrent applications run. Sure, a blazing CPU has its perks but can easily be bottlenecked by the capacity of your main memory. When the CPU starts a program, it pulls data from secondary storage into main memory (DRAM) for faster access to and from the CPU. This is where an application lives while in use. Once everything associated with a respective application is closed, persistent data is stored back on to disk and the CPU frees the virtual memory allocated to that application.
The key factors of speed here are made up of many, many things – the RAM modules latency, the housing bus speeds, and many others. When it comes to speed, if the use case demands it, it’s important to understand these variables. I don’t want to deviate from the main topic, however, and the first blog referenced expands on this a bit.
Internal Memory, CPU Registers and CPU Cache Levels
Lastly, internal memory. This refers to the on-board memory locations made available on and to the CPU – the processor cache and registers. Not all CPUs have on-board cache but generally Layer 1 and Layer 2 cache is embedded the CPU chip itself. I’m going to start with the cache first, since it’s further from the CPU, and then shift to registers.
Fair note, these two topics can get rather technical. My goal is to keep it simple but to at least understand what their purposes are.
CPU cache.
Cache is fairly similar to main memory, with far less capacity but far greater speeds. This can be thought of as a pool or buffer zone for the most commonly used functions and data. Instead of continually pulling / putting data from / to main memory, which is slower, the cache provides a faster access point for it – remember, the closer to the CPU the better the speeds.
There are other uses available in CPU cache depending on the CPU mode. Speculative execution, for instance, but this is also where vulnerabilities may lie. An example is Foreshadow – the Intel-specific vulnerability where attackers can exploit this feature to access parts of memory that otherwise should be prohibited. Briefly, speculative execution is essentially the CPU predicting what an application process is going to do next. If this “speculation” is correct, then the CPU saves clock cycles, else it reverts everything, but remnants remain. These remnants are what makes this feature vulnerable, however.
There are a few different levels of cache as well. Most commonly CPUs offer Layer 1 cache and Layer 2 cache. Some even offer a Layer 3 but it’s not as common. Rarer yet is Layer 4 cache. Note that each layer has its own speed ratings as well (refer to the above-linked Wikipedia web page).
Finally, let’s discuss processor registers.
Registers are generally small in capacity, matching the CPUs architecture, but offer lightning fast speeds. Most CPUs nowadays are 64-bit, which is how large the registers are. Now, there isn’t just one register, there are many, which is determined by the processor architecture as well. This is also known as Instruction Set Architecture (ISA) – see Understanding the Layers of a Computer System for more insight on the different layers making up a computer.
One way to greatly differentiate registers from other forms of memory is that registers are where the CPU actually performs logic on stored values, whereas other forms of memory merely store these values. An example of a register includes the memory data register, an intermediate register storing data awaiting immediate use. Another is the memory address register, which holds the address of the location in virtual memory that the CPU is going to access next. This StackExchange post offers additional insight clearing this up a tad.
Truthfully, in researching for this post, registers really deserve their own post. Look out for this in the future.
Conclusion
This post is intended to offer insight on how memory works inside computers. Frequently we hear of memory vulnerabilities, like stack overflows or buffer overflows. Eventually my posts will lead up to there, better understanding how they work and what we can do to prevent them, but I enjoy working from the ground up. This helps me to better understand the topic at hand.
There are obviously a lot of things to keep in mind, such as the ISA that designates the CPU’s registers to store what they do. As noted in this post, the CPU cache offers some attack surface and gave rise to CPU-oriented vulnerabilities. I wonder how, rather if, differently configured ISA factors affect how the attacks are carried out. A lot of low-level stuff is going on in this realm, and with time and research, we’ll get there.
References
Contributors, homepage.cs.uri.edu. How Computers Work: The CPU and Memory. Retrieved from https://homepage.cs.uri.edu/faculty/wolfe/book/Readings/Reading04.htm
Fairhead, H. (August 3, 2018). How Memory Works. Retrieved from https://www.i-programmer.info/babbages-bag/359-memory-2.html
Tyson, J. (August 23, 2000). How Computer Memory Works. Retrieved from https://computer.howstuffworks.com/computer-memory.htm
Wikipedia contributors. (2018, December 5). Memory hierarchy. Retrieved from https://en.wikipedia.org/wiki/Memory_hierarchy
Wikipedia contributors. (March 11, 2019). Computer memory. Retrieved from https://en.wikipedia.org/wiki/Computer_memory
Woodford, C. (September 24, 2018). Computer Memory. Retrieved from https://www.explainthatstuff.com/how-computer-memory-works.html