Dude, Where's My 4 Gigabytes of RAM?

blog
21

Jeff:

" Exactly where is that magical PAE memory supposed to appear?

It’ll only “appear” in apps that explicitly code to support the AWE (address windowing extensions) API, which makes use of the PAE extensions in the CPU. I don’t think the base OS does this, but applications might.
"

No - PAE increases the available pool of memory in total. AWE isn’t directly connected with it, though the two go together from necessity for super-huge apps on 32-bit.

a href="http://blogs.technet.com/tristank/archive/2006/05/27/430507.aspx"http://blogs.technet.com/tristank/archive/2006/05/27/430507.aspx/a

22

Man, are you sure 4GB will be enough at all, for Windows Vista?

23

“Virtual memory, in a nutshell, refers to the method of an OS presenting non-contiguous memory as if it were contiguous.”

Of course in practice, on 32bit Windows, you still have the problem of actually getting a contiguous memory block greater than ~1.5Gb.

Windows maps drivers and suchlike at the bottom of the virtual memory for each process, then at around 2Gb, you’ll find it places system dll’s! So your poor process can never actually map a contiguous block of memory 1.5gb, even though it can access 1.5Gb + memory from the upper limit (~1Gb) using the /3Gb switch.

A real pain when you’re constrained to 32Bit Windows and attempting to create memory hungry processes used memory mapped files for shared mem.

http://msdn2.microsoft.com/en-gb/library/ms810613.aspx

24

Jamesm:
“Of course in practice, on 32bit Windows, you still have the problem of actually getting a contiguous memory block greater than ~1.5Gb.”

I didn’t say the OS had to be any good at it… :slight_smile:

25

Nice article, again. You mentioned BIOSes that move the mapped PCI memory to a very high address. Actually the Asus BIOS I have works the other way round: the RAM between 2 GB and 4 GB is remapped to 4+ GB.

I’ve recently upgraded to 4 GB because I’m dumb and curious and have too much money. Now I found that I have to enable this setting when running Memtest86 1.70 so that it can see all 4 GB. It will test them in to chunks: 116KB (or so) to 2 GB, and 4 GB to 6 GB, because that’s where the BIOS has mapped the second 2 gigs.

But for booting Windows (XP 32-bit) I have to disable memory remapping in order to get at least a measly 3 GB (the 8800’s video RAM takes up all the rest!).

26

Here’s Microsoft’s guidance on what operating systems support the 36-bit PAE extensions, and what the memory limits are for each:

http://www.microsoft.com/whdc/system/platform/server/PAE/pae_os.mspx

It hasn’t been updated for Vista yet, but here are the physical memory limits from Thurrott:

Starter: 256 MB (!)
Home: 8 GB
Home Premium: 16 GB

every other edition of Vista will use as much physical memory as the hardware supports.

27

AWE = EMS for the early Noughties :slight_smile:

28

no… This is a real system: [snipped]

cat /proc/meminfo

[ 16 GB RAM ]

cat /proc/cpuinfo

[ 4 Opteron 2.4 GHz dual-core CPUs ]

29

Kris, I think how did you correlate the limitation of 32bit addressing with DOS compatibility?

30

So if I really want everything I get out of my memory, don’t I want 3x 1GB sticks, AND 1x 512MB stick?

then you loose the dual channel benefit.

18,446,744,073,709,551K of RAM ought to be enough for anybody.

anyway since windows has a limited life…(vista is like 5 year support)

31

So… if we buy x64 hardware, will a 32 bit OS still see all 4 GB, or do you have to have a 64 bit OS as well?

I won’t be going to a new OS anytime soon ( I want Vista, and don’t have the money for it yet). But my church is looking at upgrading to 4GB in the recording computer…

32

haha great jeff! very interesting this article!
its silly but i never thought about a limit like that in the modern computer architecture.
i just hope to live until the day that i can say: dude!! where are my 19 EB (aka Exabyte)??? hahah

ps. hey jeff i would to suggest you the topic of one of your next articles, cause i was thinking if exist a diet, or better like position to improve productivity for programmers cause i am getting pretty fat haha :D. sorry i go always off topic. Regards.

33

Actually, the physical processor does have, to this day, DOS compatibility in microcode and bios. (Maybe in stub form, running DOS probably won’t actually work anymore with new hardware anyway, I don’t know if anyone’s even tried it in years.) When they say x86 ISA is ‘legacy baggage’ they don’t just mean the silly register layout and instruction set.

The northbridge or HT bridge controls all access to memory, both DRAM and PCI chips (and bios and other device junk), and it’s the one that initializes PCI devices into certain slots. That’s why the cpu, and thus the OS, is incapable of seeing through it.

34

That, my friends, is why Apple does not offer 4GB of RAM in laptops. Which makes me wonder how all the other manufacturers explain to the rich idiots who splurged their kilobucks on their loaded Lenovos, Toshibas and HPs, why they can’t see all of their expensive RAM.

35

“The user-mode virtual address space in 64-bit Windows is a mere 8 terabytes… It’s the final solution, at least for the lifetime of everyone reading this blog post today.”

You think? In the early 90’s, I had a computer with 2MB of RAM. Today (17 years later), it is feasible to have 4GB. That’s 2,000 times more memory. If the same trend continues, we could have 8TB by 2024. Certainly within most of our lifetimes.

36

If and when Adobe’s new InDesign Suite goes to 64bit, then I will certainly buy a PC to take full advantage of those available features. Until then I will just continue to run at 2GB RAM.

37

Kris, you are embarrassing yourself with such comments, so the 1MB “DOS compatibility” is what makes the 4G limit ? LOL.

38

Ya know, I thought it was common to be using the PAE kernel these days, at least on recent NX-capable hardware.

You haven’t disabled DEP, perchance?

http://msdn2.microsoft.com/en-us/library/ms791508.aspx

I’m running X64 at home and at work, so can’t readily check :slight_smile:
Mental note to look that up sometime.

39

Jeff, I think you should edit this article, because your recommendation:

“The proper solution to this whole conundrum is to use a 64-bit operating system”

is misleading, and may well cause people serious grief. There are loads of systems out there with 64-bit CPUs that only have a 32-bit physical address range. For example, the vast majority of laptops only have a 32-bit physical address range, even though many are now available with 64-bit CPUs. And plenty of desktop systems have the same issue.

People following your advice are liable to put themselves to the inconvenience of switching to a 64-bit OS only to find that it doesn’t actually solve the problem - they’re still limited to 3.5GB of RAM, but now they have the additional problems that half their devices are no longer properly supported, and their OS now has a bigger memory footprint thus reducing the amount of memory available for doing useful work… Most people will not regard this as an improvement, and they could reasonably be annoyed with your advice.

The bottom line is this: a lot of chipsets only use 32 address bits - the higher order address bits on the processor are not wired up to anything more than terminating resistors. Turning on /PAE or switching to 64-bit won’t help you if you have such a chipset because the CPU doesn’t have a way of asking for stuff above the 4GB mark. 64-bit will let you get more than 2GB of virtual address space in a process, but you’ll still be stuck with 3.5GB or so of physical memory.

So if I were you I’d qualify your recommendation before you get a horde of angry people beating a path to your door… Make it clear that you need to have a suitable chipset. Moreover, if you do have a suitable chipset you probably don’t actually need to go 64-bit. /PAE may actually be a better option in reality, given today’s less than total support for 64-bit. PAE may be a hack, but who cares? The OS deals with it; it’s no uglier than the fact that my machine has 4GB of virtual memory (my physical memory + page file is 4GB, on my 32-bit system). 64-bit isn’t about breaking through the 4GB barrier of physical addressing - it’s only really useful if you’ve hit virtual address space limits, which is really a completely different class of problem than the one you open this blog entry with.

(Although if you need more than 64GB of memory, I believe 64-bit is necessary. PAE only goes up to 64GB of physical address space.)

In short: the 3.5GB problem is usually a chipset thing, not a 32 vs 64 thing. Unless you have individual processes requiring more than 3GB of memory, /PAE is often a better solution. (With most people I know who need this much memory, it’s because they are hosting a couple of VMs, and /PAE is just fine for that.) And if /PAE doesn’t work on your system, switching to 64-bit probably won’t either.

40

Thanks for the tip…I am in the process of planning my new rig and was about to make that mistake. The problem is that I can’t afford to lose all of the programs that don’t run well (or at all) on the 64 bit OS. And I hear that drivers for Vista 64 are a nightmare so I guess 3 gigs of ram it is.

John
http://www.monomachines.com