I've been running around, but, here's everything I've done so far.
Step 1: We all know the limitations of 32-bit, one of which is RAM use and allocation. I attempted to circumvent this the only way I knew how - flagging to be Large Address Aware.
Conclusion: LAA alone did not resolve the issue.
Step 2: The logical step is to force better memory management so we can use the now-LAA configuration as best we can. I used two custom allocators (separately), but was unable to resolve the limitation issue.
Conclusion: Custom allocators showed no improvement over vanilla; I reverted to forcing system malloc for the next step.
Step 3: This step had varying outcomes, but none resolved the issue; I did not expect it to, I was simply curious to see how UE3 reacted. I switched between 4096MB and System Managed page files on both the drive this is installed to, as well as my local disk, in several combinations.
Conclusion: Step yielded no obvious change in issue, reverted to System Managed on all drives for the remainder of this experiment.
Step 4: Forcing arguments over the executable target was something of a last-ditch effort, one combination of which (with several of previous steps) managed to get me as close to a fix as I could have hoped to get, but I will discuss that in my final conclusion. -malloc=system was an obvious choice after the trial and error of Step 2, and was used constantly. Since the dialogue box (which I still suspect was mistaken) referred SPECIFICALLY to VIDEO MEMORY, rather than virtual memory or simply memory, I gave the force v-ram argument a shot. I started off with the obvious maximum (I have) and worked my way down -512MB an iteration. It didn't achieve anything, so I tested a few iterations along with various -maxMem=0000 arguments, usually not above the 4096MB we know we can work with; this achieved nothing, too.
Conclusion: I didn't expect these arguments to do anything, really, but giving it a shot didn't feel like time wasted.
Step 5: UE3 is very obsolete, this includes its entire architecture, obviously. I hoped that LAA alone would circumvent this - so for this step, I performed a bunch of simple compatibility troubleshooting in different combinations with all the above steps, including the arguments - I moved down to 8, 7, Vista (+ Service Packs), XP (+ Service Packs), and stopped there.
Conclusion: I wasn't getting any changes in the outcome, even in combinations with prior steps, so I abandoned this.
Final Conclusion: None of these, either alone or in combination with one another, changed the outcome of loading Desolation - although every other map/game mode tested had no issues. As Step 1 (LAA) did not resolve the issue, I believe that memory is not an issue, but instead, conclude that Desolation is too ambitious for the limitations of a 32-bit space, including its hardware limitations, and nothing more. This is probably a consequence of UE3's terrible allocation, as native memory management in the engine has a reputation for being a nightmare, requiring levels, variables and functions to be assessed in the context of the level itself to stay below the memory budget - this becomes disproportionately complex as levels contain more assets, variables, etc., but the developers know this, as they've been dealing with it for a long time.
As it stands, I don't believe there is a sure solution beyond optimizing Desolation for a 32-bit space, which would be a laborious undertaking.
How I concluded this was by setting controlled restrictions to mirror the exact, absolute limitations of 32-bit hardware (on my 64-bit system) - this achieved an outcome which did not result in the dialogue box- and crash we usually see on the second splash image, about 1/3 into the progress bar at the bottom. Instead, loading proceeded beyond this, through all splash screens, and crashed (without the "Out of video memory!" dialogue box appearing) on the final splash image, when we expect the game to be loaded and open the level as usual.
I doubt any of this will result in something tangible being achieved, but, I felt I was responsible to report back on it, as I DID CLAIM x32 was a feasible workaround.
As for the artifacting and bugs mentioned; I have not encountered any during this quick testing and troubleshooting sprint, but, if I do come across any, I'll be sure to report on that. For the time being, running the 32-bit binary has very few shortcomings, considering the alternative is not playing Renegade-X at all. The only real problem beyond Desolation (and the mentioned bugs) would be running the 32-bit binary on a VERY slow 64-bit system, but considering the average hardware these days, I don't think anyone should notice any performance loss at all.
