Largest array size?

[pjc30943]

What's the largest theoretical array size in RAM for a device like the 1280n? Presumably it's not just limited by available RAM, but expected stack usage, etc.
I have a few single-type arrays of length of a few thousand elements, and width between one and four. This seems to be within the RAM capacity, but using arrays of this size results in objections from the compiler.

[mikep]

The maximum array size depends on some different factors including:

• the amount of available RAM (or extended RAM for those devices that support it)
• the device type and in particular how much RAM is needed for the registers
• the device supports native mode or has a ZBasic virtual machine
• the size of the type in the array (byte, long, structure etc)
• a native mode GCC compiler restriction of 32767 bytes as the maximum size of an array

For example a ZX1280n uses a mega1280 which requires 512 bytes for registers. If you are using a ZX1280 then you need an extra 512 bytes for the ZVM itself and additional memory for the "main" task stack. For the ZX-1280n you will also need RAM for the native mode stack. Additional RAM is also used for the heap and global variables such as additional task stacks.

When you built your code did you get an error from the compiler or a warning like this one?

Code: Select all

Warning: statically allocated variables consume all available RAM, see 'Option HeapSize' and 'Option RamSize'

I have attached an example program which creates two byte arrays each of 32,360 elements (64,720 bytes total). This seems to be about the maximum that can be used by this program. Notice the use of the Option commands to specify the RAM allocation and thereby eliminate the compiler error or warning messages about RAM usage.

In trying out code to respond to this question, I have found two potential problems for Don to resolve:

• The following does not compile. The native mode Ramstart appears to be the same value as the ZVM when it fact it should be smaller.

  Code: Select all

  Option RamSize 65536 - Register.RamStart

• Taking into account the 512 bytes of registers, the example program appears to require an extra 304 bytes (65536-64720-512) of RAM. This doesn't match the minimum suggested by the latest compiler which is only 83 bytes. If I increase the size of the arrays to say 32370 bytes then the program either hangs or continually resets the processor.

[dkinzer]

The native mode Ramstart appears to be the same value as the ZVM when it fact it should be smaller.

Confirmed and corrected. Thanks.

This doesn't match the minimum suggested by the latest compiler which is only 83 bytes.

I suspect that the 83 bytes to which you refer is just the minimum task stack size. That value does not include any RAM for user-defined variables nor for the system variables that are also required (e.g. queues for Com1, multi-tasking control data, heap management data, etc.) which is included in the RAM use data displayed after successful compilation. As a side note, with the --keep-files option the .sym file will list the data elements that comprise the RAM use. They are listed toward the end of the file with hexadecimal addresses above 00800000.

In my testing using your program on a ZX-1280n, I found that the largest array size that I could use reliably is 32329, resulting in a "RAM" used value of 64911 bytes. This suggests that the actual stack use is about 113 bytes (65536 - 64911 - 512) compared to the 79 bytes estimated by the compiler. This test case may be helpful for perfecting the stack use calculations.

[mikep]

As a side note, with the --keep-files option the .sym file will list the data elements that comprise the RAM use. They are listed toward the end of the file with hexadecimal addresses above 00800000.

I will revisit this later today and look at the SYM file.

In my testing using your program on a ZX-1280n, I found that the largest array size that I could use reliably is 32329

I forgot to mention that my testing was on a ZX-1281ne (not a ZX-1280n). I think that explains the difference.

[dkinzer]

In my testing using your program on a ZX-1280n, I found that the largest array size that I could use reliably is 32329, resulting in a "RAM" used value of 64911 bytes.

Some of the unexplained RAM use is due to using functions that return strings which requires heap space. I've modified the test program to not use any heap space by adding a procedure to output a decimal value.

The stack use of the updated program is greater than the original. The compiler calculates 117 bytes vs 79 previously. This increase required that the array size be reduced to 32321. The result is closer but there is still an unexplained 12 bytes of RAM use somewhere, of which there is a 3 byte uncertainty due to using two arrays of Integer elements.

The new version is attached.

[dkinzer]

[...] there is still an unexplained 12 bytes of RAM use somewhere

In my earlier analysis, I forgot to include two details that affect the results. To understand what is happening, refer to the diagram below which shows the memory layout for native mode.

In the test program that Mike wrote, the Heap Limit is set indirectly by specifying the size of the Main() task stack; the latter value is added to the end of the statically allocated data area to arrive at the Heap Limit. When the Main() task is being prepared to run, the AVR stack is set to the end of RAM, i.e. the heap area is temporarily used for the stack while the Main() task stack and other data structures are initialized. This fact requires a minimum size for the heap area so that the stack use during this time does not encroach on the Main() task stack. In the current release of the ZX Library, this minimum is 64 bytes so even if you specified a MainTaskStackSize that would put the HeapLimit at the end of RAM, it would be moved silently to 64 bytes from the end of RAM, making the task stack size 64 bytes less than it would otherwise appear to be.

I have made two changes to improve this situation. Firstly, the run-time minimum heap size has been reduced to 40 bytes, closer to the actual stack use during initialization but still having a comfortable margin. Secondly, the compiler was modified to include a minimum heap size in its analysis of RAM use. Thirdly, a new option was added to allow specifying a minimum heap size different from the default compile-time minimum size (but subject to the absolute run-time minimum of 40 bytes).

With the changes, the compiler now issues an error message if the size of the statically allocated data plus the main task stack size plus the compile-time minimum heap size exceeds the available RAM.

For ZX devices which use Flash memory for Program Memory, the default compile-time minimum heap size is 512 bytes, otherwise it is 256 bytes. These values are chosen to provide space for a read-modify-write buffer that is necessary for procedures like PutProgMem() as well as to provide a modicum of space for String data.

After making these changes, I can now reliably run the test program on a ZX-1280n with 32,307 elements in the arrays and a resulting task headroom of 47 bytes.