[Mulgara-general] Mulgara size limits

[Alex Hall]

Chuck Borromeo wrote:
> Hi Everyone,
>   I just wanted to check to see if anyone has figured out the issue I was encountering when I tried to load RDF/XML files larger than 98 MB.

I've been looking at this off and on since last week, but with little 
results to show.  I can tell the offending class that is filling the 
heap, and I can tell where it is being allocated, but I haven't been 
able to track down where the references to the class are not being 
released.  To add a degree of difficulty, I'm developing on a 32-bit 
Windows laptop so I can't duplicate the conditions causing the 
out-of-memory error.

However, I do have a couple of suggestions for you to try:

1.  Paul recently committed some changes that affect how objects are 
allocated.  These were meant to simplify our code and make it easier to 
track object allocation, but there's an outside chance they might 
address your memory problem.  Try building and running the latest 
revision from SVN (directions at the Mulgara Download page) and see if 
you still get out-of-memory errors.

2.  I would also try running Mulgara with assertions enabled.  Do this 
by adding the "-ea" flag to the Java VM arguments.  Scan the log file or 
console for AssertionError stack traces, and let us know if you find 
anything.

In both cases, I would also gather the heap profile, just to be sure 
that the AVLNode class is still the culprit.

Beyond these two suggestions, I can't tell how the AVLNode references 
are not being released.  The new Mulgara XA2 storage layer, planned for 
next year, will eliminate the use of the AVLNode class altogether, so 
this should not be an issue under the next-generation architecture. 
Given my limited time and resources, I just can't justify putting much 
more effort into trying to track this down when I'm unable to reproduce 
the problem myself.  If either of the previous suggestions produces new 
results, I'll be happy to take a look at them, but absent any new 
developments I'm afraid there's not much more I can do at the moment.

Regards,
Alex