[Mulgara-general] Mulgara size limits

[Chuck Borromeo]

Hello,
  OK I swapped the parser.  That didn't work.  The parser doesn't run out of memory, but it appears that Mulgara runs out of memory.  I ran the Mulgara server with the following settings:

java -d64 -Xmx4096m -jar mulgara-2.0-alpha.jar --serverconfig file:mulgara-config.xml

Here is the stack trace:

Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.util.HashMap.<init>(HashMap.java:181)
        at java.util.LinkedHashMap.<init>(LinkedHashMap.java:210)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl$Cache.<init>(XAStringPoolImpl.java:3194)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl$SPO2GNCache.getCache(XAStringPoolImpl.java:3054)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl$SPO2GNCache.put(XAStringPoolImpl.java:3069)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl$SPO2GNCache.put(XAStringPoolImpl.java:3062)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl$Phase.findGNode(XAStringPoolImpl.java:1760)
        at org.mulgara.store.stringpool.xa.XAStringPoolImpl.findGNode(XAStringPoolImpl.java:387)
        at org.mulgara.resolver.StringPoolSession.localizeSPObject(StringPoolSession.java:506)
        at org.mulgara.resolver.StringPoolSession.localize(StringPoolSession.java:417)
        at org.mulgara.resolver.StringPoolSession.localizePersistent(StringPoolSession.java:200)
        at org.mulgara.resolver.store.StatementStoreResolver.localizePersistent(StatementStoreResolver.java:457)
        at org.mulgara.resolver.PersistentResolverSession.localize(PersistentResolverSession.java:97)
        at org.mulgara.content.rio.RDFXMLStatements.getColumnValue(RDFXMLStatements.java:323)
        at org.mulgara.content.rio.RDFXMLStatements.getObject(RDFXMLStatements.java:187)
        at org.mulgara.resolver.store.StatementStoreResolver.modifyModel(StatementStoreResolver.java:362)
        at org.mulgara.resolver.InternalResolver.modifyModel(InternalResolver.java:169)
        at org.mulgara.resolver.SetModelOperation.execute(SetModelOperation.java:166)
        at org.mulgara.resolver.MulgaraInternalTransaction.execute(MulgaraInternalTransaction.java:577)
        at org.mulgara.resolver.DatabaseSession.execute(DatabaseSession.java:617)
        at org.mulgara.resolver.DatabaseSession.setModel(DatabaseSession.java:479)
        at org.mulgara.server.rmi.SessionWrapperRemoteSession.setModel(SessionWrapperRemoteSession.java:120)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:585)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:294)
        at sun.rmi.transport.Transport$1.run(Transport.java:153)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:149)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:466)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:707)


--- On Tue, 7/1/08, Paul Gearon <gearon at ieee.org> wrote:

> From: Paul Gearon <gearon at ieee.org>
> Subject: Re: [Mulgara-general] Mulgara size limits
> To: "Mulgara General" <mulgara-general at mulgara.org>
> Date: Tuesday, July 1, 2008, 1:18 PM
> On Tue, Jul 1, 2008 at 9:30 AM, Seaborne, Andy
> <andy.seaborne at hp.com> wrote:
> >
> >> [snip]
> >>
> >> The out-of-memory exception is being thrown in the
> Jena ARP parser,
> >> which we use for parsing RDF/XML.  Looking quickly
> at the Jena source, I
> >> see that ARP keeps a bunch of stuff in memory,
> namely blank node
> >> mappings, so it doesn't surprise me that it
> runs out of memory for large
> >> files.
> >
> > If it's the check for illegal reuse of bNodes ids,
> then may be it's an old version of ARP? Nowadays it
> issues a warning about being unable to track illegal reuse
> of bNode ids across the whole file and stops that checking
> while continuing parsing.  If you're using an old
> version, then if it's using an old version of Xerces,
> that might also be a factor.
> 
> I'm pretty sure that no one has updated this in a long
> time, so it's
> pretty much guaranteed to be an old version of ARP. Xerces
> got updated
> about 2 years ago, but I can't recall if it's
> happened again since
> then.
> 
> I didn't realize that the ARP code was keeping blank
> node mappings in
> memory. Whoever implemented the content handler with ARP
> must not have
> known this, as the content handler maintains it's own
> mappings. These
> mappings are file-based, so they shouldn't have memory
> restrictions,
> though I can't comment on their speed, as I don't
> know how well they
> cache in memory.
> 
> Alex's comment about RIO reminds me.... wasn't this
> parser supposed to
> replace the ARP-based one?
> 
> Paul
> _______________________________________________
> Mulgara-general mailing list
> Mulgara-general at mulgara.org
> http://mulgara.org/mailman/listinfo/mulgara-general